SendGrid

SendGrid feature snapshot

A high-level look at Stitch's SendGrid (v1) integration, including release status, useful links, and the features supported in Stitch.

Connecting SendGrid

Connecting SendGrid to Stitch is a three-step process:

1. Add SendGrid as a Stitch data source
2. Generate a Webhook URL
3. Set up webhooks in SendGrid

Add SendGrid as a Stitch data source

1. Sign into your Stitch account.

2. On the Stitch Dashboard page, click the Add Integration button.

3. Click the SendGrid icon.

4. Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

   For example, the name “Stitch SendGrid” would create a schema called stitch_sendgrid in the destination. Note: Schema names cannot be changed after you save the integration.

5. Click Save Integration.

Generate a SendGrid Webhook URL

Once Stitch has successfully saved and created the SendGrid integration, you’ll be redirected to a page that displays your SendGrid webhook URL and token (which is blurred in the image below):

Click the Copy button to copy it.

Note that this particular URL won’t display in Stitch again once you click Continue. Think of this URL like you would your login or API credentials - keep it secret, keep it safe. You can, however, generate another URL should you need it.

Once you’ve copied your webhook URL, click Continue to wrap things up in Stitch.

Set up webhooks in SendGrid

The last step is to setup webhooks in your SendGrid account.

1. Sign into your SendGrid account.
2. In the pane on the left side of the screen, click Settings.
3. In the Settings menu, click Mail Settings.
4. Click Event Notification. This will expand the notification settings.
5. Click On to enable event notifications.
6. In the HTTP Post URL field, paste the Stitch-generated webhook URL.

7. In the Actions section, select the action types you want to send to Stitch:

   

8. When finished, click the checkmark on the right side of the notification settings.

Replication

After you’ve successfully connected your SendGrid integration, Stitch will continuously replicate your webhook data into your destination.

• Webhooks and historical data
• How data is loaded into your destination
• How to query for the latest data

Webhooks and historical data

Because SendGrid data is sent to Stitch in real-time, this means that only new records are replicated to your destination. Most webhook-based integrations don’t retain historical data due to this as-it-happens approach.

In the event that our webhook service experiences downtime, you may notice some lag between an event occurring and the data appearing in your destination.

Loading behavior

This version of Stitch’s SendGrid integration loads data in an Append-Only fashion. When data is loaded using the Append-Only behavior, records are appended to the end of the table as new rows. Existing rows in the table aren’t updated even if the source has defined Primary Keys. Multiple versions of a row can exist in a table, creating a log of how a record has changed over time. Data stored this way can provide insights and historical details about how those rows have changed over time.

Refer to the Understanding loading behavior guide for more info and examples.

Query for the latest data

If you simply want the latest version of the object - or objects, if you elected to track more than one during the setup - in the integration’s table (data), you’ll have to adjust your querying strategy to account for the append-only method of replication. This is a little different than querying records that are updated using updated_at Incremental Replication.

To do this, you can use the _sdc_sequence column and the table’s Primary Key. The _sdc_sequence is a Unix epoch (down to the millisecond) attached to the record during replication and can help determine the order of all the versions of a row.

Note: If you didn’t define a Primary Key while setting up the integration, the Primary Key for the table will be __sdc_primary_key.

If you wanted to create a snapshot of the latest version of this table, you could run a query like this:

SELECT * FROM [stitch-redshift:stitch-sendgrid.data] o
INNER JOIN (
    SELECT
        MAX(_sdc_sequence) AS seq,
        [primary-key]
    FROM [stitch-redshift:stitch-sendgrid.data]
    GROUP BY [primary-key]) oo
ON o.[primary-key] = oo.[primary-key]
AND o._sdc_sequence = oo.seq

This approach uses a subquery to get a single list of every row’s Primary Key and maximum sequence number. It then joins the original table to both the Primary Key and maximum sequence, which makes all other column values available for querying.

Dropped Email Events & Stitch Replication

On occasion, SendGrid’s API will send dropped event records with email: null. In conjunction with the timestamp and event fields, Stitch uses the email field to create a composite Primary Key for SendGrid integrations.

When Primary Key fields are NULL, Stitch will not persist them to a destination.

The following record would not persist to a destination because email is NULL:

{
  "sg_event_id":"sendgrid_internal_event_id",
  "sg_message_id":"sendgrid_internal_message_id",
  "email":null,
  "timestamp":1513375274,
  "smtp-id":"<original-smtp-id@domain.com>",
  "unique_arg_key":"unique_arg_value",
  "category":["category1", "category2", "category3"],
  "reason":"Bounced Address",
  "event":"dropped"
}

This record, however, would:

{
  "sg_event_id":"sendgrid_internal_event_id",
  "sg_message_id":"sendgrid_internal_message_id",
  "email":"stitch@stitchdata.com",
  "timestamp":1513375288,
  "smtp-id":"<original-smtp-id@domain.com>",
  "unique_arg_key":"unique_arg_value",
  "category":["category1", "category2"],
  "reason":"Unsubscribed Address",
  "event":"dropped"
}

Read more about SendGrid dropped events here.

Schema

The schema of the tables created by your SendGrid integration will contain two “types” of columns:

• Columns used by Stitch (prepended with _sdc), and
• Columns sent by the provider’s webhook API

Aside from the Stitch columns, the table’s schemas will depend entirely on SendGrid’s webhook API. With the exception of the _sdc fields, Stitch does not augment webhook data nor does it have any control over the fields sent by the webhook provider.

Webhook URLs & Security

Stitch allows you to generate up to 2 SendGrid webhook URLs at a time. These URLs contain security access tokens and as such, have access to your SendGrid account.

If you ever need to change your webhook URL, you can do so in the Integration Settings page after the integration has been created:

1. Click into the integration from the Stitch Dashboard page.
2. Click the Settings button.
3. In the Webhook URLs section, click the Generate Webhook URL button.
4. A new webhook URL will display. Press the clipboard icon to copy it.
5. Follow the steps in the Setting Up Webhooks in SendGrid section to update the webhook URL in SendGrid.
6. After you’ve updated the webhook URL in SendGrid, click the Revoke button next to the oldest webhook URL in Stitch. This will invalidate the token and revoke access.

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.