Post-Load Webhook Reference

Post-load hooks allow you to configure a webhook that fires each time data is loaded into your existing destination.

Using post-load hooks, you can extend Stitch and automate dependent processes. For example: Trigger downstream processing in SQL, an Amazon Web Services Lambda function, Talend Cloud jobs, or any other system that can be controlled with an HTTP request.

Set up a post-load hook

You can configure a post-load hook in two ways:

1. In the Stitch app
2. Via the Connect API, if your Stitch plan includes API access

You can configure up to 10 post-load hooks.

Webhook triggers

Post-load hooks are sent on a per-integration, per-table basis to each configured post-load webhook URL. The post-load webhook must have a status of Enabled in order for Stitch to send requests.

If multiple tables are set to replicate for an integration, Stitch will send a request for each table every time data is successfully loaded or rejected.

If the load for a table fails, a post-load webhook will not be sent. For example: The Stitch database user has insufficient database privileges. In this scenario, Stitch will send an email notification.

Security

• Webhook URL requirements

• IP whitelisting

Webhook URL requirements

To be compatible with Stitch post-load hooks, the service must provide a properly formatted HTTPS webhook URL. For example: https://your-webhook-provider.com/webhooks

If the URL doesn’t use HTTPS, you’ll receive an Invalid URI error when you attempt to add the webhook in Stitch.

IP whitelisting

Post-load notification webhook requests will originate from one of the IP addresses for your Stitch account’s data pipeline region.

To add an additional layer of security, or if the service you’re using requires it, you can whitelist the Stitch IP addresses for your account’s data pipeline region. This ensures that only requests sent from Stitch will be accepted and processed by the webhook service you’re using.

Retries

If the service you’ve configured to receive post-load hooks encounters problems, Stitch will attempt to re-send the notification up to five times. Stitch will re-try when the following occurs:

• The request fails to connect
• The request doesn’t receive a response after 30 seconds (timeout)
• The request recieves a response code other than 2xx

If the notification can’t be delivered after the maximum number of retries, Stitch will send an email notification immediately after the last retry fails. Replaying post-load hooks isn’t currently supported.

Note: A load is still considered successful even if the post-load notification webhook fails.

Webhook payload

• Header

• Request format

Header

Post-load notification webhook requests are POST requests with a User-Agent of StitchLoadingWebhook. For example:

curl -X "POST" "https://your-webhook-service.com/webhook" \
     -H 'Content-Type: application/json' \
     -H 'User-Agent: StitchLoadingWebhook'

Note: At this time, custom request headers are not supported.

Request format

Post-load notification webhook requests have a JSON payload in the body, similar to the following example.

Refer to the Request body properties section for attribute descriptions:

{  
   "version":"1.0",
   "minimum_sdc_batched_at":"2019-07-15T14:45:08.026Z",
   "integration_name":"shopify",
   "new_version":false,
   "source_root_table_name":"orders",
   "destination_root_table_name":"orders",
   "table_version":0,
   "tables":[  
      {  
         "source_table_name":"orders",
         "destination_table_name":"orders",
         "num_rows":100,
         "num_rows_rejected":11,
         "s3_objects":[  
            "stitch-s3-bucket/orders/0_1563294550038.jsonl"
         ],
         "primary_keys":[
            "order_id"
         ],
         "bookmark_metadata":[  
            {  
               "key":"updated_at",
               "max_value":"2018-12-31T00:00:00Z",
               "min_value":"2018-12-31T00:00:00Z"
            }
         ],
         "max_time_extracted":"2019-07-15T14:43:09.030Z"
      }
   ]
}

Request body properties

A post-load notification webhook object is composed of the following:

• Top-level properties

• Table object properties

• Bookmark metadata object properties

Top-level properties

A post-load notification webhook object contains the following top-level properties:

Table object properties

The tables object contains info about the table being loaded. This list will contain at least one object, but may contain more if there are subtables.

Note: If using a destination that doesn’t natively support nested structures, this list will include subtables created as a result of denesting JSON arrays. Refer to the Nested JSON data structures guide for more info.

A table object contains the following properties:

Bookmark metadata object properties

If the table uses Key-based or Log-based Incremental Replication, the bookmark_metadata array will contain info about the table’s Replication Key.

A bookmark_metadata object contains the following properties: