When you add a new column to a table in your data source, what happens in Stitch? What about syncing additional columns on already-syncing tables? Depending on the type of integration and the Replication Method the table uses, there are a few possible outcomes.

New Columns in the Data Source & Stitch

When a new column is added in a data source, Stitch will first need to perform a structure sync to detect it.

Note that:

  • It may take some time for Stitch to detect the new column and display it in the Integration Details page.
  • The following applies to both database and non-webhook SaaS integrations.

Already-Syncing Tables

After the structure sync completes, the column will be automatically synced and, if the integration supports whitelisting columns, display in the Integration Details page. Data for the column will then replicate based on the table’s Replication Method.

Keep in mind that for Incremental tables, you may need to backfill existing rows if the addition of new data hasn’t updated the table’s Replication Key.

Non-Syncing Tables

If the table isn’t currently selected to sync, the new column will display inside Stitch (if the integration supports whitelisting columns) but will have to be manually set to sync.

Syncing Additional Columns on Already-Syncing Tables

In this case, we aren’t talking about brand new columns in the data source, but previously existing columns that have never been set to sync in Stitch. How Stitch handles the syncing of additional rows depends on the table’s Replication Method.

Full Table Replication

For tables using Full Table Replication, data in the newly-synced column will be available for all rows - including new and existing - the next time the table is successfully replicated.

Incremental Replication

For tables using Incremental Replication, data in the newly-synced column will be available only for rows added AFTER the column is synced. Existing rows must be backfilled to make the data available.

Getting newly-synced column data into existing rows requires a full re-replication of the table. Because this can significantly impact your row count and we don’t want to re-replicate data without your say-so, we leave inserting newly-synced column data into existing rows up to you.

Backfilling existing rows

If you need to backfill already-replicated rows with data from the newly selected column, you can reset the table’s Replication Key. This will force a full re-replication of the table, populate the column in existing rows, and replicate new records.

Resetting database integration Replication Keys

Replication Keys in database integrations can be reset at the integration or the table level.

  • At the integration level, the reset will clear the replication key value for ALL tables AND queue a full re-replication for all tables in the integration.
  • At the table level, the reset will clear the replication key value AND queue a full re-replication for that table only.

To reset Replication Keys, do the following:

  1. Click into the integration from the Stitch Dashboard page.
  2. Next:
    • To reset the entire integration: Click the Settings link and skip to step 3.
    • To reset a table: Locate the table you want and click into it. Click the Table Settings link, located near the top right corner, and proceed to step 3.
  3. Scroll down to the Reset Replication Keys section.
  4. Click the Reset Keys button.
  5. When prompted, click OK to confirm.
  6. A Success! message will display at the top of the page.

At this point, a full re-replication of the integration or table will be queued. Note: If there is a large volume of data to be replicated, it may take some time before you see the changes in your data warehouse.

Resetting SaaS integration Replication Keys

Resetting the Replication Keys for a SaaS integration is done by changing the Historical Sync date in the Integration Settings page. When this date is changed, all saved values will be overwritten AND a full re-replication of the integration will be queued.

Note: This feature may not be available for some integrations. Because this approach uses date-based replication, some integrations may be incompatible. For example: Pardot doesn’t support date-based replication, meaning this feature will not be available for Pardot connections.

Changing the Historical Sync date has its own set of considerations and gotchas. Please refer to the Syncing Historical SaaS Data guide for more info.

Related Troubleshooting
Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.