Understanding and Reducing Your Row Usage

In this guide, we’ll cover the basics of Stitch billing, how to check and understand your usage, and some tips for keeping your row count low.

In this guide, we’ll cover:

• The basics of Stitch billing

• Understanding your row usage

• How to check your row usage in Stitch

• Some tips for reducing your usage

Stitch billing basics

• How does Stitch billing work?

• What’s a replicated row?

How does Stitch billing work?

Much like the data part of a cell phone plan, each Stitch plan is allotted a certain number of replicated rows per month. For detailed info on pricing and what’s included in each plan, refer to the pricing page on our website.

What's a replicated row?

Stitch counts the following as a ‘replicated row’:

• A new row, or a never-before-replicated row replicated through Stitch,
• An updated row, or an existing row that’s been changed,
• A sub-row created from de-nesting nested data structures, and
• A copy of an existing row. For example: Rows in tables that are replicated fully during each replication job or rows replicated as a result of resetting Replication Keys.

Understand your usage

• Source rows ≠ replicated rows

• Impacts on usage

• Usage examples

Source rows ≠ replicated rows

When viewing the number of replicated rows in Stitch, you may be surprised by the totals. You may ask yourself:

“How did Stitch replicate this many rows? There aren’t that many in my source or destination!”

We understand that this can be confusing. Keep in mind that row usage in Stitch is the total number of replicated rows. This means that the number of rows in the source won’t necessarily be equal to your row usage in Stitch.

Because Stitch counts updated rows, copies of existing rows, and rows created from de-nesting towards your total usage, the total of replicated rows and the total number of rows in your data sources or destination may not be equal.

Take the following example. As you can see in the second replication job, the total replicated rows reported by Stitch are cumulative, or the total rows replicated across all replication jobs.

Click to enlarge.

Impacts on usage

The number of rows Stitch replicates is directly impacted by:

1. The number of tables set to replicate. The more tables that you select, the potential higher number of rows.

2. The Replication Methods used by the replicating tables. Tables using Full Table Replication can increase your row usage.

3. Integrations’ replication schedules. Integrations scheduled to replicate on a frequent basis can lead to increased row usage.

4. The volume and structure of the data in the selected tables. Some Stitch destinations - like Redshift and PostgreSQL - will break apart nested records and count each sub-record as a row. Refer to the Nested data structures guide for more info and examples.

Usage examples

Next, we’ll look at some examples of how certain factors can affect row usage in Stitch.

When looking at the examples, note the differences between the source rows and row totals reported in Stitch.

• Examples
• Replication Frequency
• Data structure

{
   "id":1,
   "name":"Finn",
   "type":"human",
   "best_friends":[
      {
         "id":2,
         "name":"Jake",
         "type":"dog"
      },
      {
         "id":3,
         "name":"Bubblegum",
         "type":"princess"
      },
      {
         "id":4,
         "name":"BMO",
         "type":"robot"
      }
   ]
}

Check your row usage

• Row usage for all integrations

• Row usage for an integration

• Row usage reset date

Row usage for all integrations

On the Stitch Dashboard page, you can view the total number of replicated rows for all of your integrations for the current billing period:

Row usage for an integration

To take a closer look at an individual integration’s usage for the current billing period, click on the integration to open the Integration Details page, and check out the Rows Loaded Over Time section:

Row usage reset date

The reset date - or the day your row count will reset to zero - can be found in the Your Usage section of your Billing page, accessed by clicking the User menu (your icon) > Billing:

Reduce your usage

While you can change your plan at any time to accommodate your data volume needs, below are some tried-and-true tips for reducing your row usage and staying within your plan’s row allotment:

• Identify high usage integrations

• Reduce Replication Frequencies

• Use an incremental Replication Method

• De-select unnecessary data

• Pause integrations

Identify high usage integrations

For many of Stitch’s integrations, row usage shouldn’t be an issue. We attempt to use Key-based Incremental Replication for SaaS integrations whenever possible.

There are, however, times when high row usage may be unavoidable. For example:

• Data contains many nested structures. This is applicable to row usage only when a destination doesn’t natively support nested JSON structures.

• Source generates large amounts of data

• Integration has a high number of tables using Full Table Replication

• Integration doesn’t currently support table selection

• Integration uses an attribution window for extraction

Click below to display integrations known to be heavy row users, and the potential reasons for their increased usage. If you’re using any of these integrations, you can use the remaining tips in this section to keep your usage down.

Reduce Replication Frequencies

Generally, the more often an integration is scheduled to replicate, the higher the number of rows Stitch replicates for the inetgration.

If you’re able to get by without the freshest data, consider changing your integrations’ Replication Frequency to something less frequent. For example: Every hour or six hours.

Keep in mind that the Replication Frequency setting applies to the entire integration, not individual tables. This is especially important if there are a lot of tables that use Full Table Replication in the integration.

Use an incremental Replication Method

For integrations that support Replication Method configuration, we recommend using either Key-based or Log-based Incremental Replication whenever possible.

De-select unnecessary data

Note: This is only applicable to the integrations that support table and/or column selection.

To keep your row count down and your destination tidy, you can also de-select any tables or columns you don’t need.

For example: If a column contains nested data, additional sub-rows may be created to accommodate loading the data to certain destination types. This will increase the total row count, as Stitch counts sub-rows towards usage. If this column is no longer needed, you could de-select it and lower your usage.

Pause integrations

If all else fails, you can temporarily pause the integration to keep from going over your row limit.

Note: Pausing an integration will only prevent the extraction of additional records. Loading will continue for records that have been extracted prior to the pause.

For example: If there are records currently in Preparing when an integration is paused, Stitch will continue to load these records, complete the current replication job, and count them towards your usage.

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.