Stitch Import API Reference

Introduction

The Stitch Import API enables you to push arbitrary data from a source to Stitch. The Import API acts as a receiving point for data that is sent to Stitch. The Import API will ‘listen’ for new data, which is then processed by Stitch as it’s received.

This API is a RESTful, method-oriented API that allows you to push data from a source (including those Stitch doesn’t currently have an integration for) and send it to Stitch. Each endpoint uses standard HTTP verbs like GET and POST, and will return standard HTTP response codes to indicate request status or errors.

The API is built to accept JSON requests and to return JSON in all responses, including errors.

API functionality

Using the Import API, you can:

• Push data from a source to Stitch
• Check the status of the Import API
• Validate push requests and batches without persisting them to Stitch

Accessing the API

Anyone with a Stitch account can use the Import API. If you don’t currently have an account, sign up for a free trial here.

After you create a Stitch account, refer to the Managing Import API access tokens in Stitch guide for instructions on generating an Import API access token in the Stitch app.

Note: If you’re a Stitch Connect user, refer to the Managing Import API access tokens via the Connect API guide.

API support

While the Import API is a certified Stitch integration, using it does require some technical knowledge. This means that Stitch will address bugs and issues with the API that are within Stitch’s infrastructure, but Stitch doesn’t provide assistance for building or maintaining the scripts that send data to the API.

The table below provides some examples of the types of requests Stitch will and will not provide assistance for.

Refer to the Import API guides and resources for assistance when developing and testing your script.

Additionally, ask the Stitch Community if you’re still stuck.

Base URLs

The base URL you use to interact with the Import API depends on the data pipeline region your Stitch account is in.

For example: If your account is in the Europe (eu-central-1) region, you’ll use this base URL in your requests to the Import API: https://api.eu-central-1.stitchdata.com.

Not sure what data pipeline region your account is in? Click here for help.

Your base URL is currently set to:

Note: If your requests uses the incorrect base URL, the API will respond with a 401 - Unauthorzied code and an error.

Authentication

The Import API uses an API access token to authenticate requests. Import API access tokens can be generated and managed in the Integration Settings page for any Import API integration in your Stitch account. If you’re a Stitch Connect user, you can also use the Connect API to create Import API sources and manage their access tokens.

Authentication is performed via bearer auth, where your Import API access token is provided in the header of your request as -H 'Authorization: Bearer at_[IMPORT_API_ACCESS_TOKEN]'.

Your API access token has write access to the Stitch integration schema or dataset in your destination. Because of this, API access tokens should be treated like passwords - don’t share them in publicly accessible places like Stackoverflow, GitHub, etc. Note: You also shouldn’t share your API access token with Stitch Support. If sharing requests, remember to redact the API access token before sending.

If an API access token is ever lost or compromised, you can revoke it and create a new token.

Refer to the Managing Import API access tokens in Stitch guide for more info.

Stitch client IDs

For some the Push and Validate push request endpoints, you’ll need to include your Stitch client ID for every record contained in a request body. Your Stitch client ID is the unique ID associated with your Stitch account.

curl -X "POST" "https://api.stitchdata.com/v2/import/push" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'[
    {
      "client_id": 7723,              /* Stitch client ID */
      "table_name": "customers",
      "sequence": 1565880017,
      "data": {
        "id": 4,
        "name": "Beamo"
      },
      "key_names": [
        "id"
      ],
      "action": "upsert"
    },
    {
      "client_id": 7723,              /* Stitch client ID */
      "table_name": "orders",
      "sequence": 1565838645,
      "key_names": [
        "order_id"
      ],
      "data": {
        "order_id": 561,
        "customer_id": 4
      },
      "action": "upsert"
    }
  ]'

To locate your client ID, sign into your Stitch account and look at the URL. The client ID is the four to six-digit number between client and pipeline in the URL.

For example: If the URL is https://app.stitchdata.com/v2/client/7723/pipeline/connections, the client ID would be 7723.

A client ID may have multiple API access tokens associated with it, but an API access token will only ever be associated with a single client ID.

Response Codes

The API will attempt to return HTTP status codes to indicate the success or failure of a request sent to the API.

In general:

• Codes in the 2xx range indicate success
• Codes in the 4xx range indicate an error with the request, such as an invalid API access token, missing client ID, malformed request body, etc.
• Codes in the 5xx range indicate an issue with Stitch. Persistent 5xx codes may indicate an outage, which you can monitor on Stitch’s status page.

Your application should handle each of the following statuses gracefully:

Response and error message formats

The API will return responses - successful or errors - in JSON format. Below are descriptions and examples of each type of response.

Response message formats

For successful responses (codes in the 2xx range) the API will return messages as a JSON object:

{
  "status": "OK",
  "message": "Batch Accepted!"
}

In the Import API documentation, this is referred to as a Batch Status object.

Error message formats

For error responses (codes in the 4xx range), the API will return error messages in JSON format. In the Import API documentation, this is referred to as an Error object.

Some error messages may be returned as strings:

Content-Type must be application/json

While others may be JSON objects:

{
  "status": "ERROR",
  "error": "Forbidden",
  "errors": {
    "error": "Access token is not associated with this client."
  }
}

Error message text

The text in error messages will vary by root cause and endpoint.

Each endpoint section contains a list of the errors specific to that endpoint. Refer to the documentation for the endpoint for specifics on errors, their possible causes, and the messages the API will return.

Versioning

When we make backwards-incompatible changes to the API, we release a new version. The current version of the Import API is 2.

Methods

API Status

An object describing the status of the Import API at the time a request is sent.

Endpoints

An object describing the status of the Import API at the time a request is sent.

Example object

{
  "name": "pipeline.gate",
  "version": "0.3.3-SNAPSHOT",
  "revision": "a154360ad8c43182965049dbf6239aa7c1f3c84d",
  "status": "OK",
  "reason": null
}

Determines if the Import API is operating correctly. This endpoint doesn’t require any authentication.

Resource list

Returns

If successful, the API will return a 200 OK status and an API status object.

• Request
• Response

curl "https://api.stitchdata.com/v2/import/status"

{
  "name": "pipeline.gate",
  "version": "0.3.3-SNAPSHOT",
  "revision": "a154360ad8c43182965049dbf6239aa7c1f3c84d",
  "status": "OK",
  "reason": null
}

Batch

An object containing a table name, a table schema, and message objects representing records to be pushed to Stitch.

Endpoints

An object containing a table name, a table schema, and message objects representing records to be pushed to Stitch.

Example object

{
  "table_name": "customers",
  "schema": {
    "properties": {
      "id": {
        "type": "integer"
      },
      "name": {
        "type": "string"
      },
      "age": {
        "type": "integer"
      },
      "has_magic": {
        "type": "boolean"
      }
    }
  },
  "messages": [
    {
      "action": "upsert",
      "sequence": 1565880017,
      "data": {
        "id": 1,
        "name": "Finn",
        "age": 15,
        "has_magic": false
      }
    }
  ],
  "key_names": [
    "id"
  ]
}

Pushes a record or multiple records for a specified table to Stitch. Each request to this endpoint may only contain data for a single table.

When data for a table is pushed for the first time, Stitch will create the table in the destination in the specified integration schema.

How data is loaded during subsequent pushes depends on:

1. The loading behavior types used by the destination. Stitch supports Upsert and Append-Only loading.
2. Whether the key_names property specifies Primary Key fields. If Primary Keys aren’t specified, data will be loaded using Append-Only loading.

Refer to the Understanding loading behavior guide for more info and examples.

Structuring request body data

Refer to the Structuring data for the Import API guide for instructions and examples.

Resource list

Arguments

This endpoint accepts the following parameters:

Returns

If successful, the API will return a 2xx status and a Batch status object:

• 201 - The request was accepted and will be processed later. New data will be added once the request is processed.

• 202 - The request was accepted, but cannot currently be processed due to an internal error. Data will be automatically re-processed.

• Request
• Response
• Errors

curl -X "POST" "https://api.stitchdata.com/v2/import/batch" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'{
    "table_name": "customers",
    "schema": {
      "properties": {
        "id": {
          "type": "integer"
        },
        "name": {
          "type": "string"
        },
        "age": {
          "type": "integer"
        },
        "has_magic": {
          "type": "boolean"
        },
        "modified_at":{
            "type":"string",
            "format":"date-time"
         }
      }
    },
    "messages": [
      {
        "action": "upsert",
        "sequence": 1565880017,
        "data": {
          "id": 1,
          "name": "Finn",
          "age": 15,
          "has_magic": false,
          "modified_at":"2020-01-13T21:25:03+0000"
        }
      }
    ],
    "key_names": [
      "id"
    ]
  }'

curl -X "POST" "https://api.stitchdata.com/v2/import/batch" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'{
   "table_name":"customers",
   "schema":{
      "properties":{
         "id":{
            "type":"integer"
         },
         "name":{
            "type":"string"
         },
         "age":{
            "type":"integer"
         },
         "has_magic":{
            "type":"boolean"
         },
         "modified_at":{
            "type":"string",
            "format":"date-time"
         }
      }
   },
   "messages":[
      {
         "action":"upsert",
         "sequence":1565881320,
         "data":{
            "id":2,
            "name":"Jake",
            "age":6,
            "has_magic":true,
            "modified_at":"2020-01-13T21:25:03+0000"
         }
      },
      {
         "action":"upsert",
         "sequence":1565838645,
         "data":{
            "id":3,
            "name":"Bubblegum",
            "age":17,
            "has_magic":true,
            "modified_at":"2020-01-14T13:34:25+0000"
         }
      }
   ],
   "key_names":[
      "id"
   ]
}'

curl -X "POST" "https://api.stitchdata.com/v2/import/batch" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'{
   "table_name":"customers_no_primary_keys",
   "schema":{
      "properties":{
         "name":{
            "type":"string"
         },
         "age":{
            "type":"integer"
         },
         "has_magic":{
            "type":"boolean"
         },
         "modified_at":{
            "type":"string",
            "format":"date-time"
         }
      }
   },
   "messages":[
      {
         "action":"upsert",
         "sequence":1565881320,
         "data":{
            "name":"BMO",
            "age":2,
            "has_magic":false,
            "modified_at":"2020-01-20T05:57:01+0000"
         }
      }
   ]
}'

{
  "status": "OK",
  "message": "Batch Accepted!"
}

{
  "status": "Accepted",
  "message": "The batch is queued to be processed."
}

{
  "error": "Request failed validation:#: required key [<ARGUMENT_NAME>] not found"
}

{
  "error": "Request failed validation:#: extraneous key [<ARGUMENT_NAME>] is not permitted"
}

{
  "error": "Request failed validation:#/<ARGUMENT_NAME>: expected type: <CORRECT_TYPE>, found: <TYPE>"
}

{
  "error": "Record 0 did not conform to schema: #/<FIELD_NAME>: expected: <DATA_TYPE>, found: <DATA_TYPE>"
}

{
  "error": "Record is missing key property <KEY_NAME>"
}

{
  "error": "Invalid JSON schema: unknown type: [<TYPE>]"
}

{
  "error": "Request failed validation:#/messages/0: #: 0 subschemas matched instead of one #/messages/0: 2 schema violations found #/messages/0: required key [<ARGUMENT_NAME>] not found #/messages/0: extraneous key [data] is not permitted #/messages/0: required key [ARGUMENT_NAME] not found"
}

{
  "error": "Request failed validation:#: sequence can not be above 9223372036854775807"
}

{
  "error": "Request failed validation:#/schema: expected type: JSONObject, found: JSONArray"
}

{  
   "message":"Not Authorized"
}

{  
   "message": "Invalid region."
}

{
  "error": "Forbidden",
  "errors": {
    "error": "Connection is not active."
  }
}

{
  "status": "ERROR",
  "message": "Request rejected: request size (<BYTE>s bytes) exceeds the maximum"
}

Validate

An object containing table information, your client ID, and data objects representing records to be validated without being persisted to Stitch. The Validate object is identical to the Push object.

Endpoints

An object containing table information, your client ID, and data objects representing records to be validated without being persisted to Stitch. The Validate object is identical to the Push object.

Example object

{
  "client_id":7723,
  "table_name":"customers",
  "sequence":1565880017,
  "data":{
     "id":1,
     "name":"Finn"
  },
  "key_names":[
     "id"
  ],
  "action":"upsert"
}

Note: With the exception of not persisting data, this endpoint is functionally identical to the Push endpoint. The Validate endpoint will not work if a request body intended for the Batch endpoint is sent.

Validates request credentials and batch data, but doesn’t persist the data to Stitch. This allows you to validate your API credentials and batch data during development.

Regardless of whether the Import API is functional, this endpoint will never return a 503 Service Unavailable response. Use the Status endpoint to determine if the Import API is experiencing issues.

Resource list

Arguments

This endpoint accepts the following parameters:

Returns

If successful, the API will return a 200 OK status and a Batch Status object.

• Request
• Response
• Errors

curl -X "POST" "https://api.stitchdata.com/v2/import/validate" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'[
   {
      "client_id":7723,
      "table_name":"customers",
      "sequence":1565881320,
      "key_names":[
         "id"
      ],
      "data":{
         "id":1,
         "name":"Finn"
      },
      "action":"upsert"
   }
]'

{
  "status": "OK",
  "message": "Batch is valid!"
}

{
  "status": "ERROR",
  "message": "An array of records is expected"
}

{
  "status": "ERROR",
  "message": "Malformed json in the body!",
  "error": "Unexpected character (']' (code 93)): expected a value\n at [Source: java.io.StringReader@796a657e; line: 16, column: 2]",
  "input": null
}

{  
   "message": "Invalid region."
}

{
  "status": "ERROR",
  "error": "Not Authenticated.",
  "errors": null
}

{
  "error": "Forbidden",
  "errors": {
    "error": "Connection is not active."
  }
}

{
  "status": "ERROR",
  "error": "Forbidden",
  "errors": {
    "error": "Access token is not associated with this client."
  }
}

{
  "status": "ERROR",
  "message": "Request rejected: request size (<BYTE>s bytes) exceeds the maximum"
}

{
  "status": "ERROR",
  "message": "Unsupported content-type application/<TYPE>. Please send application/json or application/transit+json"
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": "The batch contains data points for multiple clients. Only client_id <CLIENT_ID> is allowed"
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "<REQUIRED_ARGUMENT_NAME>": {
        "table_name": [
          "can't be blank"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "sequence": [
          "should be an integer",
          "should be a number"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must be an object"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must include keys"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "keys cannot not be null in data"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "key_names": [
          "can't be blank"
        ],
        "key_properties": [
          "unknown key"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "String keys cannot be longer than 1024 characters"
        ]
      }
    }
  ]
}

Push

An object containing table information, your client ID, and data objects representing records to be pushed to Stitch.

Endpoints

An object containing table information, your client ID, and data objects representing records to be pushed to Stitch.

Example object

{
  "client_id":7723,
  "table_name":"customers",
  "sequence":1565880017,
  "data":{
     "id":1,
     "name":"Finn"
  },
  "key_names":[
     "id"
  ],
  "action":"upsert"
}

Pushes data for a specified table or tables to Stitch.

When data for a table is pushed for the first time, Stitch will create the table in the destination in the specified integration schema.

During subsequent pushes, one of two things will happen depending on the destination being used:

1. If the destination supports upserts, Stitch will perform an update operation on applicable existing rows to overwrite the data.
2. If the destination doesn’t support upserts, Stitch will load the records in an append-only fashion. This means that existing records in the destination table will not be updated, and all records in subsequent pushes will be appended to the end of the table.

Structuring request body data

Refer to the Structuring data for the Import API guide for instructions and examples.

Resource list

Arguments

This endpoint accepts the following parameters:

Returns

If successful, the API will return a 2xx status and a Batch Status object:

• 201 - The request was accepted and will be processed later. New data will be added once the request is processed.

• 202 - The request was accepted, but cannot currently be processed due to an internal error. Data will be automatically re-processed.

• Request
• Response
• Errors

curl -X "POST" "https://api.stitchdata.com/v2/import/push" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'[
   {
      "client_id":7723,
      "table_name":"customers",
      "sequence":1565880017,
      "data":{
         "id":1,
         "name":"Finn"
      },
      "key_names":[
         "id"
      ],
      "action":"upsert"
   }
]'

curl -X "POST" "https://api.stitchdata.com/v2/import/push" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'[
   {
      "client_id":7723,
      "table_name":"customers",
      "sequence":1565880017,
      "data":{
         "id":4,
         "name":"BMO"
      },
      "key_names":[
         "id"
      ],
      "action":"upsert"
   },
   {
      "client_id":7723,
      "table_name":"customers",
      "sequence":1565838645,
      "data":{
         "id":5,
         "name":"Ice King"
      },
      "key_names":[
         "id"
      ],
      "action":"upsert"
   }
]'

curl -X "POST" "https://api.stitchdata.com/v2/import/push" \
     -H 'Authorization: Bearer [ACCESS_TOKEN]' \
     -H 'Content-Type: application/json' \
     -d \
'[
   {
      "client_id":7723,
      "table_name":"customers",
      "sequence":1565880017,
      "data":{
         "id":4,
         "name":"BMO"
      },
      "key_names":[
         "id"
      ],
      "action":"upsert"
   },
   {
      "client_id":7723,
      "table_name":"orders",
      "sequence":1565838645,
      "data":{
         "order_id":561,
         "customer_id":4
      },
      "key_names":[
         "order_id"
      ],
      "action":"upsert"
   }
]'

{
  "status": "OK",
  "message": "Batch Accepted!"
}

{
  "status": "Accepted",
  "message": "The batch is queued to be processed."
}

{
  "status": "ERROR",
  "message": "An array of records is expected"
}

{
  "status": "ERROR",
  "message": "Malformed json in the body!",
  "error": "Unexpected character (']' (code 93)): expected a value\n at [Source: java.io.StringReader@796a657e; line: 16, column: 2]",
  "input": null
}

{  
   "message": "Invalid region."
}

{
  "status": "ERROR",
  "error": "Not Authenticated.",
  "errors": null
}

{
  "error": "Forbidden",
  "errors": {
    "error": "Connection is not active."
  }
}

{
  "status": "ERROR",
  "error": "Forbidden",
  "errors": {
    "error": "Access token is not associated with this client."
  }
}

{
  "status": "ERROR",
  "message": "Request rejected: request size (<BYTE>s bytes) exceeds the maximum"
}

{
  "status": "ERROR",
  "message": "Unsupported content-type application/<TYPE>. Please send application/json or application/transit+json"
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": "The batch contains data points for multiple clients. Only client_id <CLIENT_ID> is allowed"
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "<REQUIRED_ARGUMENT_NAME>": {
        "table_name": [
          "can't be blank"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "sequence": [
          "should be an integer",
          "should be a number"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must be an object"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must include keys"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "keys cannot not be null in data"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "key_names": [
          "can't be blank"
        ],
        "key_properties": [
          "unknown key"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "String keys cannot be longer than 1024 characters"
        ]
      }
    }
  ]
}

Data Structures

A Batch Status object provides information about the validation and acceptance status of a batch of data in a request body. Note: This object is only returned if the request body complies with the endpoint’s requirements.

Example object

• Validate - 200 response
• Push - 201 response
• Push - 202 response

{
  "status": "OK",
  "message": "Batch is valid!"
}

{
  "status": "OK",
  "message": "Batch Accepted!"
}

{
  "status": "Accepted",
  "message": "The batch is queued to be processed."
}

Contained in an Error object, the Error Reason object contains additional details about the reason a request resulted in an error response.

Example object

• With reason as a string
• With reason as an object

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": "The batch contains data points for multiple clients. Only client_id <CLIENT_ID> is allowed"
    }
  ]
}

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must include keys"
        ]
      }
    }
  ]
}

An Error object provides information about the reason a request resulted in an error response from the Import API.

Example object

• With error reason object
• Without error reason object

{
  "status": "ERROR",
  "error": "Request cannot be processed; see errors.",
  "errors": [
    {
      "reason": {
        "data": [
          "data must include keys"
        ]
      }
    }
  ]
}

{
  "status": "ERROR",
  "message": "An array of records is expected"
}

A Message object contains information about a record to be upserted into a table.

Example object

{
  "action": "upsert",
  "sequence": 1574796577000,
  "data": {
    "id": 1,
    "name": "Finn",
    "age": 16,
    "has_magic": false
  }
}

A Schema object contains a valid JSON schema describing the record(s) in a Message object’s data property.

Refer to the JSON schema docs for more info about JSON schemas.

Example object

{
   "schema":{
      "properties":{
         "id":{
            "type":"integer"
         },
         "name":{
            "type":"string"
         },
         "age":{
            "type":"number"
         },
         "has_magic":{
            "type":"boolean"
         }
      }
   }
}