Apache Airflow vs. AWS Glue vs. Stitch
ETL software comparison
ETL software comparison
Most businesses have data stored in a variety of locations, from in-house databases to SaaS platforms. To get a full picture of their finances and operations, they pull data from all those sources into a data warehouse or data lake and run analytics against it. But they don't want to build and maintain their own data pipelines.
Fortunately, it’s not necessary to code everything in-house. Here's an comparison of two such tools, head to head.
Apache Airflow is an open source project that lets developers orchestrate workflows to extract, transform, load, and store data.
Amazon Web Services (AWS) has a host of tools for working with data in the cloud. Glue focuses on ETL. It's one of two AWS tools for moving data from sources to analytics destinations; the other is AWS Data Pipeline, which is more focused on data transfer.
Stitch and Talend partner with AWS. While this page details our products that have some overlapping functionality and the differences between them, we're more complementary than we are competitive. AWS offers lots of products beyond what's mentioned on this page, and we have thousands of customers who successfully use our solutions together.
Stitch Data Loader is a cloud-based platform for ETL — extract, transform, and load. More than 3,000 companies use Stitch to move billions of records every day from SaaS applications and databases into data warehouses and data lakes, where it can be analyzed with BI tools. Stitch is a Talend company and is part of the Talend Data Fabric.
Focus | Orchestration, scheduling, workflows | ETL, data catalog | Data ingestion, ELT | |||||||||||
Database replication | Only via plugins | Full table; incremental via change data capture through AWS Database Migration Service (DMS) | Full table; incremental via change data capture or SELECT/replication keys | |||||||||||
SaaS sources | Only via plugins | None | More than 100 | |||||||||||
Ability for customers to add new data sources | Yes | Developers can write custom Scala or Python code and import custom libraries and Jar files into Glue ETL jobs to access data sources not natively supported by AWS Glue. | Yes | |||||||||||
Connects to data warehouses? Data lakes? | Yes / Yes | Yes / Yes | Yes / Yes | |||||||||||
Transparent pricing | Yes | Yes | Yes | |||||||||||
G2 customer satisfaction | Not Rated | 4.1/5 | 4.8/5 | |||||||||||
Support SLAs | No | Available | Available | |||||||||||
Purchase process | Free to download and use | Options for self-service and talking with sales | Options for self-service or talking with sales. Also available from the AWS store. | |||||||||||
Compliance, governance, and security certifications | None | HIPAA, GDPR | HIPAA, GDPR, SOC 2 | |||||||||||
Data sharing | Yes, via plugins | Yes, within AWS | Yes, through Talend Data Fabric | |||||||||||
Vendor lock-in | Free to use | AWS Glue is strongly tied to the AWS platform. Usage is billed monthly. | Month to month or annual contracts. Open source integrations | |||||||||||
Developer tools | Experimental REST API | Import API, Stitch Connect API for integrating Stitch with other platforms, Singer open source project |
Let's dive into some of the details of each platform.
Apache Airflow is a powerful tool for authoring, scheduling, and monitoring workflows as directed acyclic graphs (DAG) of tasks. A DAG is a topological representation of the way data flows within a system. Airflow manages execution dependencies among jobs (known as operators in Airflow parlance) in the DAG, and programmatically handles job failures, retries, and alerting. Developers can write Python code to transform data as an action in a workflow.
AWS Glue provides 16 built-in preload transformations that let ETL jobs modify data to match the target schema. Glue generates Python code for ETL jobs that developers can modify to create more complex transformations, or they can use code written outside of Glue.
Stitch is an ELT product. Within the pipeline, Stitch does only transformations that are required for compatibility with the destination, such as translating data types or denesting data when relevant. Stitch is part of Talend, which also provides tools for transforming data either within the data warehouse or via external processing engines such as Spark and MapReduce. Transformations can be defined in SQL, Python, Java, or via graphical user interface.
Try Stitch for free for 14 days
Each of these tools supports a variety of data sources and destinations.
Airflow orchestrates workflows to extract, transform, load, and store data. It run <b>tasks</b>, which are sets of activities, via <b>operators</b>, which are templates for tasks that can by Python functions or external scripts. Developers can create operators for any source or destination. In addition, Airflow supports plugins that implement operators and <b>hooks</b> — interfaces to external platforms. The Airflow community has built plugins for databases like MySQL and Microsoft SQL Server and SaaS platforms such as Salesforce, Stripe, and Facebook Ads.
AWS Glue supports AWS data sources — Amazon Redshift, Amazon S3, Amazon RDS, and Amazon DynamoDB — and AWS destinations, as well as various databases via JDBC. Glue can also serve as an orchestration tool, so developers can write code that connects to other sources, processes the data, then writes it out to the data target.
Stitch supports more than 100 database and SaaS integrationsas data sources, and eight data warehouse and data lake destinations. Customers can contract with Stitch to build new sources, and anyone can add a new source to Stitch by developing it according to the standards laid out in Singer, an open source toolkit for writing scripts that move data. Singer integrations can be run independently, regardless of whether the user is a Stitch customer. Running Singer integrations on Stitch’s platform allows users to take advantage of Stitch's monitoring, scheduling, credential management, and autoscaling features.
Data integration tools can be complex, so vendors offer several ways to help their customers. Online documentation is the first resource users often turn to, and support teams can answer questions that aren't covered in the docs. Vendors of the more complicated tools may also offer training services.
The open source community provides Airflow support through a Slack community. Documentation includes quick start and how-to guides. Other than a tutorial on the Apache website there are no training resources.
AWS provides several levels of support. Free Basic support provides access to support forums. Develop support adds client-side diagnostic tools and guidance on how to use AWS products, features, and services together. Business and Enterprise plans add additional options. Documentation is comprehensive. Both digital training and classroom training services are available.
Stitch provides in-app chat support to all customers, and phone support is available for Enterprise customers. Support SLAs are available. Documentation is comprehensive and is open source — anyone can contribute additions and improvements or repurpose the content. Stitch does not provide training services.
Airflow is free and open source, licensed under Apache License 2.0.
AWS Glue ETL jobs are billed at an hourly rate based on data processing units (DPU), which map to performance of the serverless infrastructure on which Glue runs. For the AWS Glue Data Catalog, users pay a monthly fee for storing and accessing Data Catalog the metadata. The first million objects stored are free, and the first million accesses are free. A development endpoint provisioned to interactively develop ETL code is billed per second.
Stitch has pricing that scales to fit a wide range of budgets and company sizes. All new users get an unlimited 14-day trial. Standard plans range from $100 to $1,250 per month depending on scale, with discounts for paying annually. Enterprise plans for larger organizations and mission-critical use cases can include custom features, data volumes, and service levels, and are priced individually.
Which tool is better overall? That's something every organization has to decide based on its unique requirements, but we can help you get started. Sign up now for a free trial of Stitch.
Select your integrations, choose your warehouse, and enjoy Stitch free for 14 days.
Set up in minutesUnlimited data volume during trial