LookML integration summary

Stitch’s LookML integration connects to GitHub repositories where your Looker LookML is hosted for your Looker project. Stitch then replicates it using GitHub v3 API to extract LookML components using lkml parser.

Refer to the Schema section for a list of objects available for replication.

LookML feature snapshot

A high-level look at Stitch's LookML (v1) integration, including release status, useful links, and the features supported in Stitch.

STITCH
Release status

Beta

 Supported by

Singer Community
Stitch plan

Standard

 API availability

Available
Singer GitHub repository

singer-io/tap-lookml
REPLICATION SETTINGS
Anchor Scheduling

Supported

 Advanced Scheduling

Supported
Table-level reset

Unsupported

 Configurable Replication Methods

Unsupported
DATA SELECTION
Table selection

Supported

 Column selection

Supported
Select all

Supported
TRANSPARENCY
Extraction Logs

Supported

 Loading Reports

Supported

Connecting LookML

LookML setup requirements

To set up LookML in Stitch, you need:

  • Access to the GitHub repositories you want to replicate data from. Stitch will only be able to access the same repositories as the user who creates the access token.

Step 1: Create access token

  1. Sign into your GitHub account.
  2. Click the User menu (your icon) > Settings.
  3. Click Developer settings in the navigation on the left side of the page.
  4. Click Personal access tokens.
  5. On the Personal access tokens page, click the Generate a personal access token button. If prompted, enter your password.
  6. In the Description field, enter stitch. This will allow you to easily idenfiy what application is using the token.

  7. In the Select Scopes section, check the repo option:

    Highlighted repo scopes on the GitHub Personal Access Tokens page

    Note: While these are full permissions, Stitch will only ever read your data. The repo scope is required due to how GitHub structures permissions.

  8. Click the Generate token button.
  9. The new access token will display on the next page. Copy the token before navigating away from the page - GitHub won’t display it again.

Step 2: Add LookML as a Stitch data source

  1. Sign into your Stitch account.

  2. On the Stitch Dashboard page, click the Add Integration button.

  3. Click the LookML icon.

  4. Enter a name for the integration. This is the name that will display on the Stitch Dashboard for the integration; it’ll also be used to create the schema in your destination.

    For example, the name “Stitch LookML” would create a schema called stitch_lookml in the destination. Note: Schema names cannot be changed after you save the integration.

  5. In the API Token field, paste the access token you copied from step 1.

  6. In the Git Owner field, enter the owner of the repositories you want to replicate. You can find this info in the repository’s URL. For example: The Git owner of https://github.com/stitch/stitch-repo would be stitch.

    Note: While you can replicate data from multiple repositories, this integration only supports replicating from one owner at a time.

  7. In the Git Repositories field, enter the repository or repositories you want to replicate. You can find this info in the repository’s URL. For example: The repository of https://github.com/stitch/stitch-repo would be stitch-repo.

    To track multiple repositories, enter them as a comma-delimited list. For example: repo-1, repo-2.

Step 3: Define the historical replication start date

The Sync Historical Data setting defines the starting date for your LookML integration. This means that data equal to or newer than this date will be replicated to your data warehouse.

Change this setting if you want to replicate data beyond LookML’s default setting of 1 year. For a detailed look at historical replication jobs, check out the Syncing Historical SaaS Data guide.

Step 4: Create a replication schedule

In the Replication Frequency section, you’ll create the integration’s replication schedule. An integration’s replication schedule determines how often Stitch runs a replication job, and the time that job begins.

LookML integrations support the following replication scheduling methods:

To keep your row usage low, consider setting the integration to replicate less frequently. See the Understanding and Reducing Your Row Usage guide for tips on reducing your usage.

Step 5: Set objects to replicate

The last step is to select the tables and columns you want to replicate. Learn about the available tables for this integration.

Note: If a replication job is currently in progress, new selections won’t be used until the next job starts.

For LookML integrations, you can select:

  1. Individual tables and columns

  2. All tables and columns

Click the tabs to view instructions for each selection method.

  1. In the integration’s Tables to Replicate tab, locate a table you want to replicate.

  2. To track a table, click the checkbox next to the table’s name. A blue checkmark means the table is set to replicate.

  3. To track a column, click the checkbox next to the column’s name. A blue checkmark means the column is set to replicate.

  4. Repeat this process for all the tables and columns you want to replicate.
  5. When finished, click the Finalize Your Selections button at the bottom of the screen to save your selections.
  1. Click into the integration from the Stitch Dashboard page.

  2. Click the Tables to Replicate tab.

  3. In the list of tables, click the box next to the Table Names column.

  4. In the menu that displays, click Track all Tables and Fields:

    The Track all Tables and Fields menu in the Tables to Replicate tab

  5. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Initial and historical replication jobs

After you finish setting up LookML, its Sync Status may show as Pending on either the Stitch Dashboard or in the Integration Details page.

For a new integration, a Pending status indicates that Stitch is in the process of scheduling the initial replication job for the integration. This may take some time to complete.

Free historical data loads

The first seven days of replication, beginning when data is first replicated, are free. Rows replicated from the new integration during this time won’t count towards your quota. Stitch offers this as a way of testing new integrations, measuring usage, and ensuring historical data volumes don’t quickly consume your quota.

LookML table reference

model_files

The model_files table contains information about model files in your GitHub about using the Git API Search filename and extension filters for model and lkml.

Replication Method

Key-based Incremental

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified
Useful links

model_files schema on GitHub

LookML API method

model_files table schema

content

STRING

download_url

STRING

encoding

STRING

git_owner

STRING

git_repository

STRING

git_url

STRING

html_url

STRING

last_modified

DATE-TIME

name

STRING

path

STRING

sha

STRING

size

INTEGER

type

STRING

url

STRING

models

The models table contains information about the model file parse items in your GitHub account using the lkml filter.

Replication Method

Full Table

Primary Keys

git_owner

git_repository

path
Useful links

models schema on GitHub

LookML API method

models table schema

access_grants

ARRAY

Click to expand access_grants

allowed_values

ARRAY

name

STRING

user_attribute

STRING

case_sensitive

STRING

connection

STRING

datagroups

ARRAY

Click to expand datagroups

max_cache_age

STRING

name

STRING

sql_trigger

STRING

explores

ARRAY

Click to expand explores

access_filters

ARRAY

Click to expand access_filters

field

STRING

user_attribute

STRING

always_filter

OBJECT

Click to expand always_filter

filters

ARRAY

Click to expand filters

field

STRING

value

STRING

always_join

ARRAY

cancel_grouping_fields

ARRAY

case_sensitive

STRING

conditionally_filter

OBJECT

Click to expand conditionally_filter

filters

ARRAY

Click to expand filters

field

STRING

value

STRING

unless

ARRAY

description

STRING

extends

ARRAY

extension

STRING

fields

ARRAY

from

STRING

group_label

STRING

hidden

STRING

joins

ARRAY

Click to expand joins

fields

ARRAY

foreign_key

STRING

from

STRING

name

STRING

outer_only

STRING

relationship

STRING

required_access_grants

ARRAY

required_joins

ARRAY

sql_foreign_key

STRING

sql_on

STRING

sql_table_name

STRING

sql_where

STRING

type

STRING

view_label

STRING

label

STRING

name

STRING

persist_for

STRING

persist_with

STRING

required_access_grants

ARRAY

sql_always_having

STRING

sql_always_where

STRING

sql_table_name

STRING

symmetric_aggregates

STRING

view_label

STRING

view_name

STRING

fiscal_month_offset

STRING

git_owner

STRING

git_repository

STRING

includes

ARRAY

label

STRING

last_modified

DATE-TIME

map_layers

ARRAY

Click to expand map_layers

extents_json_url

STRING

feature_key

STRING

file

STRING

format

STRING

label

STRING

max_zoom_level

INTEGER

min_zoom_level

INTEGER

name

STRING

projection

STRING

property_key

STRING

property_label_key

STRING

url

STRING

named_value_formats

ARRAY

Click to expand named_value_formats

name

STRING

strict_value_format

STRING

value_format

STRING

path

STRING

persist_for

STRING

persist_with

STRING

sha

STRING

week_start_day

STRING

view_files

The view_files table contains information about view files, using the Git API Search filename and extension filters for views and lkml.

Replication Method

Key-based Incremental

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified
Useful links

view_files schema on GitHub

LookML API method

view_files table schema

content

STRING

download_url

STRING

encoding

STRING

git_owner

STRING

git_repository

STRING

git_url

STRING

html_url

STRING

last_modified

DATE-TIME

name

STRING

path

STRING

sha

STRING

size

INTEGER

type

STRING

url

STRING

views

The views table contains information about the view file parse items in your GitHub account using the lkml filter.

Replication Method

Full Table

Primary Keys

git_owner

git_repository

path

Replication Key

last_modified
Useful links

views schema on GitHub

LookML API method

views table schema

derived_table

OBJECT

Click to expand derived_table

cluster_keys

ARRAY

create_process

OBJECT

datagroup_trigger

STRING

distribution

STRING

distribution_style

STRING

explore_source

OBJECT

Click to expand explore_source

bind_filters

OBJECT

Click to expand bind_filters

from_field

STRING

to_field

STRING

columns

ARRAY

Click to expand columns

field

STRING

name

STRING

derived_columns

ARRAY

Click to expand derived_columns

name

STRING

sql

STRING

expression_custom_filter

STRING

filters

ARRAY

Click to expand filters

field

STRING

value

STRING

limit

STRING

name

STRING

sort

OBJECT

Click to expand sort

desc

STRING

field

STRING

timezone

STRING

indexes

ARRAY

partition_keys

ARRAY

persist_for

STRING

sortkeys

ARRAY

sql

STRING

sql_create

STRING

sql_trigger_value

STRING

dimension_groups

ARRAY

Click to expand dimension_groups

alias

ARRAY

allow_fill

STRING

bypass_suggest_restrictions

STRING

can_filter

STRING

convert_tz

STRING

datatype

STRING

description

STRING

drill_fields

ARRAY

fanout_on

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

intervals

ARRAY

label

STRING

name

STRING

order_by_field

STRING

required_access_grants

ARRAY

skip_drill_filter

STRING

sql

STRING

sql_end

STRING

sql_start

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggestable

STRING

tags

ARRAY

timeframes

ARRAY

type

STRING

view_label

STRING

dimensions

ARRAY

Click to expand dimensions

actions

ARRAY

Click to expand actions

form_params

ARRAY

Click to expand form_params

default

STRING

label

STRING

name

STRING

options

ARRAY

Click to expand options

label

STRING

name

STRING

required

STRING

type

STRING

form_url

STRING

icon_url

STRING

label

STRING

params

ARRAY

Click to expand params

name

STRING

value

STRING

url

STRING

user_attribute_params

ARRAY

Click to expand user_attribute_params

name

STRING

user_attribute

STRING

alias

ARRAY

alpha_sort

STRING

bypass_suggest_restrictions

STRING

can_filter

STRING

case

OBJECT

Click to expand case

whens

ARRAY

Click to expand whens

label

STRING

sql

STRING

case_sensitive

STRING

convert_tz

STRING

datatype

STRING

description

STRING

drill_fields

ARRAY

fanout_on

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

label

STRING

label_from_parameter

STRING

links

ARRAY

map_layer_name

STRING

name

STRING

order_by_field

STRING

primary_key

STRING

required_access_grants

ARRAY

required_fields

ARRAY

skip_drill_filter

STRING

sql

STRING

sql_end

STRING

sql_latitude

STRING

sql_longitude

STRING

sql_start

STRING

style

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

tiers

ARRAY

type

STRING

value_format

STRING

value_format_name

STRING

view_label

STRING

extends

ARRAY

extension

STRING

filters

ARRAY

Click to expand filters

alias

ARRAY

bypass_suggest_restrictions

STRING

case_sensitive

STRING

convert_tz

STRING

datatype

STRING

default_value

STRING

description

STRING

full_suggestions

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

label

STRING

name

STRING

required_access_grants

ARRAY

required_fields

ARRAY

sql

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

type

STRING

view_label

STRING

git_owner

STRING

git_repository

STRING

last_modified

DATE-TIME

measures

ARRAY

Click to expand measures

actions

ARRAY

Click to expand actions

form_params

ARRAY

Click to expand form_params

default

STRING

label

STRING

name

STRING

options

ARRAY

Click to expand options

label

STRING

name

STRING

required

STRING

type

STRING

form_url

STRING

icon_url

STRING

label

STRING

params

ARRAY

Click to expand params

name

STRING

value

STRING

url

STRING

user_attribute_params

ARRAY

Click to expand user_attribute_params

name

STRING

user_attribute

STRING

alias

ARRAY

approximate

STRING

approximate_threshold

NUMBER

can_filter

STRING

convert_tz

STRING

datatype

STRING

description

STRING

direction

STRING

drill_fields

ARRAY

fanout_on

STRING

filters

ARRAY

Click to expand filters

field

STRING

value

STRING

group_item_label

STRING

group_label

STRING

hidden

STRING

html

STRING

label

STRING

label_from_parameter

STRING

links

ARRAY

list_field

STRING

name

STRING

percentile

NUMBER

precision

INTEGER

required_access_grants

ARRAY

required_fields

ARRAY

sql

STRING

sql_distinct_key

STRING

suggest_dimension

STRING

suggest_explore

STRING

suggestable

STRING

tags

ARRAY

type

STRING

value_format

STRING

value_format_name

STRING

view_label

STRING

name

STRING

parameters

ARRAY

Click to expand parameters

alias

ARRAY

allowed_values

ARRAY

Click to expand allowed_values

label

STRING

value

STRING

bypass_suggest_restrictions

STRING

convert_tz

STRING

default_value

STRING

description

STRING

full_suggestions

STRING

hidden

STRING

label

STRING

name

STRING

required_access_grants

ARRAY

required_fields

ARRAY

suggest_dimension

STRING

suggest_explore

STRING

suggest_persist_for

STRING

suggestable

STRING

suggestions

ARRAY

tags

ARRAY

type

STRING

view_label

STRING

path

STRING

required_access_grants

ARRAY

sets

ARRAY

Click to expand sets

fields

ARRAY

name

STRING

sha

STRING

sql_table_name

STRING

suggestions

STRING

view_label

STRING

Related Troubleshooting
Questions? Feedback?

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.