Import Singer Source
Description
Section titled “Description”The Import Singer Source step (shown as Import: Singer Source in the step picker) pulls one or more streams from a Singer source into project tables. A Singer source is a connection of kind Singer Source that wraps a Singer tap — a connector for a specific SaaS app, API, or database, such as Stripe, GitHub, Slack, or BigQuery. The step discovers the tap’s available streams, you choose which to import and where each one lands, and the runner extracts the data and loads it into the targets.
For a full walkthrough, see the Singer Sources guide; for the available sources and their configuration docs, see the Singer Sources catalog.
Configuration
Section titled “Configuration”- Connection (required) — a connection of kind Singer Source.
- Sync Mode (required) — Full table (replace each run), Incremental (append new data), or Upsert (merge on key). Incremental works only for streams the tap can sync incrementally (those with a replication key); the step resumes from the previous run’s position. Upsert merges each run into the target on the stream’s key columns.
- Discover Streams — runs the tap’s catalog discovery on the runner and lists the streams it offers.
- Streams (at least one) — for each discovered stream: Import this stream to include it, and a Target Table for where it lands. Every selected stream needs a target table. When the sync mode is Upsert, each selected stream also needs Key Columns (one column name per line) — the columns a row is matched on; they default to the tap’s declared primary key.
Behavior
Section titled “Behavior”- Discovery runs as its own job and can take up to about three minutes on a cold start; the result is cached, so repeat discovery is instant.
- Full table replaces the target table each run; incremental appends only the rows that are new since the last run, using the tap’s replication key; upsert re-extracts the full stream and merges it into the target on each stream’s key columns — matching rows are updated, new keys are inserted, and unmatched existing rows are kept. The target is created on the first run. Rows with a null key column never match and are always inserted, so choose key columns that are always present.
- The selected streams are the saved set — to change the selection, re-discover. Streams that reappear keep the target table and selection you set.
- Source credentials are read from the connection at run time, so rotating them on the connection applies to every step that uses it — no per-step updates.