domain: dlthub.com · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Install dlt: pip install dlt and any extras for the destination (e.g. pip install dlt[bigquery] or dlt[snowflake]).
Define a source using the @dlt.resource or @dlt.source decorator, yielding or returning data as dicts, dataframes, or iterables.
Create a pipeline: pipeline = dlt.pipeline(pipeline_name='my_pipeline', destination='bigquery', dataset_name='my_dataset').
Run the pipeline: load_info = pipeline.run(my_source()); pass write_disposition='replace' or 'append' or 'merge' as needed.
Inspect load_info for load package details and check pipeline.last_trace for row counts, timing, and any errors.
Known gotchas
Destination credentials are read from environment variables (e.g. DESTINATION__BIGQUERY__CREDENTIALS) or a secrets.toml file in the .dlt directory; missing credentials raise a configuration error at runtime.
Schema inference happens on first run; subsequent runs may encounter schema evolution issues if source data types change — use dlt's schema contracts to enforce or relax schema rules.
The merge write disposition requires a primary key to be declared on the resource; without it, dlt falls back to append and emits a warning rather than raising an error.
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?