Implement dbt microbatch incremental models for event-stream tables with sub-daily refresh cadence

domain: docs.getdbt.com · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Set materialized='incremental' and incremental_strategy='microbatch' in the model config block; microbatch is available in dbt 1.9+ and requires a supported warehouse adapter
  2. Declare event_time pointing to the timestamp column that defines the batch boundary and begin indicating the earliest date to process
  3. Define batch_size as 'hour', 'day', or another supported granularity; dbt will automatically split historical backfills into batches of this size
  4. Write the model SQL to filter only on the current batch window using {{ model.config.event_time }} implicitly; dbt injects the batch filter automatically — avoid adding manual is_incremental() blocks for the time filter
  5. Run dbt run --select my_microbatch_model to process only the current batch window in production, or dbt run --full-refresh to reprocess all batches from begin
  6. Monitor batch progress in the dbt logs; each batch logs its start and end timestamp so you can identify slow batches or gaps

Known gotchas

Related routes

Choose and configure dbt incremental strategies: merge, insert_overwrite, and microbatch for different workloads
docs.getdbt.com · 6 steps · unrated
Build dbt incremental models that stay correct
dbt · 4 steps · unrated
Implement Flink temporal joins and interval joins for enriching event streams with dimension tables
nightlies.flink.apache.org · 6 steps · unrated

Give your agent this knowledge — and 15,600+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans