{"id":"197bf248-fbac-41ba-8a1f-203b2a7e33a7","task":"Snowflake Streams and Tasks: build an automated incremental processing pipeline","domain":"docs.snowflake.com","steps":["Create a stream on a source table with CREATE STREAM my_stream ON TABLE source_table to capture INSERT, UPDATE, and DELETE changes since the last consumption offset","Create a task with CREATE TASK my_task SCHEDULE = '1 minute' AS INSERT INTO target_table SELECT ... FROM my_stream WHERE SYSTEM$STREAM_HAS_DATA('my_stream')","Grant the task the EXECUTE TASK privilege and resume it with ALTER TASK my_task RESUME","Monitor task history in the TASK_HISTORY view under the ACCOUNT_USAGE schema to verify successful runs and inspect error messages on failures","Handle the case where the stream's offset is not advanced when the consuming query DML fails; confirm that Snowflake rolls back the stream offset on transaction failure so no changes are lost"],"gotchas":["A stream's offset is only advanced when a DML statement consuming the stream commits successfully within a transaction; a failed or rolled-back DML does not advance the offset and the same changes will be re-processed on the next run","Streams on shared tables (data sharing) are read-only and cannot be consumed by tasks in the consumer account; streams must be created on local copies","Task scheduling is approximate; Snowflake does not guarantee sub-minute precision on task execution timing, and tasks may be delayed under heavy warehouse load or during Snowflake maintenance windows"],"contributor":"waymark-seed","created":"2026-06-13T07:22:33.576Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/197bf248-fbac-41ba-8a1f-203b2a7e33a7"}