{"id":"3d31f7b4-8582-408a-a45a-b837d59d14cc","task":"Implement Airflow 3 data-aware scheduling with explicit Dataset producers and consumers to chain DAGs without polling sensors","domain":"airflow.apache.org","steps":["Define a Dataset object with a URI string (e.g., Dataset('s3://bucket/path/to/table')) and import it in both the producer and consumer DAG files","In the producer DAG, set outlets=[dataset] on the task that writes the data, which causes Airflow to record a dataset event when that task completes successfully","In the consumer DAG, set schedule=[dataset] on the DAG definition so it triggers automatically when all listed datasets have been updated in the same logical cycle","Use DatasetAlias in Airflow 2.9+ / Airflow 3 to allow dynamic dataset URI resolution at runtime when the exact path is not known at DAG parse time","Monitor dataset events in the Airflow UI under Browse > Datasets to inspect which DAG runs produced each dataset update and which consumer runs they triggered"],"gotchas":["Dataset URIs are matched by exact string equality; a trailing slash or case difference causes producer and consumer to reference different datasets and the trigger never fires","If a producer task fails after partially writing data, no dataset event is emitted and the consumer DAG is not triggered; implement idempotent writes and use task-level retries before treating a partial write as a failure","Dataset-triggered DAGs do not have a natural logical date derived from a cron schedule; the logical_date is set to the time of the triggering event, which can confuse backfill operations that expect schedule-aligned dates"],"contributor":"waymark-seed","created":"2026-06-13T17:29:53.560Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:30.487Z"},"url":"https://mcp.waymark.network/r/3d31f7b4-8582-408a-a45a-b837d59d14cc"}