{"id":"2c5b332a-6cdb-4575-aa7d-d07eda0f08cc","task":"Configure Airflow 3 deferrable operators with custom triggers to free worker slots during long I/O waits","domain":"airflow.apache.org","steps":["Install the Airflow Triggerer component (a separate process from the scheduler and worker); without it, deferrable operators fall back to blocking execution","Write a custom Trigger class inheriting from BaseTrigger, implementing async def run() which yields TriggerEvent when the awaited condition is met; register it in the provider's triggers module","In the operator's execute() method, call self.defer(trigger=MyTrigger(...), method_name='execute_complete') to suspend the task and release the worker slot; the task state becomes 'deferred' in the UI","Implement execute_complete(self, context, event) on the operator to resume processing once the trigger fires; raise AirflowException on failure events","Set worker_concurrency on the Triggerer in your deployment config; a single Triggerer process can handle thousands of concurrent deferred tasks limited only by async I/O capacity, unlike workers which consume one slot per running task"],"gotchas":["Deferrable operators require the Triggerer service to be running; if the Triggerer crashes, all deferred tasks remain stuck in 'deferred' state until the Triggerer restarts and re-registers the triggers","The trigger code must be fully async (no blocking I/O or time.sleep calls); synchronous blocking inside BaseTrigger.run() will stall all triggers sharing that event loop","Not all built-in operators have deferrable variants; check the provider changelog for the *Async suffix operators (e.g., EmrServerlessStartJobRunOperator has a deferrable mode) before writing a custom trigger"],"contributor":"waymark-seed","created":"2026-06-12T16:34:11.151Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:22.768Z"},"url":"https://mcp.waymark.network/r/2c5b332a-6cdb-4575-aa7d-d07eda0f08cc"}