{"id":"f059d748-1082-414b-9c59-ce5adaf60921","task":"Avoid duplicate task execution in Celery when using SQS as the broker by tuning visibility_timeout against task runtime","domain":"celery","steps":["Set broker_transport_options = {'visibility_timeout': <seconds>} in the Celery config — this only takes effect when Celery creates the SQS queue itself, not when using a pre-existing queue","Know the documented default is 1800 seconds (30 minutes) and the AWS/SQS-imposed ceiling is 43200 seconds (12 hours)","Set visibility_timeout to at least the longest expected task runtime, including any ETA/countdown delay, since a task that runs past the visibility timeout becomes eligible for SQS redelivery to another worker","Keep task_time_limit (or task_soft_time_limit) comfortably under visibility_timeout so a legitimately slow task doesn't get redelivered and double-executed before it finishes","Recognize that Celery redelivers messages on worker shutdown regardless of visibility_timeout — a long timeout only delays redelivery after a crash, it doesn't prevent duplicate execution in that scenario","Make task logic idempotent as a backstop, since visibility_timeout tuning reduces but does not eliminate the chance of duplicate delivery"],"gotchas":["visibility_timeout only applies at queue-creation time — changing it in config has no effect on a queue that already exists","A long ETA/countdown delay exceeding the visibility timeout can cause a task to be redelivered and picked up again before it's even due to run, per Celery's own issue tracking","Worker shutdown triggers redelivery independent of visibility_timeout, so tuning this setting alone doesn't guarantee exactly-once execution"],"contributor":"waymark-seed","created":"2026-07-09T17:43:57.795Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/f059d748-1082-414b-9c59-ce5adaf60921"}