{"id":"8fdaaa5f-dd7d-480a-8efc-12c29c4a61fc","task":"Diagnose and recover from ProducerFencedException in a Kafka exactly-once pipeline","domain":"data-engineering","steps":["Recognize that ProducerFencedException means Kafka detected a newer producer instance registered under the same transactional.id — every initTransactions() call bumps an internal epoch, and any producer still using an older epoch is fenced as a zombie.","Treat it as fatal and non-recoverable for that producer instance — don't retry on the same KafkaProducer object.","Recover by closing the fenced producer, constructing a brand-new KafkaProducer with the same transactional.id, and calling initTransactions() once before any beginTransaction().","Distinguish it from InvalidTxnTimeoutException, which fires when a producer's transaction.timeout.ms (client-side, default 60000 ms) exceeds the broker's transaction.max.timeout.ms (default 900000 ms) — that's a config mismatch, not a zombie-producer event.","Audit for accidental duplicate producer instances sharing a transactional.id (e.g. two replicas briefly running during a rolling deploy) — that's the most common real-world cause of fencing."],"gotchas":["A rolling deployment that briefly runs two instances with the same transactional.id will fence the older one by design — expected exactly-once behavior, not a bug, though it looks alarming in logs.","Confusing ProducerFencedException with InvalidTxnTimeoutException leads to the wrong fix — one requires recreating the producer, the other requires reconciling client/broker timeout configs."],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/8fdaaa5f-dd7d-480a-8efc-12c29c4a61fc"}