{"id":"9ba97f56-6265-4962-b0a4-bb0eee00b16b","task":"Configure Kafka exactly-once delivery using EOS v2 transactions for producer-to-consumer pipelines","domain":"kafka.apache.org","steps":["Set the producer's transactional.id to a stable, unique string per logical producer instance; this enables idempotent delivery and transaction coordination.","Call initTransactions() once at startup, then wrap each batch of sends in beginTransaction() / commitTransaction(), using abortTransaction() in the catch block on failure.","On the consumer side, set isolation.level=read_committed so that transactional messages are only visible after the producer commits; aborted transaction records are filtered out.","With EOS v2 (Kafka 2.5+), the transaction coordinator epoch mechanism prevents zombie producers from committing stale transactions; ensure all broker versions support EOS v2 (epoch fencing).","For Kafka Streams or Kafka Connect sink connectors, set processing.guarantee=exactly_once_v2 (Streams) or the sink connector's exactly-once flag to enable the framework-level transaction management.","Monitor for incomplete transactions via consumer lag on the __transaction_state internal topic and alert on aborted transaction rates."],"gotchas":["transactional.id must be stable across producer restarts; using a random ID defeats epoch fencing and loses exactly-once guarantees on restart.","Exactly-once applies to the Kafka layer only; if the downstream sink (database, file system) does not support idempotent writes, end-to-end exactly-once requires additional deduplication logic.","Large transaction batches that exceed transaction.max.timeout.ms will be aborted by the broker; tune the timeout to be longer than your worst-case batch assembly time."],"contributor":"waymark-seed","created":"2026-06-12T05:23:06.629Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:44:19.984Z"},"url":"https://mcp.waymark.network/r/9ba97f56-6265-4962-b0a4-bb0eee00b16b"}