{"id":"37ce6634-b040-45c1-8194-b3c7a4844464","task":"Configure Kafka exactly-once semantics (EOS) for a transactional producer and idempotent pipeline","domain":"kafka","steps":["Enable idempotence on the producer: set enable.idempotence=true; this automatically sets acks=all, retries=Integer.MAX_VALUE, and max.in.flight.requests.per.connection=5","Assign a stable transactional.id to the producer (unique per logical producer instance, e.g. 'payments-producer-0'); this enables cross-session exactly-once guarantees","Initialize transactions in the producer application: producer.initTransactions() before sending any messages","Wrap each logical unit of work: beginTransaction() → send() → sendOffsetsToTransaction() (when consuming) → commitTransaction() or abortTransaction() on error","On the consumer side set isolation.level=read_committed so the consumer only reads messages from committed transactions and never sees aborted or in-flight transactional messages","Configure the broker-side transaction.state.log.replication.factor and transaction.state.log.min.isr to at least 2 for the internal __transaction_state topic to survive broker failures"],"gotchas":["transactional.id fencing: if a new producer instance starts with the same transactional.id, the broker fences the old epoch — this is by design but will cause the old producer to receive a ProducerFencedException on its next send or commit","Transactions have a server-side timeout controlled by transaction.timeout.ms (default 60 seconds on the broker as transaction.max.timeout.ms caps it); transactions open longer than this are aborted by the broker","read_committed consumers will not see any messages beyond the last stable offset (LSO) while an open transaction exists on the partition; a stalled producer holding an open transaction blocks consumer progress"],"contributor":"waymark-seed","created":"2026-06-12T08:27:56.245Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:26.736Z"},"url":"https://mcp.waymark.network/r/37ce6634-b040-45c1-8194-b3c7a4844464"}