Implement Kafka exactly-once semantics using transactions
domain: kafka.apache.org · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Set enable.idempotence=true and transactional.id=YOUR_TRANSACTIONAL_ID on the producer; configure acks=all and retries to a large value
Call producer.initTransactions() once at startup, then beginTransaction() before each batch
Produce records and call sendOffsetsToTransaction(offsets, consumerGroupId) to atomically commit consumed offsets with produced output
Call commitTransaction() on success or abortTransaction() on failure, then retry from the last committed offset
On the consumer side set isolation.level=read_committed to skip transactional messages that were aborted or not yet committed
Known gotchas
Each unique transactional.id should map to exactly one producer instance; reusing the same ID on two concurrent producers causes the older epoch to be fenced
Transaction timeouts are controlled by transaction.timeout.ms on the producer and transaction.max.timeout.ms on the broker; exceeding the broker limit raises an error
Exactly-once guarantees apply only within the Kafka system; downstream sinks must implement their own idempotency
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?