Add an outbox table to the source database (e.g., outbox_events with columns: id UUID, aggregate_type VARCHAR, aggregate_id VARCHAR, event_type VARCHAR, payload JSONB, created_at TIMESTAMP).
In application code, write business entity changes and outbox rows in the same local database transaction; this guarantees atomicity — either both are committed or neither is.
Configure Debezium to capture only the outbox table and apply the OutboxEventRouter SMT: transforms.outbox.type=io.debezium.transforms.outbox.EventRouter, which routes events to Kafka topics based on aggregate_type.
The outbox table should be captured with a tombstone on delete (or truncated periodically) to prevent unbounded growth; use DELETE after confirmed Kafka acknowledgement or truncate via a scheduled job.
On the consumer side, implement idempotent processing keyed on the event id (UUID) to handle the at-least-once delivery guarantee of Kafka and prevent double-processing on retries.
Known gotchas
The outbox pattern achieves exactly-once write atomicity between the database and Kafka, but Kafka itself provides at-least-once delivery to consumers; consumers must still implement idempotency to achieve true end-to-end exactly-once semantics.
Polling the outbox table via Debezium CDC is preferable to application-side polling because it leverages the WAL/binlog, avoiding the need for a separate polling thread that can miss events or create ordering issues.
Outbox table growth must be managed; if DELETE or TRUNCATE statements run while Debezium is capturing the outbox, ensure tombstone handling is configured correctly so downstream consumers process the delete events without errors.
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp