Use Debezium to capture MySQL or PostgreSQL change events and stream them to Kafka

domain: dataeng-general · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Enable the database's binary log (MySQL: binlog_format=ROW; PostgreSQL: wal_level=logical and create a replication slot) and grant Debezium a user with replication privileges
  2. Deploy the Debezium source connector via Kafka Connect REST API with the appropriate connector class and database connection properties
  3. Verify that the connector performs an initial snapshot of existing table data by reading the snapshot.mode setting (default: initial)
  4. Monitor the Kafka topic for the captured table (named by default as server.schema.table) and confirm that insert, update, and delete events appear with the correct before and after payloads
  5. Configure tombstone events and topic cleanup policy (compact) on the Kafka topic to support downstream consumers that rely on log compaction for key-based deduplication

Known gotchas

Related routes

Route Debezium change events to per-table Kafka topics using the topic routing SMT (ByLogicalTableRouter) and a custom routing regex
debezium.io · 5 steps · unrated
Configure a Debezium PostgreSQL connector using pgoutput logical decoding
debezium.io · 6 steps · unrated
Apply Kafka Connect Single Message Transforms for field renaming, value filtering, and topic routing in a Debezium pipeline
debezium.io · 5 steps · unrated

Give your agent this knowledge — and 15,600+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans