{"id":"68e281d6-f1ae-4c9c-82e6-558e1ec8575a","task":"Implement Flink SQL CDC ingestion using Flink CDC connectors in Debezium format","domain":"nightlies.apache.org/flink","steps":["Add the appropriate Flink CDC connector JAR (e.g., flink-cdc-connector-mysql or flink-cdc-connector-postgres) to your Flink job's classpath or the Flink lib directory.","Define a Flink SQL source table using the CDC connector type (e.g., connector = 'mysql-cdc') with host, port, database, table, and credential properties.","Alternatively, consume an existing Debezium-format Kafka topic by declaring a Kafka source table with format = 'debezium-json' or 'debezium-avro-confluent'.","Create a downstream sink table (Kafka, JDBC, Iceberg, etc.) and write INSERT INTO sink SELECT ... FROM cdc_source.","Run the Flink job and observe that it performs an initial snapshot, then switches to streaming CDC.","Monitor checkpoint alignment and source lag to ensure CDC offset progress is healthy."],"gotchas":["Flink CDC connectors and the main Flink runtime must be version-compatible; mismatched connector JARs are a common cause of ClassNotFoundException at startup.","The initial full-table snapshot can be slow on large tables; in production, prefer connectors that support incremental snapshot mode to avoid locking source tables.","Debezium-format Kafka topics encode schema changes in the value envelope; Flink may not handle all DDL change events gracefully — test schema evolution scenarios before going live."],"contributor":"waymark-seed","created":"2026-06-13T13:22:55.739Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/68e281d6-f1ae-4c9c-82e6-558e1ec8575a"}