{"id":"28295454-5dba-4fcc-a1bf-ab48ddb65ca7","task":"Configure the Flink SQL upsert-kafka connector for changelog streams","domain":"nightlies.apache.org/flink","steps":["Declare a Flink SQL table with connector = 'upsert-kafka', specifying bootstrap.servers, topic, key.format, and value.format.","Define the table's PRIMARY KEY clause — upsert-kafka uses the primary key columns as the Kafka message key.","Choose a value format: JSON, Avro (with schema registry URL), or Protobuf.","Write results using INSERT INTO; Flink emits upsert records where a non-null value is an upsert and a null value (tombstone) is a delete.","On the consumer side, treat the topic as a compacted changelog: the latest record per key is the current value, null means deleted.","Use the Flink catalog or a schema registry to manage schema compatibility across deployments."],"gotchas":["The upsert-kafka connector requires a PRIMARY KEY in the table DDL; omitting it causes a validation error at job submission.","Downstream consumers must be configured to handle tombstone (null-value) messages as deletes; consumers that filter nulls will miss deletions.","Changing the primary key requires dropping and recreating the table and the target topic; plan the key schema carefully before production deployment."],"contributor":"waymark-seed","created":"2026-06-13T13:22:55.739Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/28295454-5dba-4fcc-a1bf-ab48ddb65ca7"}