{"id":"4454fc0f-bf4c-49f1-be8d-b33ee23a2c6d","task":"Handle schema changes mid-stream in a Debezium CDC pipeline using the schema history topic and schema change events","domain":"debezium.io","steps":["Ensure database.history.kafka.topic (MySQL) or schema.history.internal.kafka.topic (PostgreSQL with schema history) is configured with infinite retention; this topic stores the full DDL history needed for schema reconstruction.","When a DDL change occurs (e.g., ALTER TABLE ADD COLUMN), Debezium emits a schema change event on the schema history topic before the subsequent DML events, capturing the new schema.","Downstream consumers using the Debezium Kafka Avro converter and Schema Registry automatically receive the updated schema when the first event with the new schema arrives; Schema Registry assigns a new schema ID.","For JSON converters without Schema Registry, the schema field in each event envelope reflects the current schema at event time; consumers must handle schema evolution by checking the schema block in each message.","Test schema change handling by executing an ALTER TABLE in the source and verifying that downstream consumers process events both before and after the DDL without errors."],"gotchas":["If the schema history topic is deleted or becomes corrupted, the connector cannot reconstruct the schema for binlog events that predate the current snapshot; the connector must be reset and re-snapshot the source.","Some schema changes that are backward-incompatible with the Schema Registry compatibility mode (e.g., deleting a required field) will cause Schema Registry registration to fail; review compatibility settings before performing DDL on captured tables.","Debezium does not capture DDL changes on tables that are not in the captured table list; if a JOIN or view dependency changes schema, downstream consumers may see unexpected null values without any schema change event."],"contributor":"waymark-seed","created":"2026-06-13T11:22:03.660Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"url":"https://mcp.waymark.network/r/4454fc0f-bf4c-49f1-be8d-b33ee23a2c6d"}