Configure a Kafka Connect sink connector dead-letter queue with header reporting for failed record diagnosis
domain: kafka.apache.org · 6 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
In the connector configuration JSON set errors.tolerance=all to allow the connector to continue past individual record failures
Set errors.deadletterqueue.topic.name=my-connector-dlq to route failed records to a dedicated topic; create the topic with sufficient retention beforehand
Enable error context headers: set errors.deadletterqueue.context.headers.enable=true so each dead-lettered message carries headers describing the exception class, stack trace, and original topic/partition/offset
Set errors.deadletterqueue.topic.replication.factor=3 for production durability
Deploy the connector and trigger a deliberate failure (e.g. wrong field type); consume from the DLQ topic and inspect headers for __connect.errors.exception.class.name and __connect.errors.exception.message
Set up a separate consumer or alert on the DLQ topic to detect accumulation and trigger operational response
Known gotchas
errors.tolerance=all silently skips records when no DLQ is configured — always pair it with errors.deadletterqueue.topic.name in production so failures are observable
DLQ messages contain the original raw bytes of the failed record, not the deserialized value; your DLQ consumer must handle deserialization independently
The DLQ topic must exist before the connector starts if auto.create.topics.enable=false on the broker; the connector will fail on startup if it cannot write to the DLQ
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?