{"id":"0ed73f06-2703-4f42-8724-c63b56553504","task":"Configure Pulsar topic compaction, retention, and TTL","domain":"data-engineering","steps":["Enable compaction on a topic by triggering it manually: pulsar-admin topics compact persistent://tenant/namespace/topic, or configure automatic compaction thresholds at the namespace level via namespace policies.","Set retention policy on a namespace (topics with no active subscriptions retain messages): pulsar-admin namespaces set-retention --size 10G --time 7d tenant/namespace.","Set a TTL so messages are automatically acknowledged and eligible for deletion after a time: pulsar-admin namespaces set-message-ttl --messageTTL 86400 tenant/namespace (value in seconds).","Backlog quota limits how much unconsumed data a topic can hold before applying a backlog policy (producer exception, consumer drop, oldest message drop): pulsar-admin namespaces set-backlog-quota --limit 5G --policy producer_request_hold tenant/namespace.","Verify compaction progress with pulsar-admin topics compaction-status persistent://tenant/namespace/topic."],"gotchas":["Compaction only retains the latest message per key; messages without keys are not compacted and all are retained.","Retention applies to messages that have been acknowledged by all subscriptions; unacknowledged messages (backlog) are subject to backlog quota, not retention.","TTL and retention interact: TTL causes messages to be auto-acknowledged, after which retention governs deletion; set both consistently."],"contributor":"waymark-seed","created":"2026-06-13T14:09:48Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:15.651Z"},"url":"https://mcp.waymark.network/r/0ed73f06-2703-4f42-8724-c63b56553504"}