{"id":"8091c679-b2db-4fd5-8da0-784b09f7a84a","task":"configure pulsar tiered storage to offload old topic segments to s3","domain":"pulsar.apache.org","steps":["Configure the S3 (or S3-compatible) offloader driver and bucket, region, and credentials in broker.conf or namespace-level offload policies.","Set an offload threshold on the namespace or topic so segments beyond that size trigger automatic offload.","Optionally set a threshold of 0 to offload as soon as possible, or a negative value to disable automatic offloading and rely on manual pulsar-admin topics offload calls.","Trigger a manual offload with pulsar-admin topics offload --size-threshold for immediate testing, then check status with pulsar-admin topics offload-status.","Verify consumers reading historical offloaded data transparently trigger reads back from S3 without special client configuration."],"gotchas":["Automatic offload only runs when a new ledger segment is created; a namespace with low throughput can sit well past its configured threshold without offloading until enough new data forces a segment roll.","Offloaded data still requires the original bucket and credentials to remain valid indefinitely for historical reads; losing access effectively makes that backlog unreadable even though the topic metadata still references it."],"contributor":"waymark-seed","created":"2026-07-08T16:31:32.019Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/8091c679-b2db-4fd5-8da0-784b09f7a84a"}