Choose an offloader driver (AWS S3, GCS, Azure Blob, or filesystem) and configure the driver class and credentials in broker.conf: managedLedgerOffloadDriver and related driver-specific properties such as s3ManagedLedgerOffloadBucket
Set the offload threshold at broker level via managedLedgerOffloadThresholdInBytes; when a topic's total retained data exceeds this threshold the broker automatically offloads older ledgers
Set the namespace-level offload deletion lag: pulsar-admin namespaces set-offload-deletion-lag --lag <seconds> my-tenant/my-namespace to control how long after offload the original BookKeeper data is retained as a safety buffer
Configure retention policy separately from offload; retention controls how long data is kept in tiered storage after being consumed, while offload controls when it moves from BookKeeper to object storage
Verify offloaded segments: pulsar-admin topics offload-status persistent://tenant/namespace/topic shows which ledgers are offloaded and their object storage keys
Known gotchas
Offloaded data is still accessible to consumers transparently, but reads from object storage are significantly slower than BookKeeper reads; consumers catching up from very old offloaded data will experience higher latency
The offload deletion lag ensures BookKeeper data is not deleted immediately after offload; setting it too low risks data loss if the object storage write completes but the confirmation is not received before the BookKeeper data is purged
Credentials for object storage access are configured in broker.conf in plaintext by default; use a secrets manager integration or environment variable injection to avoid committing credentials to config files
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp