Deploy the Thanos sidecar as a container alongside each Prometheus pod; configure it to read Prometheus data from the shared data directory and upload TSDB blocks to an object storage bucket using a storage config secret containing YOUR_KEY
Configure the sidecar's --objstore.config-file to reference a YAML file specifying the object storage type (s3, gcs, azure) and credentials via environment variable references, not inline secrets
Deploy a Thanos Store Gateway pointed at the same object storage bucket; the store gateway serves historical blocks that have been uploaded from sidecars
Deploy Thanos Query, add the sidecar gRPC endpoint addresses and the store gateway address as --store flags; Thanos Query merges results from all stores into a global view
Point Grafana at the Thanos Query HTTP endpoint as a Prometheus data source; queries spanning multiple Prometheus instances are deduplicated using the --query.replica-label flag
Deploy Thanos Compactor (as a single instance, not replicated) to compact and downsample historical blocks in object storage for faster long-range queries
Known gotchas
The Thanos Compactor must run as a single instance; running multiple compactors against the same bucket simultaneously causes block corruption — use a locking mechanism or ensure only one is deployed
Thanos deduplication requires that replica Prometheus instances scrape identical targets and that they are labeled with the same replica label; any scrape target mismatch will result in incomplete deduplication
Object storage egress costs can be significant when running Thanos Store Gateway in a different region from the bucket; co-locate the store gateway with the bucket or use caching to reduce cross-region reads
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp