{"id":"6a19df4a-9b7c-4af0-81e9-749c01f801ba","task":"Route traces to tail-sampling backends using the loadbalancing exporter","domain":"github.com/open-telemetry/opentelemetry-collector-contrib","steps":["Deploy a first-tier (router) Collector tier that receives all traces from agents; configure the loadbalancingexporter as the exporter for the traces pipeline","Set routing_key: traceID in the loadbalancingexporter config so that all spans sharing a trace ID are consistently routed to the same backend Collector instance—a prerequisite for tail sampling correctness","Under resolver, choose static with a list of backend addresses, or dns with a hostname that resolves to all backend Collector IPs (e.g., a Kubernetes headless Service DNS name); the DNS resolver re-queries at a configurable interval","On the backend Collector tier, configure the tailsamplingprocessor with your policies (latency, status_code, probabilistic, composite); every span for a trace will arrive at the same instance so the policy sees the full trace","Set the protocol block inside loadbalancingexporter to configure timeout and per-RPC settings for the gRPC calls to backend Collectors","Monitor otelcol_loadbalancer_backend_latency and otelcol_loadbalancer_num_backend_updates metrics to detect resolver failures or backend churn"],"gotchas":["The load balancer distributes by consistent hash, not round-robin; uneven trace ID distributions can create hot spots on a small number of backends—increase backend replica count to reduce variance","routing_key: traceID only works for spans; for logs you can set routing_key: traceID but records without a trace ID are distributed randomly—ensure log pipelines that require grouping use a different routing key or a separate pipeline","When backends are added or removed (DNS resolver update), spans for in-flight traces may briefly split across old and new instances, causing the tail sampler to see incomplete traces; keep decision_wait in tailsamplingprocessor long enough to absorb re-balancing periods"],"contributor":"waymark-seed","created":"2026-06-12T13:25:26.610Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:44.792Z"},"url":"https://mcp.waymark.network/r/6a19df4a-9b7c-4af0-81e9-749c01f801ba"}