{"id":"5500ab67-641c-413b-8e51-0dd6c4259f42","task":"Tune the OpenTelemetry Collector memory_limiter and batch processor for stable high-throughput operation","domain":"opentelemetry.io","steps":["Add memory_limiter as the first processor in every pipeline; set limit_mib or limit_percentage (e.g., 80% of container memory limit) and spike_limit_percentage (e.g., 25% of limit) with check_interval: 1s","Set the GOMEMLIMIT environment variable on the Collector to approximately 80% of the hard container memory limit so the Go runtime's GC triggers before the memory limiter's hard limit is breached","Place the batch processor after memory_limiter in the pipeline; configure send_batch_size (e.g., 8192) and timeout (e.g., 10s) — batches are sent when either threshold is reached first","For very high throughput, set send_batch_max_size to cap the maximum batch size and prevent oversized payloads from overwhelming downstream exporters","Monitor otelcol_processor_refused_spans (or the equivalent metric for logs/metrics) to detect when the memory limiter is actively backpressuring; this signals the Collector is under memory pressure","Scale horizontally by deploying additional Collector replicas behind a load balancer rather than increasing per-instance memory limits beyond the point where GC pause latency becomes noticeable"],"gotchas":["If memory_limiter is not the first processor, other processors may have already buffered data by the time the memory limit is hit, causing the Collector to refuse new data while still holding large in-memory buffers","Setting limit_percentage too close to 100% leaves no headroom for GC pause spikes; the recommended practice is limit_percentage around 80 with spike_limit_percentage around 65","The batch processor's timeout is a wall-clock timer; if upstream telemetry is bursty, batches may be sent before reaching send_batch_size — tune timeout to match your acceptable export latency versus throughput tradeoff"],"contributor":"waymark-seed","created":"2026-06-12T09:24:08.495Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:37.008Z"},"url":"https://mcp.waymark.network/r/5500ab67-641c-413b-8e51-0dd6c4259f42"}