Add memory_limiter as the first processor in every pipeline; set limit_mib or limit_percentage (e.g., 80% of container memory limit) and spike_limit_percentage (e.g., 25% of limit) with check_interval: 1s
Set the GOMEMLIMIT environment variable on the Collector to approximately 80% of the hard container memory limit so the Go runtime's GC triggers before the memory limiter's hard limit is breached
Place the batch processor after memory_limiter in the pipeline; configure send_batch_size (e.g., 8192) and timeout (e.g., 10s) — batches are sent when either threshold is reached first
For very high throughput, set send_batch_max_size to cap the maximum batch size and prevent oversized payloads from overwhelming downstream exporters
Monitor otelcol_processor_refused_spans (or the equivalent metric for logs/metrics) to detect when the memory limiter is actively backpressuring; this signals the Collector is under memory pressure
Scale horizontally by deploying additional Collector replicas behind a load balancer rather than increasing per-instance memory limits beyond the point where GC pause latency becomes noticeable
Known gotchas
If memory_limiter is not the first processor, other processors may have already buffered data by the time the memory limit is hit, causing the Collector to refuse new data while still holding large in-memory buffers
Setting limit_percentage too close to 100% leaves no headroom for GC pause spikes; the recommended practice is limit_percentage around 80 with spike_limit_percentage around 65
The batch processor's timeout is a wall-clock timer; if upstream telemetry is bursty, batches may be sent before reaching send_batch_size — tune timeout to match your acceptable export latency versus throughput tradeoff
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp