Push logs to Loki's ingest endpoint using the Loki push API (POST /loki/api/v1/push) with a JSON body containing streams, each stream identified by a set of label key-value pairs and an array of log entries
Choose low-cardinality labels only: environment, cluster, namespace, app, and job are appropriate; avoid dynamic values like user_id, request_id, or pod_ip as label values since each unique combination creates a separate stream
Store high-cardinality values (trace IDs, user IDs, request paths) as structured metadata or embedded in the log line itself, then query them with LogQL's | json, | logfmt, or | pattern parsers at query time
Configure per-tenant stream limits in Loki's limits_config: set max_streams_per_user and max_global_streams_per_user to hard limits; set max_label_names_per_series (Loki defaults to 15) to prevent label explosion
Use Loki's cardinality analysis endpoint or the Grafana Explore label browser to identify streams with unexpectedly high cardinality; drop or replace high-cardinality label values in the log shipper before they reach Loki
Enable automatic stream sharding (ingester_streams_shard_factor in Loki config) so Loki can internally split very high-volume streams across ingesters without requiring upstream label changes
Known gotchas
Each unique label set creates a separate TSDB chunk file; too many streams (more than ~100 000 active streams per tenant) degrades ingester memory and compaction performance significantly
The Loki push API returns a 204 on success and a 400 with an error body on rejection (e.g., stream limit exceeded); clients that ignore non-2xx status codes silently drop logs at the limit boundary
Changing label values for an existing stream without changing the label key creates a new stream and orphans the old one in storage; schema migrations that rename labels require rewriting historical data or accepting a query-time union
Give your agent this knowledge — and 200+ more routes
One MCP install gives any agent live access to the full route map, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp