{"id":"6d0b1a74-726e-4e6d-93b7-0f85d264b08e","task":"NVIDIA Triton Inference Server: monitor server and per-model metrics via the built-in Prometheus metrics endpoint","domain":"ml-ops","steps":["Start `tritonserver` normally — Prometheus metrics are exposed by default at `http://localhost:8002/metrics` as plain text; nothing is pushed to a remote server, you must scrape/pull it","View or scrape metrics with `curl localhost:8002/metrics`, or point a Prometheus scrape config at that endpoint","Adjust exposure with CLI flags: `--metrics-port` to change the port, `--metrics-address` to bind a specific address, `--allow-metrics=false` to disable all metrics, `--allow-gpu-metrics=false`/`--allow-cpu-metrics=false` to disable just GPU/CPU metrics, and `--metrics-interval-ms` to change how often per-interval metrics are polled","Track core per-model counters such as `nv_inference_request_success`/`_failure`, `nv_inference_count`, `nv_inference_exec_count`, `nv_inference_pending_request_count`, and latency counters like `nv_inference_request_duration_us` and `nv_inference_queue_duration_us`","Track resource gauges such as `nv_gpu_utilization`, `nv_gpu_power_usage`, `nv_gpu_memory_used_bytes` (collected via DCGM), and `nv_cpu_utilization`/`nv_cpu_memory_used_bytes`","Optionally enable richer latency distributions with `--metrics-config histogram_latencies=true` or `--metrics-config summary_latencies=true` (both experimental) for percentile-level detail beyond the default counters"],"gotchas":["GPU metrics depend on DCGM and require Triton to be built with GPU metrics support (`TRITON_ENABLE_METRICS_GPU`) plus `--allow-gpu-metrics` left enabled","Histogram and Summary latency metric types are explicitly documented as experimental and may change in future releases","Request Count, Inference Count, and Execution Count diverge once dynamic batching kicks in — use Inference Count / Execution Count together (not Request Count alone) to compute real effective batch size and throughput"],"contributor":"waymark-seed","created":"2026-07-09T00:09:27Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/6d0b1a74-726e-4e6d-93b7-0f85d264b08e"}