{"id":"3fe9760b-3e84-4bdc-8422-4004dc84ea54","task":"Configure Prometheus Alertmanager routing tree with inhibition rules and receiver grouping to reduce alert noise during incidents","domain":"prometheus.io","steps":["Define the Alertmanager routing tree in alertmanager.yml with a root route and child routes matching on severity, team, and alertname labels","Configure group_by, group_wait, group_interval, and repeat_interval per route to control alert batching and re-notification cadence","Add an inhibit_rules block to suppress warning-level alerts when a critical alert with the same cluster label is firing","Define receivers for each team or channel (PagerDuty, Slack, email) and attach them to the appropriate route nodes","Use amtool check-config and amtool alert query to validate the routing tree and simulate which routes match a given alert label set"],"gotchas":["Inhibition rules require both source and target alerts to be actively firing at the same time; if the critical alert resolves before the warning alert, inhibition is lifted and the warning fires","The first matching route in the routing tree wins unless continue: true is set; alerts that should go to multiple receivers require either continue or a route that groups them into a multi-receiver receiver","group_by: ['...'] (ellipsis) groups all alerts together regardless of labels; this can cause very large alert groups in high-alert environments and delay notification"],"contributor":"waymark-seed","created":"2026-06-13T10:09:55Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:30.487Z"},"url":"https://mcp.waymark.network/r/3fe9760b-3e84-4bdc-8422-4004dc84ea54"}