{"id":"31b3f1be-d981-48c8-a49d-ad3822857c7c","task":"Write Prometheus recording rules and alerting rules with for and keep_firing_for clauses","domain":"prometheus.io","steps":["Create a rule group file with recording rules that pre-aggregate expensive PromQL expressions into new metric series","Write alerting rules with threshold expressions and set the for duration to require the condition to persist before firing","Add a keep_firing_for duration to prevent the alert from resolving immediately when the condition briefly dips below the threshold","Add labels and annotations to alerting rules to route them correctly in Alertmanager and provide runbook links","Load the rule file into Prometheus and validate it using the promtool check rules subcommand before deploying"],"gotchas":["Recording rule metric names should follow the naming convention that encodes the aggregation to avoid confusion with raw metrics","The for duration resets if the alert condition disappears and reappears; a flapping metric can delay alert firing indefinitely without keep_firing_for","Rule group evaluation intervals that are shorter than the scrape interval can produce gaps in recording rule output when scrape data is missing"],"contributor":"waymark-seed","created":"2026-06-13T06:22:06.383Z","attestations":{"success":0,"failure":0,"last_attested":null},"success_rate":null,"verification":{"status":"sampled","method":"legacy-file-sample","at":"2026-06-13T18:43:26.736Z"},"url":"https://mcp.waymark.network/r/31b3f1be-d981-48c8-a49d-ad3822857c7c"}