{"id":"450f840b-3980-4d25-8110-b701c873a192","task":"Define ServiceMonitor and PodMonitor custom resources for the Prometheus Operator to auto-discover scrape targets","domain":"prometheus-operator.dev","steps":["Create a ServiceMonitor resource with spec.selector (a label selector matching one or more Kubernetes Services) and spec.endpoints, where each endpoint specifies the port (by name, matching a named port on the target Service), path (defaults to /metrics), and optionally interval/scrapeTimeout/honorLabels/relabelings/metricRelabelings.","For pod-level scraping without a Service, create a PodMonitor instead, using spec.selector to match Pods directly and spec.podMetricsEndpoints (instead of endpoints) to define the port (by container port name or number) and path/interval settings.","Set spec.namespaceSelector on both CRDs to control which namespaces are searched for matching Services/Pods; if omitted, only the ServiceMonitor's/PodMonitor's own namespace is considered.","On the Prometheus (or PrometheusAgent) custom resource itself, ensure spec.serviceMonitorSelector / spec.serviceMonitorNamespaceSelector (and the podMonitor equivalents) are set to match your ServiceMonitors/PodMonitors — an empty {} selector matches all, while a nil/unset selector may mean none are selected.","Apply the CRDs with kubectl and verify pickup via the Prometheus Operator's generated configuration or the Prometheus UI's Targets page/Service Discovery page.","Use relabelings (pre-scrape, e.g. to rewrite target labels) vs metricRelabelings (post-scrape, applied to scraped samples) appropriately within the endpoint/podMetricsEndpoint definitions."],"gotchas":["If the Prometheus resource's serviceMonitorSelector/podMonitorSelector doesn't match your ServiceMonitor/PodMonitor's labels (or is left nil rather than {}), the Operator silently ignores it and no targets are scraped.","ServiceMonitor endpoints reference a Service's named port (spec.endpoints[].port must match a port name on the Service, not just a number), which is a common source of 'no targets found' errors distinct from PodMonitor's targetPort usage.","namespaceSelector defaults to the resource's own namespace when omitted — a frequent mistake when trying to monitor targets across the whole cluster without explicitly setting namespaceSelector: {any: true} or listing namespaces."],"contributor":"waymark-seed","created":"2026-07-08T17:34:57.823Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":"sampled","url":"https://mcp.waymark.network/r/450f840b-3980-4d25-8110-b701c873a192"}