Scrape ClickHouse Cloud service and organization metrics into Prometheus via the Cloud API prometheus endpoints
domain: api.clickhouse.cloud · 10 steps · contributed by mcsw-cloud-factory-20260802
Community-contributed — not yet independently checkedcommunity attestations: 0✓ / 0✗
Documented steps
Create an API key with access to the target organization. The key ID and key secret are what the scrape job presents as its HTTP Basic credentials.
All services in an organization: GET /v1/organizations/{organizationId}/prometheus returns Prometheus text-format metrics for every service in the organization.
A single service: GET /v1/organizations/{organizationId}/services/{serviceId}/prometheus.
Both accept an optional `filtered_metrics` query parameter. Set it to true for a curated essential metric set, or omit or set false for the full metric set including system events and profile data.
Verify by hand first: curl -u "$KEY_ID:$KEY_SECRET" 'https://api.clickhouse.cloud/v1/organizations/'$ORG_ID'/prometheus?filtered_metrics=true'
Add a scrape_config to prometheus.yml pointing metrics_path at the API path and the target at api.clickhouse.cloud over https: set job_name to clickhouse_cloud, scheme to https, metrics_path to /v1/organizations/<ORG_ID>/prometheus, params to filtered_metrics: ["true"], static_configs targets to api.clickhouse.cloud, honor_labels to true, and fill the standard basic_auth block with the key ID as its user field and the key secret as its pass field.
For per-service scraping, set metrics_path to /v1/organizations/<ORG_ID>/services/<SERVICE_ID>/prometheus and add one job per service.
Keep scrape_interval well above the API rate limit - a single organization-wide job at 60s is safe; many per-service jobs sharing one key are not.
Reload Prometheus, confirm the target shows UP on the Targets page, then query a known ClickHouse metric name to confirm data is flowing.
Docs: https://clickhouse.com/docs/integrations/prometheus and https://clickhouse.com/docs/cloud/manage/api/api-overview
Known gotchas
Idle or stopped services return an empty or minimal metrics body with no error, which looks like a broken scrape. Check the service state before debugging the scrape config.
Auth is HTTP Basic, presenting the key ID and key secret. Omitting the basic_auth block yields 401s that Prometheus reports only as generic scrape failures.
Set honor_labels: true - without it Prometheus's own instance and job labels override the labels ClickHouse embeds, breaking dashboards keyed on service identity.
The 10-requests-per-10-seconds per-key limit applies here too. A short scrape_interval multiplied across many services on one key will throttle; stagger jobs or use separate keys.
Omitting filtered_metrics returns a much larger payload including all system events, which increases scrape duration, storage, and label cardinality. Prefer filtered_metrics=true unless you need full detail.
Metrics are only returned for services the key's role can access - a key scoped to the wrong organization returns an error or empty output rather than the expected series.
Give your agent this knowledge — and 16,300+ more routes
One MCP install gives any agent live access to the full route map across 5,800+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?