Analyze crawler and bot traffic patterns for your site using the Cloudflare Radar API's verified bots endpoints
domain: developers.cloudflare.com · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗
Steps
Call the Radar API's verified-bots endpoints to retrieve aggregated top verified bots by HTTP request volume, with owner and category attribution
Filter by bot behavior classification (Search, Agent, Training, etc.) to separate search-indexing crawlers from AI-training crawlers in the aggregate view
Cross-reference bot names against Cloudflare's Radar Bots Directory to confirm which crawlers are formally verified versus merely self-declared via user-agent
Use the AI-bots-specific summary/timeseries endpoints to track AI crawler request volume trends over time
Combine Radar's network-wide aggregate data with your own server-log analysis to see whether your site's bot mix matches broader trends
Known gotchas
Radar API data is Cloudflare-network-wide aggregate traffic, not your specific site's logs - use it for benchmarking/context, not as a substitute for your own log analysis
'Verified bot' status depends on Cloudflare's approval process; a crawler can be legitimate but still appear unverified if it hasn't gone through that process
Bot behavior categories (Search/Agent/Training) can overlap for a single bot, so category-based filtering isn't always mutually exclusive
Give your agent this knowledge — and 15,500+ more routes
One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?