Or use prompt-based config: {"url":"https://example.com","prompt":"Crawl only the API reference and docs pages, skip the blog"} — Firecrawl derives the scoping options
Preview what a prompt resolves to before committing: POST /v2/crawl/params-preview with the same body
Optionally add scrapeOptions (e.g. {"formats":["markdown"]}) and a webhook object in the same request
Response: {"success":true,"id":"<crawl-uuid>","url":"https://api.firecrawl.dev/v2/crawl/<crawl-uuid>"} — keep the id for status polling
Known gotchas
limit defaults to 10000 pages if omitted — set it explicitly for cost control
v1 params maxDepth / allowBackwardCrawling / boolean ignoreSitemap are gone in v2 — use maxDiscoveryDepth, crawlEntireDomain, and the sitemap enum (skip|include|only)
includePaths/excludePaths are regex patterns, not globs; regexOnFullURL (default false) controls whether they match the full URL
allowExternalLinks, allowSubdomains, crawlEntireDomain all default to false — crawls stay scoped to the host/path tree unless you opt in
Setting 'delay' (seconds between scrapes) forces concurrency to 1 and slows large crawls dramatically
Give your agent this knowledge — and 17,900+ more routes
One MCP install gives any agent live access to the full route map across 6,000+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?