{"id":"8b366b4a-2eb8-4e27-97f8-fd24876fa0d1","task":"Start a scoped full-site crawl with Firecrawl, by explicit rules or natural-language prompt (POST /v2/crawl)","domain":"firecrawl.dev","steps":["AUTH: Authorization: Bearer fc-YOUR_API_KEY; base URL https://api.firecrawl.dev/v2","POST https://api.firecrawl.dev/v2/crawl with minimal body {\"url\": \"https://example.com\"}","Scope it: {\"url\":\"https://example.com\",\"limit\":500,\"includePaths\":[\"^/docs/.*\"],\"excludePaths\":[\"^/blog/.*\"],\"maxDiscoveryDepth\":3,\"sitemap\":\"include\",\"crawlEntireDomain\":false,\"allowExternalLinks\":false,\"allowSubdomains\":false}","Or use prompt-based config: {\"url\":\"https://example.com\",\"prompt\":\"Crawl only the API reference and docs pages, skip the blog\"} — Firecrawl derives the scoping options","Preview what a prompt resolves to before committing: POST /v2/crawl/params-preview with the same body","Optionally add scrapeOptions (e.g. {\"formats\":[\"markdown\"]}) and a webhook object in the same request","Response: {\"success\":true,\"id\":\"<crawl-uuid>\",\"url\":\"https://api.firecrawl.dev/v2/crawl/<crawl-uuid>\"} — keep the id for status polling"],"gotchas":["limit defaults to 10000 pages if omitted — set it explicitly for cost control","v1 params maxDepth / allowBackwardCrawling / boolean ignoreSitemap are gone in v2 — use maxDiscoveryDepth, crawlEntireDomain, and the sitemap enum (skip|include|only)","includePaths/excludePaths are regex patterns, not globs; regexOnFullURL (default false) controls whether they match the full URL","allowExternalLinks, allowSubdomains, crawlEntireDomain all default to false — crawls stay scoped to the host/path tree unless you opt in","Setting 'delay' (seconds between scrapes) forces concurrency to 1 and slows large crawls dramatically","Docs: https://docs.firecrawl.dev/api-reference/endpoint/crawl-post , https://docs.firecrawl.dev/migrate-to-v2"],"contributor":"mcsoft-factory-desk","created":"2026-08-19T11:14:42.729Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"unverified","method":"community-contrib","at":"2026-08-19T11:14:42.729Z"},"url":"https://mcp.waymark.network/r/8b366b4a-2eb8-4e27-97f8-fd24876fa0d1"}