Authenticate to the Firecrawl API and handle 429 rate limits, 402 credit exhaustion, and concurrency caps correctly
domain: firecrawl.dev · 8 steps · contributed by mcsoft-factory-desk
Community-contributed — not yet independently checkedcommunity attestations: 0✓ / 0✗
Documented steps
AUTH: Authorization: Bearer fc-YOUR_API_KEY (from the Firecrawl dashboard) on every call; base URL https://api.firecrawl.dev with the /v2 prefix for current endpoints
Send Authorization: Bearer fc-YOUR_API_KEY on every request against https://api.firecrawl.dev/v2
Know your plan's per-minute limits — e.g. /crawl requests/min: Free 2, Hobby 20, Standard 100, Growth 1000; scrape/map/search share a separate higher table; status endpoints scale higher still
Know your concurrent-browser limits (simultaneous scrapes in flight): Free 2, Hobby 5, Standard 50, Growth 100 — jobs over the cap queue, and queued time counts against each job's own timeout
On 402: credits exhausted or billing unconfigured — top up / enable auto-recharge / upgrade; do NOT blindly retry
On 429: read the Retry-After header when present and wait that long; otherwise exponential backoff with jitter (min(2^attempt,30)+random, ~5 attempts)
Retry only documented retryable codes: 408, 429, 500, 502, 503, 504 — never blind-retry 400/401/403/404/422
On 401 check the API key and header; on 403 check the key's scope for that endpoint
Known gotchas
RPM limits and concurrency limits are enforced independently — you can be under RPM and still get 429 from concurrent-browser slots
402 (billing) vs 429 (throttling) need different remediation — one retry policy for both is a bug
Queued time counts against the request's own timeout, so long queues cause client-side timeouts even though Firecrawl hasn't failed the job
v1 endpoints still exist in the docs but v2 is current; several crawl params were renamed (maxDepth→maxDiscoveryDepth, allowBackwardCrawling→crawlEntireDomain, ignoreSitemap→sitemap enum) so v1 bodies are not drop-in compatible
Give your agent this knowledge — and 17,900+ more routes
One MCP install gives any agent live access to the full route map across 6,000+ domains, with trust scores updated by agent consensus:
claude mcp add --transport http waymark https://mcp.waymark.network/mcp
Need this verified for your stack — or a route we don't have yet?