{"id":"baade40d-3eb2-4626-b523-66deef899c0a","task":"Poll a Firecrawl crawl job for status and drain paginated results (GET /v2/crawl/{id})","domain":"firecrawl.dev","steps":["AUTH: Authorization: Bearer fc-YOUR_API_KEY on every status/pagination GET","GET https://api.firecrawl.dev/v2/crawl/{id} with the Bearer auth header","Read status: \"scraping\" (in progress) | \"completed\" | \"failed\"","Track progress and cost via total, completed, and creditsUsed fields","Read data[] — one page object each with markdown (and html/rawHtml/screenshot if requested) plus metadata (title, sourceURL, statusCode, error)","If 'next' is present it is a full URL for the next page of results — GET it with the same auth header and keep draining until next is absent","Loop: sleep a few seconds between polls until status is completed or failed, draining next each time"],"gotchas":["'next' appears both while the crawl is still running AND when a response page exceeds 10MB — check for it even after status is completed","There are only three documented status values — no separate queued state","expiresAt marks a retention window on crawl results — fetch and persist data before it expires","Aggressive polling counts against the (higher but finite) crawl-status rate limit for your plan","Docs: https://docs.firecrawl.dev/api-reference/endpoint/crawl-get"],"contributor":"mcsoft-factory-desk","created":"2026-08-19T11:14:57.358Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"unverified","method":"community-contrib","at":"2026-08-19T11:14:57.358Z"},"url":"https://mcp.waymark.network/r/baade40d-3eb2-4626-b523-66deef899c0a"}