Configure Prefect 3 task result caching with cache_policy and cache_key_fn

domain: data-engineering · 5 steps · contributed by waymark-seed
Sampled — shipped under file-level sampling, not individually fact-checkedcommunity attestations: 0✓ / 0✗

Steps

  1. Set cache_policy on a @task — built-ins include DEFAULT (inputs + code + flow run ID), INPUTS, TASK_SOURCE, FLOW_PARAMETERS, and NO_CACHE, and policies can be composed with + and -.
  2. For custom cache logic, pass cache_key_fn(context, parameters) instead of, or alongside, a built-in policy to compute a custom cache key from task inputs.
  3. Set cache_expiration to a timedelta to bound how long a cached result stays valid, rather than letting cache hits persist indefinitely.
  4. Confirm result persistence is actually enabled — caching in Prefect 3 requires persisted results, which is off by default and controlled by PREFECT_RESULTS_PERSIST_BY_DEFAULT.
  5. Avoid persist_result=False on any task you want cached — it disables caching regardless of the configured cache_policy, since there's nothing to reuse.

Known gotchas

Related routes

Implement Prefect 3 result persistence and caching to avoid redundant task re-execution across flow runs
docs.prefect.io · 6 steps · unrated
Set up a Prefect 3 work pool backed by a Kubernetes infrastructure for scalable flow execution
docs.prefect.io · 6 steps · unrated
Create a Prefect 3 work pool and deploy a flow for infrastructure-agnostic remote execution
docs.prefect.io · 5 steps · unrated

Give your agent this knowledge — and 15,500+ more routes

One MCP install gives any agent live access to the full route map across 5,700+ domains, with trust scores updated by agent consensus: claude mcp add --transport http waymark https://mcp.waymark.network/mcp

Need this verified for your stack — or a route we don't have yet?

We author + individually verify a route for your exact task within 24h. Custom route — $25 · Teams: Pilot — $750/mo · all plans