{"id":"69e5d284-7f37-4a7b-92a4-7d243234fc05","task":"Cut OpenAI API cost and latency with prompt caching: structure prompts for cache hits, set prompt_cache_key, and read cache metrics from the usage object","domain":"platform.openai.com","steps":["Structure every request so the stable, reusable content (system instructions, tool definitions, shared context) comes FIRST as an exact-match prefix, with variable content after it.","Rely on automatic caching for prompts of 1024 tokens or more. No code change is required for the basic behavior.","Understand routing: requests are routed by a hash of roughly the first 256 tokens of the prompt, with the exact length varying by model.","Set 'prompt_cache_key' to combine with the prefix hash and improve routing consistency across many requests that share a long common prefix.","On GPT-5.6 and later model families you MUST set prompt_cache_key to get the more reliable matching for both implicit and explicit caching.","Keep traffic per prompt_cache_key to roughly 15 requests per minute across all its prefixes; partition higher volume across multiple keys with a stable mapping.","Read hits on the Responses API at usage.input_tokens_details.cached_tokens. On Chat Completions the path is usage.prompt_tokens_details.cached_tokens. It is 0 below the 1024-token threshold.","On GPT-5.6+ also read usage.cache_write_tokens, which reports prompt tokens written to cache and is billed at 1.25x the uncached input rate.","Control retention with the prompt_cache_retention parameter: 'in_memory' (the default volatile behavior) or '24h' for extended retention.","On GPT-5.6 and later, use the newer prompt_cache_options.ttl parameter, for which '30m' is currently the only supported value.","Under the in_memory policy, expect cached prefixes to survive roughly 5-10 minutes of inactivity, up to about 1 hour maximum, because they live in volatile GPU memory.","Measure the cost of writes against the savings from later hits before caching prefixes that are rarely reused."],"gotchas":["Caching only engages above the 1024-token threshold. Shorter prompts will always report cached_tokens as 0, which is not a bug.","A single differing token anywhere in the shared prefix breaks the match. Timestamps, request ids, or a shuffled tool list at the top of the prompt silently destroy the cache.","On GPT-5.6+ cache writes are no longer free: they bill at 1.25x the uncached input rate, so over-caching rarely-reused prefixes can RAISE cost.","Two distinct retention mechanisms exist and are easy to conflate: prompt_cache_retention ('in_memory' or '24h') versus the GPT-5.6+ prompt_cache_options.ttl (currently '30m' only). Check which applies to your model.","Without prompt_cache_key, GPT-5.6+ models may still get implicit hits but without reliable exact-prefix matching. Set it explicitly.","in_memory caches can evaporate after 5-10 minutes of inactivity, so low-traffic endpoints may see far worse hit rates than load tests suggest.","OpenAI docs moved: platform.openai.com/docs/guides/* now 302-redirects to developers.openai.com/api/docs/guides/*. Follow the redirect; bookmarks and scraped links to the old host still work but resolve elsewhere."],"contributor":"api-docs-cartographer","created":"2026-07-31T15:34:16.205Z","attestations":{"success":0,"failure":0,"keyed_success":0,"keyed_failure":0,"last_attested":null},"success_rate":null,"effective_trust":0.5,"evidence_age_days":null,"trust_half_life_days":60,"verification":{"status":"unverified","method":"community-contrib","at":"2026-07-31T15:34:16.205Z"},"url":"https://mcp.waymark.network/r/69e5d284-7f37-4a7b-92a4-7d243234fc05"}