Cut OpenAI API cost and latency with prompt caching: structure prompts for cache hits, set prompt_cache_key, and read cache metrics from the usage object

Documented steps

1. Structure every request so the stable, reusable content (system instructions, tool definitions, shared context) comes FIRST as an exact-match prefix, with variable content after it.
2. Rely on automatic caching for prompts of 1024 tokens or more. No code change is required for the basic behavior.
3. Understand routing: requests are routed by a hash of roughly the first 256 tokens of the prompt, with the exact length varying by model.
4. Set 'prompt_cache_key' to combine with the prefix hash and improve routing consistency across many requests that share a long common prefix.
5. On GPT-5.6 and later model families you MUST set prompt_cache_key to get the more reliable matching for both implicit and explicit caching.
6. Keep traffic per prompt_cache_key to roughly 15 requests per minute across all its prefixes; partition higher volume across multiple keys with a stable mapping.
7. Read hits on the Responses API at usage.input_tokens_details.cached_tokens. On Chat Completions the path is usage.prompt_tokens_details.cached_tokens. It is 0 below the 1024-token threshold.
8. On GPT-5.6+ also read usage.cache_write_tokens, which reports prompt tokens written to cache and is billed at 1.25x the uncached input rate.
9. Control retention with the prompt_cache_retention parameter: 'in_memory' (the default volatile behavior) or '24h' for extended retention.
10. On GPT-5.6 and later, use the newer prompt_cache_options.ttl parameter, for which '30m' is currently the only supported value.
11. Under the in_memory policy, expect cached prefixes to survive roughly 5-10 minutes of inactivity, up to about 1 hour maximum, because they live in volatile GPU memory.
12. Measure the cost of writes against the savings from later hits before caching prefixes that are rarely reused.