Prompt caching lets providers reuse the processed state of repeated prompt prefixes, long system prompts, shared documents, stable few-shot examples, billing ca
A feature that lets providers reuse the processed state of repeated prompt prefixes, long system prompts, shared documents, stable few-shot examples, billing those cached input tokens at a steep discount and cutting latency.
Structure prompts deliberately: put the stable prefix first and variable content last. For RAG assistants and agents with large stable contexts, caching is routinely the single biggest cost optimization available.