Glossary
Prompt caching
Reusing the work a model has already done on the unchanged start of a prompt, so repeated prompts come back faster and cost less.
OpenAI keeps the processed form of a prompt's unchanged opening and discounts those cached input tokens by up to 95 percent. Caching has a security side: researchers showed at ICML 2025 that cached prompts come back measurably faster, and found caches shared between users at seven API providers, including OpenAI, so the timing could reveal something about other users' prompts. OpenAI's documentation says its caches are not shared across organizations.