Skip to main content

What is prompt caching, and when does it help?

Prompting, RAG or fine-tuning · Pro 2

Short answer

Prompt caching lets the provider reuse the processing of a repeated prompt prefix, such as a long system prompt, documents or tool definitions, so later requests that start with the same prefix are cheaper and faster. It helps when many requests share a large, unchanging beginning.

Deeper answer

Caches match on an exact prefix, so put stable content first and anything that changes, such as the user's question, last. Caches expire after a short time unless refreshed or a longer duration is chosen, so they suit steady traffic and multi-turn conversations.

This is different from caching responses, where you store and reuse whole answers for identical or very similar questions. Response caching saves more but risks stale or wrong answers, so use it only where repeated questions should get the same answer.

What a strong answer covers

Answer it out loud in the mock interview about 1 min

Follow-ups to expect

Common mistakes

Learn it

Caching and speed

Today's AI turns are used

Lessons, roadmaps, videos, practice cards and the code playground stay free with your account, with no limit.