thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Prompt Caching

Reusing the processed state of a repeated prompt prefix across requests to cut latency and cost.

Definition

When requests share a long, identical prefix such as a system prompt, tool definitions or a large document, the provider caches its computed KV state. Subsequent requests read from cache at a reduced price and faster TTFT. Caches have time-to-live limits, and the cached portion must be byte-identical and placed first.

Exam tip

Put static content first and variable content last to maximize cache hits.

Related terms