thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Prefill and Decode

The two phases of LLM inference: processing the whole prompt in parallel (prefill), then generating tokens sequentially (decode).

Definition

Prefill is compute-bound and builds the KV cache; decode is memory-bandwidth-bound because each step reads all weights and cache to produce one token. Disaggregated serving, as in NVIDIA Dynamo, runs these phases on separate GPU pools.

Exam tip

Prefill is compute-bound and decode is memory-bandwidth-bound; optimizations target them differently.

Related terms