thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Next-Token Prediction

The training objective of predicting the next token in a sequence given the preceding tokens.

Definition

Decoder-only LLMs learn language, facts and reasoning patterns by minimizing cross-entropy on next-token prediction across huge corpora. Generation repeats the prediction step token by token.

Exam tip

Next-token prediction explains why LLMs produce plausible rather than verified text, which leads to hallucinations.

Related terms