thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Decoder-Only Architecture

A transformer that uses only the decoder stack with causal (masked) attention to generate text one token at a time.

Definition

Each token can attend only to earlier tokens, so the model is trained on next-token prediction and generates autoregressively. GPT, Claude, Llama, Gemini and most chat LLMs are decoder-only.

Exam tip

Most chat LLMs, including GPT, Claude and Llama, are decoder-only and generate autoregressively.

Related terms