thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Foundational

Transformer

A neural network architecture based on self-attention that processes all tokens in a sequence in parallel; the basis of modern LLMs.

Definition

Introduced in the 2017 paper 'Attention Is All You Need', the transformer replaces recurrence with attention, letting every token attend to every other token. Blocks combine multi-head attention, feed-forward layers, residual connections and normalization. Variants include encoder-only (BERT), decoder-only (GPT, Claude, Llama) and encoder-decoder (T5).

Exam tip

Parallel processing and long-range context via attention are why transformers replaced RNNs.

Related terms