thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Positional Encoding

Information added to token embeddings so a transformer knows the order of tokens.

Definition

Because attention itself is order-agnostic, transformers inject position via sinusoidal encodings, learned position embeddings or rotary position embeddings (RoPE). The positional scheme affects how well a model handles long contexts.

Exam tip

Without positional information, a transformer would treat a sentence as an unordered bag of tokens.

Related terms