thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Layer Normalization

Normalizing activations across the features of each individual example rather than across a batch.

Definition

Layer norm is independent of batch size, so it suits transformers and sequence models. Many LLMs use RMSNorm, a simplified variant, and apply normalization before each sublayer (pre-norm).

Exam tip

Transformers use layer norm; CNNs traditionally use batch norm.

Related terms