thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Masked Language Modeling (MLM)

A pretraining objective in which random tokens are hidden and the model predicts them from surrounding context.

Definition

MLM, used by BERT, lets models learn bidirectional context. It produces strong representations for understanding tasks but is not suited to free-form generation.

Exam tip

MLM = BERT-style; CLM/next-token = GPT-style.

Related terms