thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Optimizer

The algorithm that updates model weights using computed gradients, such as SGD, Adam or AdamW.

Definition

Optimizers decide how gradients translate into parameter changes, often maintaining state such as momentum or per-parameter adaptive learning rates. Adaptive optimizers store extra values per parameter, which significantly increases training memory.

Exam tip

Adam stores two extra values per parameter, so optimizer state can exceed weight memory during training.

Related terms