thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Foundational

Learning Rate

The hyperparameter that sets how large a step the optimizer takes when updating weights.

Definition

A learning rate that is too high causes loss to oscillate or diverge, while one that is too low makes training slow or stuck. Learning rate schedules and warmup adjust it over training. It is usually the single most important hyperparameter to tune.

Example

Setting learning_rate=2e-4 for LoRA fine-tuning.

Exam tip

Diverging or exploding loss: lower the learning rate first.

Related terms