thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Gradient Descent

An optimization algorithm that iteratively adjusts parameters in the direction that most reduces the loss.

Definition

Gradient descent computes the gradient (partial derivatives) of the loss with respect to each parameter and moves parameters a small step, scaled by the learning rate, in the opposite direction. Variants include batch, stochastic and mini-batch gradient descent. Most modern training uses adaptive variants such as Adam.

Exam tip

If loss oscillates or diverges, the learning rate is too high; if it barely moves, too low.

Related terms