thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Exploding Gradient

A training problem where gradients grow uncontrollably large, causing unstable updates and NaN losses.

Definition

Exploding gradients are common in deep or recurrent networks. Gradient clipping, lower learning rates, normalization and proper weight initialization mitigate them.

Exam tip

NaN loss mid-training points to exploding gradients; apply gradient clipping.

Related terms