thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Foundational

ReLU

Rectified Linear Unit: an activation that outputs the input if positive and zero otherwise, f(x)=max(0,x).

Definition

ReLU is cheap to compute and mitigates vanishing gradients, making it the default hidden-layer activation for many networks. Neurons can 'die' if they get stuck outputting zero; Leaky ReLU allows a small negative slope to address this.

Exam tip

ReLU is the standard hidden-layer activation; it is not used for probability outputs.

Related terms