thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Residual Connection

A skip connection that adds a layer's input to its output, easing gradient flow in deep networks.

Definition

Introduced by ResNet, residual connections let layers learn modifications to an identity mapping, enabling networks with hundreds of layers. Every transformer block uses residual connections around attention and feed-forward sublayers.

Exam tip

Skip or residual connections are the standard fix that lets very deep networks train without vanishing gradients.

Related terms