thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Knowledge Distillation

Training a smaller student model to reproduce the outputs or behavior of a larger teacher model.

Definition

Distillation transfers capability into a cheaper, faster model, using the teacher's soft probabilities or generated responses as training data. Amazon Bedrock Model Distillation and NVIDIA NeMo support it.

Exam tip

Choose distillation to cut inference cost and latency while keeping most quality.

Related terms