thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Quantization

Reducing the numeric precision of model weights (and sometimes activations) to shrink memory and speed inference.

Definition

Quantizing from 16-bit to 8-bit or 4-bit formats cuts memory 2-4x and raises throughput, with small accuracy loss when done well. Post-training quantization (such as GPTQ, AWQ, FP8) needs no retraining; quantization-aware training simulates low precision during training. TensorRT Model Optimizer supports FP8, INT4 and NVFP4.

Example

Quantizing a 70B model to 4-bit lets it fit in about 35-40 GB of GPU memory.

Exam tip

Quantization reduces memory and cost; expect a small accuracy tradeoff.

Related terms