thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Mixture of Experts (MoE)

An architecture where a router activates only a few expert sub-networks per token, increasing capacity without proportional compute.

Definition

MoE models have many total parameters but a smaller number of active parameters per token, giving high quality at lower inference FLOPs. All experts must still fit in memory, and expert parallelism distributes them across GPUs. Mixtral, DeepSeek-V3 and many frontier models use MoE.

Exam tip

MoE reduces compute per token, not total memory needed to host the model.

Related terms