thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

High Bandwidth Memory (HBM)

Stacked DRAM packaged alongside the GPU that provides very high memory bandwidth.

Definition

HBM capacity determines how large a model and KV cache fit on a GPU, and bandwidth largely sets LLM decode speed. H100 has 80 GB HBM3; H200 has 141 GB HBM3e; B200-class GPUs have about 180-192 GB.

Exam tip

LLM token generation is usually memory-bandwidth-bound, so HBM bandwidth matters as much as FLOPS.

Related terms