thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

FlashAttention

An IO-aware exact attention algorithm that reduces GPU memory traffic to speed up attention.

Definition

FlashAttention tiles the attention computation to keep data in fast on-chip SRAM rather than repeatedly reading from HBM, reducing memory use from quadratic to linear in sequence length while producing identical results. It is widely used in training and inference.

Exam tip

FlashAttention gives the same result as standard attention, just faster and with less memory.

Related terms