thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Byte-Pair Encoding (BPE)

A subword tokenization algorithm that iteratively merges the most frequent adjacent symbol pairs into new tokens.

Definition

BPE starts from characters or bytes and builds a vocabulary of common subwords, so rare words are split into pieces and no input is ever out of vocabulary. Byte-level BPE is used by GPT-family and many other LLMs; WordPiece (BERT) and SentencePiece are related approaches.

Exam tip

Subword tokenization like BPE means rare or new words are split into pieces rather than being unknown.

Related terms