thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

Cross-Entropy Loss

A loss that measures the difference between predicted class probabilities and the true labels; standard for classification and LLMs.

Definition

Cross-entropy (log loss) heavily penalizes confident wrong predictions. Binary cross-entropy is used for two-class or multilabel problems and categorical cross-entropy for multiclass. Language models are trained by minimizing cross-entropy of the next token, and perplexity is its exponential.

Exam tip

Classification uses cross-entropy; regression uses MSE or MAE.

Related terms