thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Intermediate

HumanEval

A code-generation benchmark of programming problems scored by whether generated code passes unit tests.

Definition

HumanEval reports pass@k, the probability that at least one of k samples passes the tests. It measures functional correctness rather than textual similarity.

Exam tip

Code is best evaluated by executing tests (pass@k), not BLEU.

Related terms