thinQmaster Virtual testing partner Adaptive practice exams that find your weak spots and drill them. Every answer explained. From $69
Enroll

thinQmaster

Find your exam

Five questions free, before you decide.

Step 1 · choose a vendor

In one of our camps? Your practice exams come with the tuition — they are on your study page already.

AI & Machine Learning · Advanced

Evaluation Harness

A framework that runs a model or application over test cases and computes metrics automatically and repeatably.

Definition

Harnesses manage datasets, prompts, scoring functions and result tracking so teams can compare versions and catch regressions. Examples include EleutherAI's lm-evaluation-harness for benchmarks, NVIDIA NeMo Evaluator, MLflow evaluation and custom eval suites run in CI.

Exam tip

Treat evals like unit tests: run them automatically on every prompt or model change.

Related terms