FROM SCORES TO UNDERSTANDING

AI Evaluation.

Understand the scores.
Choose the right tests.

Start with the lesson

20 evaluation milestones11 contributor profilesSources reviewed 2026-11-07

PUT IT INTO PRACTICE

Your task. A clearer test plan.

Pick what you’re building. Find the measures that matter.

Learn AI: evaluation lesson ↗Explore Model-Benchmark ↗

Four rules to keep close

Test your taskUse examples that resemble real user needs.
Grade the outcomeCheck correctness, evidence and the final result.
Compare like with likeKeep versions, tools and attempt budgets visible.
Inspect failuresRepeat trials; check cost, latency and edge cases.

CHOOSE METRICS FOR MY TASK

What are you building?

Useful measures for chat or support

STARTING POINT

Outcome

Task resolution

Did the user’s problem get solved?

Trust

Evidence & correctness

Are factual claims supported?

Experience

Latency & refusal quality

Is the response timely and appropriate?

A pleasing answer can still be incorrect. Include answerable, ambiguous and unanswerable cases.

Short lesson: hallucinations →Arena Text · human preference ↗

These boards provide related evidence; they do not measure every quality above.

Suggested mappings, not a universal scoring standard. Choose thresholds from your task requirements.

Three common pitfalls

One score becomes the whole story. Check task coverage and failures, not just the average.
A confident answer passes as correct. Verify its evidence; calibrate model judges.
Different setups look comparable. Check dataset version, harness and attempt budget.

Try a judging exerciseOptional · one example

Which answer is better supported?

Illustrative source excerpt: “The deductible is $500.” No coverage details are supplied.

Task: Explain the deductible and whether glass damage is covered. These are authored examples, not outputs from named models.

My evaluation checklist0/8 complete · saved locally

Saved in this browser. Clearing browser data removes progress.

0/8 complete

Define the task and acceptable outcomeCreate representative held-out cases, including failuresWrite a grading rubric and calibrate it with human reviewCheck factual support, permissions and sensitive actionsRepeat trials and inspect failuresMeasure latency and cost per successful taskRecord model, harness, prompt and dataset versionsRun the same regression suite after changes

Completion is a planning aid, not a production-readiness certificate.

Sources & editorial approach · 39 original references

Evaluation sources reviewed 2026-11-07; illustrated history sources reviewed 2026-11-08. Dates refer to first public research or the named milestone. Scores belong to their original benchmark versions; this page explains methods and does not generate rankings.

Turing · 1950 paper archive ↗NIST TREC history and purpose ↗Papineni et al 2002 BLEU ↗Lin 2004 ROUGE ↗ImageNet ILSVRC history and paper ↗Russakovsky et al ILSVRC paper ↗Wang et al GLUE and SuperGLUE ↗SuperGLUE ↗Hendrycks et al MMLU ↗Chen et al HumanEval ↗Lin et al TruthfulQA ↗Srivastava et al BIG-bench ↗Muennighoff et al MTEB ↗Stanford CRFM HELM introduction ↗Zheng et al MT-Bench and Chatbot Arena ↗Es et al Ragas ↗Jimenez et al. · SWE-bench paper ↗Yue et al MMMU ↗Jain et al LiveCodeBench ↗White et al LiveBench ↗Yao et al tau-bench ↗MLCommons AILuminate launch ↗MLCommons AILuminate Safety versioning ↗METR long software tasks study ↗Stanford profile ↗Dartmouth proposal · 1955 ↗Princeton tribute ↗University of Toronto profile ↗Nobel Prize · 2024 ↗McCulloch & Pitts · original paper ↗Dartmouth · founding history ↗Weizenbaum · ELIZA paper ↗Rumelhart, Hinton & Williams · Nature ↗IBM · Deep Blue history ↗AlexNet · 2012 research paper ↗Google DeepMind · AlphaGo ↗Vaswani et al. · Transformer paper ↗Brown et al. · GPT-3 paper ↗OpenAI · Introducing ChatGPT ↗

Further reading: Agent evaluation practice. Model architectures, hardware and connectivity protocols provide context, the illustrated history connects those breakthroughs to evaluation takeaways. The detailed timeline focuses on measurement.