Metrics & Evaluations — All Variants & Tricks

Updated July 2026 with 2025–2026 SOTA additions — new entries marked ★. Algorithm names link to their papers (arXiv / project page).

April 2026 · Version 1.0

Why Evaluation Is the Moat
Foundations: Statistical Setup
Image Quality Metrics
Video Quality Metrics
Audio / Speech Metrics
LLM Quality Metrics
LLM-as-Judge
Human Evaluation
Arena-Style Evaluation
Multimodal / VLM Evaluation
Agent and Code-Agent Evaluation
Reasoning Evaluation
Safety, Bias, Truthfulness
Online / Production Evaluation
Contamination and Cheating
Reward Hacking and Goodharting
Eval Harness Design
Specialized Domains
Multi-Aspect / Holistic Frameworks
Eval at Different Stages
Reporting and Communication
Frontier 2025–2026
Production Stack 2026

Appendix A: Twenty-Five Things to Know

Appendix B: Decision Tree — "Which Metric / Benchmark?"

Appendix C: Year-by-Year Eval Milestones

1. Why Evaluation Is the Moat

1.1 The thesis

Key

At the frontier, the team with the best continuous, contamination-free, human-grounded eval pipeline wins, regardless of architecture or data choices. Evals shape every other decision: model selection, hyperparameters, ablations, releases.

1.2 The Goodhart problem

"When a measure becomes a target, it ceases to be a good measure." Every benchmark gets gamed eventually:

GLUE / SuperGLUE: saturated by 2020.
GSM8K: saturated by 2024.
HumanEval: saturated by 2024.
MMLU: saturated by 2025.
Eval engineering = a continuous arms race to design un-gameable signals.

1.3 Three eval phases

Pre-launch: held-out benchmarks, ablations, internal vibes.
Online: A/B, shadow, guardrails on production traffic.
Post-launch: drift / regression detection, slice-based monitoring.

1.4 Eval design principles

Held-out: never train on eval.
Diverse: cover the surface area you care about.
Adversarial: include cases that probe failure modes.
Calibrated: numbers should mean something repeatable.
Multi-axis: no single number is enough.
Cheap to run frequently: continuous integration eval.
Expensive ground truth: human / closed-loop where it counts.

1.5 The hierarchy of evidence

Closed-loop downstream metric (deployed-system success).
Calibrated human preference at scale (Arena Elo).
Held-out human ratings on rubric.
LLM-as-judge (calibrated).
Automatic benchmarks.
Self-eval / proxy.

Higher = more trustworthy + more expensive.

2. Foundations: Statistical Setup

2.1 Population vs sample

A benchmark is a sample from some implicit "task population." Sample size matters:

GSM8K (1.3k): noisy; can swing ±1% on order.
MMLU (14k): more stable.
Always report N + confidence intervals.

2.2 Confidence intervals

Wilson score for a proportion (e.g., accuracy):

\[p \pm z\sqrt{\frac{p(1-p)}{n}}.\]

$z = 1.96$ for 95%. For $n = 1000$, $p = 0.5$: $\pm 3.1\%$.

2.3 Bootstrap

Resample data with replacement; recompute metric; gives confidence bands. Standard for non-trivial metrics (FID, BLEU, etc.).

2.4 Significance testing

Paired permutation test for two-model comparison.
Wilcoxon signed-rank for ranks.
Calibrated to cost: don't claim significance smaller than measurement noise.

2.5 Reporting standards

Mean ± std across K seeds (training).
Confidence interval over examples (eval).
Per-task breakdown, not just aggregate.
Compute spent (tokens / FLOPs).

2.6 Pearson / Spearman correlation

Pearson: linear correlation; sensitive to outliers.
Spearman: rank correlation; robust.
Used to validate proxy metrics against ground truth.

3. Image Quality Metrics

3.1 Pixel-level

MSE / RMSE: simplest; weak perceptual correlation.
PSNR: $10\log_{10}(\mathrm{MAX}^2/\mathrm{MSE})$; standard but coarse.
$\ell_1$: mean absolute error; less weight on outliers.

3.2 Structural

SSIM (Wang et al.): structural similarity index in luminance, contrast, structure:

\[\mathrm{SSIM}(x,y) = \frac{(2\mu_x\mu_y + c_1)(2\sigma_{xy} + c_2)}{(\mu_x^2 + \mu_y^2 + c_1)(\sigma_x^2 + \sigma_y^2 + c_2)}.\]

MS-SSIM: multi-scale variant.

3.3 Perceptual: LPIPS

Learned Perceptual Image Patch Similarity: distance in deep network features (VGG / AlexNet). Strongly correlates with human similarity judgments. Standard since 2018.

3.4 Distributional: FID, KID

Fréchet Inception Distance:

\[\mathrm{FID} = \|\mu_r - \mu_g\|^2 + \mathrm{tr}\!\left(\Sigma_r + \Sigma_g - 2(\Sigma_r\Sigma_g)^{1/2}\right),\]

on Inception-V3 features. Standard for GANs / diffusion since 2017. Limitations: backbone choice matters; broken at high quality.

KID: kernel-based MMD; less biased at small sample.

IS (Inception Score): confidence + diversity; deprecated.

CMMD (CLIP MMD): replaces Inception with CLIP; better for modern generation.

3.5 Precision / Recall on manifold

Separate fidelity (precision) from diversity (recall). Useful when FID misleading.

3.6 Aesthetic predictors

LAION-Aesthetic (CLIP + small MLP).
HPSv2 / HPSv3 (Human Preference Score).
PickScore.
ImageReward.
Trained on human ratings; better correlation with preference than CLIP-Score.

3.7 Compositional / VQA-based

VQAScore: VLM-as-judge for compositional fidelity. "Is the cat sitting on the chair?" → probability "yes".

GenEval: object counting + attribute binding + spatial.

T2I-CompBench: multi-aspect compositional.

3.8 Detection / segmentation specific

IoU: $|A \cap B|/|A \cup B|$. Generalized variants:

GIoU: $\mathrm{IoU} - |C \setminus (A \cup B)|/|C|$, $C$ = enclosing box.
DIoU: + center-distance penalty.
CIoU: + aspect-ratio.

3.9 Mean Average Precision (mAP)

Sweep score threshold; compute precision-recall curve; AP = area under PR curve. COCO mAP: averaged over IoU $\in \{0.50, 0.55, \ldots, 0.95\}$ and 80 classes. Standard for detection.

3.10 Segmentation: Dice / IoU / Panoptic Quality

Dice: $2|A \cap B|/(|A| + |B|)$. Panoptic Quality (PQ): combined recognition + segmentation:

\[\mathrm{PQ} = \frac{\sum_{(p,g)\in \mathrm{TP}} \mathrm{IoU}(p,g)}{|\mathrm{TP}| + \tfrac{1}{2}|\mathrm{FP}| + \tfrac{1}{2}|\mathrm{FN}|}.\]

4. Video Quality Metrics

4.1 FVD (Fréchet Video Distance)

Same idea as FID with I3D / InternVideo features.

Standard for video generation since 2018.

Limitations: backbone choice; not perceptually well-calibrated.

4.2 KVD

Kernel variant of FVD.

4.3 VBench / VBench-2

Multi-axis benchmark with 16+ dimensions (subject consistency, motion smoothness, dynamic degree, scene transition, object class, temporal flicker, etc.). Aggregated score; standard for T2V comparison since 2024.

4.4 T2V-CompBench, EvalCrafter

Compositional + multi-aspect benchmarks. Counts, spatial relations, attribute binding, action over time.

4.5 VideoScore

Learned video reward model trained on human preferences. Used in alignment + filtering.

4.6 Motion-specific metrics

Optical-flow consistency.
Frame-to-frame variance (flicker).
Trajectory smoothness.
Background-stability tests.

4.7 Identity preservation

Face / object embedding distance across frames.
ArcFace / DINO embedding consistency.

4.8 Audio sync (for video+audio)

LSE-D, LSE-C (Lip Sync Error).
SyncNet score.
Phoneme-level alignment.

5. Audio / Speech Metrics

5.1 Speech recognition

WER (Word Error Rate): edit distance / reference length.
CER (Character Error Rate).
TER (Translation Error Rate) for multilingual ASR.

5.2 Speech synthesis

MOS (Mean Opinion Score): human 1–5 rating; gold standard.
PESQ, STOI: perceptual / intelligibility metrics.
NMOS: predicted MOS via ML.

5.3 Music / audio generation

FAD (Fréchet Audio Distance): VGGish / OpenL3 features.
CLAP-Score: text-audio alignment.
KL on Mel-spectrogram features.
Human MOS still standard.

5.4 Diarization

DER (Diarization Error Rate): speaker attribution accuracy.

5.5 TTS naturalness

Modern TTS evals: AB preference vs reference, MOS-N (naturalness), MOS-S (similarity to target speaker for cloning).

6. LLM Quality Metrics

6.1 Perplexity

\[\mathrm{PPL} = \exp\!\left(-\frac{1}{N}\sum_i \log p(x_i \mid x_{<i})\right).\]

Lower = better. Useful for training tracking; weak proxy for downstream quality. Comparable only at fixed tokenizer.

6.2 Cross-entropy in nats / bits-per-byte

Bits-per-byte normalizes across tokenizers (most fair cross-tokenizer comparison).

6.3 Classification-style benchmarks

MMLU: 57 subjects, multiple choice; saturated 2025.
MMLU-Pro: harder, 10-choice; current standard.
HellaSwag, ARC, WinoGrande: legacy.
BBH (BIG-Bench Hard): 23 challenging subset of BIG-Bench.
GPQA: graduate-level science MCQ; current frontier.

6.4 Open-ended: BLEU, ROUGE, METEOR

BLEU: n-gram precision; standard for translation.
ROUGE: n-gram recall; standard for summarization.
METEOR: + synonyms / paraphrasing.
All correlate poorly with human preference for modern LLMs.

6.5 Modern open-ended: LLM-as-judge

MT-Bench, AlpacaEval, AlpacaEval 2: pairwise comparison judged by GPT-4-class. Higher correlation with human preference than BLEU.

6.6 Math benchmarks

GSM8K: grade-school math; saturated.
MATH: competition; mostly solved by frontier 2025.
AIME 2024 / 2025: olympiad qualifier; current.
Putnam: hard college-level; frontier.
Omni-MATH, OlympiadBench: broad.
MathArena: continually updated competition results.

6.7 Code benchmarks

HumanEval: 164 problems; saturated.
MBPP, MBPP+.
LiveCodeBench: continually updated; contamination-resistant; current standard.
SWE-bench, SWE-bench-Verified: real GitHub issues; agentic SWE.
Codeforces / IOI / USACO: competitive programming.

6.8 Reasoning benchmarks

ARC-AGI, ARC-AGI-2: abstraction + reasoning.
GPQA: graduate science.
Humanity's Last Exam: post-2024 hardest.
FrontierMath: hidden hard math.
MuSR: multi-step soft reasoning.

6.9 pass@k

\[\mathrm{pass@}k = 1 - \prod_{i=1}^{k}(1 - p_i),\]

$p_i$ = success probability of $i$-th independent sample. Standard for code; reported at $k = 1, 5, 10, 100$.

7. LLM-as-Judge

7.1 The pattern

Use a strong LLM (GPT-4, Claude, Gemini) to rate / compare model outputs. Cheaper than human; biased.

7.2 Pairwise vs absolute

Pairwise ("A or B?"): higher reliability.
Absolute ("rate 1–10"): noisier; calibration issues.

7.3 Known biases

Position bias: prefers first / second by default.
Length bias: prefers longer.
Self-preference: prefers outputs from same / similar model.
Verbosity / hedging bias.
Format bias: prefers structured, bulleted.

7.4 Mitigations

Order swap: average over both orderings.
Multiple judges: ensemble.
Chain-of-thought before judgment.
Rubric-guided scoring.
Calibrate against human eval on subset.

7.5 LLM-as-judge benchmarks

MT-Bench (LMSYS): 80 multi-turn questions judged by GPT-4.
AlpacaEval / AlpacaEval 2: open eval framework.
Arena-Hard / Arena-Hard-Auto: harder MT-Bench.
LiveBench: contamination-free judge eval.
Frontier labs build custom internal judges.

7.6 Constitutional / rubric judges

Provide explicit criteria.
Per-criterion scoring.
Aggregated final score.
Used in Anthropic's Constitutional AI eval.

7.7 Process Reward Models as judges

For step-level evaluation (math, agents): use PRM trained on per-step labels (PRM800K, Math-Shepherd).

★ 2026 SOTA update — Judge & reward-model benchmarks

RewardBench 2: multi-skill RM eval on 1,865 unseen-human-prompt cases (Factuality, Focus, Math, Precise IF, Safety, Ties); correlates with best-of-N and PPO downstream use.
JudgeBench: 620 objectively-labeled challenging response pairs (knowledge/reasoning/math/code) built from hard datasets; tests judge correctness, not just instruction-following.

8. Human Evaluation

8.1 Why human eval is gold

Captures subjective quality benchmarks miss.
Resilient to Goodhart-style gaming.
Required for creative / safety-critical / open-ended.

8.2 Side-by-side (SBS)

Show model A vs B; ask preference. Standard pattern.

Anonymized order.
Rubric provided.
Multiple raters per item.

8.3 Rating scales

Likert 1–5 / 1–7.
Per-aspect (helpfulness, accuracy, safety).
Free-text rationale.
MOS-style (mean opinion score).

8.4 Inter-annotator agreement

Cohen's κ, Fleiss' κ, Krippendorff's α.
> 0.8: high; 0.6–0.8: moderate; < 0.6: re-spec the task.

8.5 Calibration techniques

Hidden gold-standard items; track per-rater accuracy.
Removed-rater experiments.
Multi-rater majority vote.
Periodic re-training.

8.6 Cost

Quick rating: $0.50–$2 per item.
Detailed rubric: $5–$20 per item.
Expert (medical, legal, code): $50–$200 per item.

8.7 Common eval workflows

Pre-launch: 100–1000 items, multi-rater.
A/B online: smaller sample, larger volume.
Periodic regression: stable item set, frequent re-evaluation.

8.8 Vendors

Surge AI, Scale AI, Mercor, Outlier (formerly Scale's expert side), Snorkel, in-house at every frontier lab.

9. Arena-Style Evaluation

9.1 Chatbot Arena (LMSYS)

Anonymous side-by-side; user picks preference.
Aggregate via Bradley-Terry / Elo.
Crowdsourced; massive sample.
Currently the closest thing to ground truth for general LLM quality.

9.2 Bradley-Terry / Elo math

For pair $(i, j)$, $P(i \succ j) = \dfrac{e^{r_i}}{e^{r_i} + e^{r_j}}$. Update via online estimator. Elo: same idea with K-factor.

9.3 Other arenas

Image Arena (Artificial Analysis): T2I.
Video Arena: T2V.
Search Arena: search quality.
WebDev Arena: code-generation prompts for web.
Vision Arena: VLM.

9.4 Arena pros

Hard to game (anonymized + diverse user prompts).
Reflects real-user value.
Continually updated.

9.5 Arena cons

Sample bias toward power users.
Slow to populate for new models.
Multi-modal arenas have small sample.
Can be gamed by prompt-engineering teams (LMSYS's been hardening).

9.6 Per-category breakdown

Arena-Hard, MT-Bench, others: per-task category breakdowns (math, coding, creative). Important; aggregate hides signals.

9.7 Style control

Arena Hard adds style control to remove length/markdown bias from preference. "Same length / structure" comparisons.

10. Multimodal / VLM Evaluation

10.1 General VLM benchmarks

MMMU, MMMU-Pro: massive multimodal MCQ.
MathVista: visual math.
ChartQA: charts + figures.
DocVQA, InfographicVQA, VisualMRC: documents.
MMVet: multi-task across 6 capabilities.
BLINK, HRBench, RealWorldQA: real-world reasoning.
MMBench, SEED-Bench: broad capability.

10.2 Vision-only

ImageNet: classic; saturated for SOTA encoders.
COCO mAP: detection / segmentation.
ADE20K: scene segmentation.
Open Images V7: large-scale detection.

10.3 Long-video

Video-MME: comprehensive video MCQ.
LongVideoBench: hour-long.
NExT-QA, EgoSchema, MVBench: action / event.
∞-Bench-Video: very long.

10.4 3D / spatial

ScanQA, Scan2CAD: 3D scene QA.
SQA3D: situated QA.
3DLLM bench.

10.5 Embodied / robotics

LIBERO: 130 manipulation tasks.
RoboCasa: kitchen tasks.
SimplerEnv: calibrated sim eval.
ManiSkill 3: GPU sim.
CALVIN: language-conditioned long-horizon.

10.6 VBench / VBench-2 (video gen)

Already covered; standard for video gen.

10.7 Multimodal Arena

Vision Arena (LMSYS): pairwise comparison for VLMs.

★ 2026 SOTA update — Frontier VLM reasoning benchmark

ZeroBench: 100 hand-crafted multi-step visual-reasoning questions designed to be impossible for current LMMs (0% at release) with 334 subquestions; maximal headroom, exact answers.

11. Agent and Code-Agent Evaluation

11.1 Standard agent benchmarks

SWE-bench / SWE-bench-Verified: real GitHub issues.
WebArena, VisualWebArena: 4 web domains.
WebVoyager: real websites.
OSWorld: full-OS task.
AgentBench: 8 environments.
GAIA: general assistant; 466 questions, 3 difficulty levels.
ToolBench: 16k tools.
τ-bench: customer-service dialog.
MLE-bench: ML engineering tasks.
HAL: holistic agent benchmarks.

11.2 Trajectory evaluation

Beyond final success:

Step efficiency (number of actions).
Tool-call accuracy (right tool, right args).
Cost per task.
Recovery from errors.
Off-task wandering.

11.3 LLM-as-judge for agents

Strong LLM evaluates trajectories per rubric. Standard for free-form agent tasks.

11.4 Closed-loop downstream

For robotics / AV: real-world success rate is gold.

11.5 Agent leaderboards

SWE-bench leaderboard: continually updated.
HAL Hub (Princeton): meta-leaderboard.
Vending-Bench, Multi-Agent Bench: emerging.

★ 2026 SOTA update — Conversational & general agent benchmarks

τ²-Bench: dual-control agent eval (Telecom Dec-POMDP) where user AND agent act with tools; compositional verifiable tasks; large drops vs single-control τ-bench.
GAIA2 (ARE): 1,120 asynchronous smartphone-env scenarios with time limits, noisy events, ambiguity, agent-agent collaboration; per-write-action verifiers (RLVR-ready); GPT-5 ~42% pass@1.

★ 2026 SOTA update — Coding / CLI agent benchmarks

SWE-Lancer: 1,400+ real Upwork freelance tasks worth $1M; e2e-tested independent tasks ($50–$32k) + managerial choice tasks; maps agent skill to dollars; frontier still solves minority.
Terminal-Bench: hard, human-verified command-line tasks in containerized Docker envs (compile, train, configure, debug); oracle solutions + test suites; frontier agents <65%.

12. Reasoning Evaluation

12.1 Verifiable-reward benchmarks

Math (extractable answer).
Code (test cases pass).
Logic (constraint satisfaction).

Programmatic checking; un-gameable in principle.

12.2 Major benchmarks (April 2026)

AIME 2024 / 2025 (math).
Putnam (college math).
FrontierMath (Epoch AI; hidden problems).
HMMT, USAMO, IMO (olympiad-class).
Humanity's Last Exam (PHD-level cross-domain).
ARC-AGI / ARC-AGI-2 (abstraction).
GPQA (grad science).
LiveCodeBench, USACO, Codeforces (code).

12.3 Best-of-N / pass@k

For sampling-based reasoning:

\[\mathbb{E}\!\left[\max_{i\le N} r_i\right] \approx \mu + \sigma\sqrt{2 \ln N}.\]

Diminishing returns; reported as pass@1, pass@10, pass@100.

12.4 Self-consistency eval

Sample K reasoning chains; majority vote; report consistency. Useful for uncertainty quantification.

12.5 Process-level eval

PRM800K-style step labels.
Math-Shepherd auto-labels.
OmegaPRM Monte Carlo.

Catches reasoning quality, not just final answer.

12.6 Token efficiency

Compute spent per problem solved. Important for cost-aware comparison: a model that solves at 10× token cost is often worse than one that solves at 1×.

13. Safety, Bias, Truthfulness

13.1 Refusal / harm

HarmBench: 510 harmful prompts; measure compliance.
JailbreakBench, AdvBench: adversarial prompts.
XSTest: false-refusal benchmark.
AILuminate (MLCommons): standardized safety.

13.2 Truthfulness

TruthfulQA: deceptive-pattern probes.
HaluEval: hallucination detection.
Retrieval-grounded fact-check (Q&A with sources).

13.3 Bias / fairness

BBQ: bias in QA.
BOLD: open-ended generation.
StereoSet: stereotypical association.
CrowS-Pairs: 9 demographic dimensions.
Per-group performance gap analysis.

13.4 Toxicity

Perspective API.
Detoxify.
ToxicBERT.

13.5 Red-team metrics

Attack success rate.
Time-to-jailbreak.
Diversity of successful attacks.

13.6 Calibration

ECE (Expected Calibration Error): predicted prob vs actual frequency.
Reliability diagrams.
Brier score.

★ 2026 SOTA update — Grounding & factuality benchmarks

FACTS Grounding: 1,719 long-form grounding tasks (up to 32k-token docs); multi-judge scoring of eligibility + full grounding (no hallucination); public/private split leaderboard.
SimpleQA Verified: 1,000-prompt short-form parametric-knowledge factuality; cleaned/de-biased rebuild of OpenAI SimpleQA; F1 (correct vs not-attempted); contamination-aware.
BrowseComp: 1,266 hard, short-answer web-browsing questions requiring deep multi-site search; easy to verify; targets deep-research/browsing agents (legacy retrieval evals saturated).

14. Online / Production Evaluation

14.1 A/B testing

Random assignment.
Sufficient sample for statistical power.
Pre-registered hypothesis.
Multiple-testing correction.

14.2 Interleaving

Both models' outputs shown together; users pick. More efficient than A/B for ranking signal.

14.3 Shadow traffic

Route copy of production traffic to candidate model; compare offline. Lower risk than A/B.

14.4 Counterfactual eval

Compute candidate model's output on logged inputs; compare to logged response. Cheap; assumes input distribution stable.

14.5 Guardrail metrics

p50, p95, p99 latency.
Error rate.
Cost per request.
Content-policy violations.
User retention.

14.6 Drift detection

Embedding-space KL between train / prod.
Per-feature distribution shift (KS test).
Per-segment performance dashboards.
Statistical-process-control alerts.

14.7 Per-segment dashboards

Slice metrics by:

User cohort.
Geo / language.
Task type.
Time of day.
Model version.

Aggregate metrics hide regressions in slices.

14.8 Long-horizon retention

DAU, WAU, MAU after model swap.
Repeat-use rate.
Task-completion rate.
Customer satisfaction (CSAT, NPS).

15. Contamination and Cheating

15.1 The problem

Models trained on internet may have seen eval examples. "Solving" may be memorization, not generalization.

15.2 Detecting contamination

N-gram overlap ($n = 8$–13) between training and eval.
Exact-match search.
Embedding-similarity cluster check.
Per-example verbatim-recall test.

15.3 Reducing contamination

Filter eval-overlap from training (always).
Use post-cutoff benchmarks (LiveCodeBench, current AIME).
Use hidden / private test sets (FrontierMath, HLE).
Continually generate new evals.

15.4 LiveBench (continually updated)

Monthly new questions.
Sourced from recent papers, news, contests.
Auto-graded.
Resists contamination.

15.5 Adversarial probes

Perturbations of standard problems (rename variables, change numbers).
Drift in semantically equivalent rephrasing.
Memorization vs reasoning ablation.

15.6 Contamination-resistant patterns

Verifiable-reward problems (model can solve, can't memorize answer).
Per-test stochastic generation.
Hold-out by data cutoff.
Rotating private test sets.

16. Reward Hacking and Goodharting

16.1 The phenomenon

Models / teams optimize the metric instead of the underlying quality. Once a benchmark gets traction, gaming begins.

16.2 Common gaming patterns

Length inflation (verbose responses).
Format conformity (markdown headers in everything).
Self-confidence (reduce "I don't know").
Calibration drift (overconfident on ambiguous).
Refusal regression (calibrate harm classifier rather than improve helpfulness).

16.3 Reward overoptimization

Gao et al. scaling law: true reward

\[R(\mathrm{KL}) \sim a\sqrt{\mathrm{KL}} - b\,\mathrm{KL}.\]

Proxy reward keeps rising while true quality declines past a peak.

16.4 Mitigations

KL anchor in RL (prevents drift).
Reward model ensembles (use min / median).
WARM (Weight-Averaged Reward Models).
Held-out human eval as final arbiter.
Multiple judges + diverse benchmarks.

16.5 Length-bias defenses

Length-normalized log-prob (SimPO).
Style-control in arenas.
Length-balanced training data.
Explicit length penalty in reward.

16.6 Sycophancy

Model agrees with user even when wrong. Test via deliberate-wrong prompts; track agreement rate.

17. Eval Harness Design

17.1 The eval harness

A reproducible system to run any benchmark on any model. Open-source standards:

lm-evaluation-harness (EleutherAI): the de facto open standard.
HuggingFace Evaluate.
OpenAI Evals.
Mosaic Composer Eval.
BIG-Bench.
Per-vendor internal tools.

17.2 Best practices

Reproducible: pin seed, model, prompt format.
Configurable: easy to swap model.
Per-task per-config metrics.
Cache results.
Audit trail.

17.3 Prompt format issues

Same model, different prompt format, ±10% accuracy.
Standardize: canonical chat template, few-shot count, instruction wording.
Ablate prompt sensitivity.

17.4 Eval configurations

Zero-shot vs few-shot.
Chain-of-thought on / off.
Generation hyperparameters (temp, max tokens).
Number of samples (for pass@k / self-consistency).

17.5 Continuous eval during training

Cheap proxy eval every $\sim$ 1k steps.
Full eval at milestones / end.
Detect divergence early.
"Vibes" set: small hand-curated; subjective check.

17.6 Eval-driven development

Define eval before optimizing.
Locked / versioned eval set.
Metrics linked to user-value hypothesis.
No secret optimization without eval signal.

18. Specialized Domains

18.1 Medical

USMLE (medical licensing).
MedQA, MedMCQA, PubMedQA.
Med-PaLM 2 benchmarks.
Per-specialty performance.

18.2 Legal

LegalBench, LexGLUE.
LawBench, JEC-QA.

18.3 Financial

FinanceBench, FinQA.
FinanceIQ.
Trading / market-making proprietary.

18.4 Scientific

SciBench, ChemBench, PubMedQA.
LAB-Bench.
Genomics / biology specific.

18.5 Multilingual

MMLU translations.
XNLI, XTREME, FLORES-200.
HELM Lite multilingual.
Per-language performance breakdown critical.

18.6 Long-context

NIAH (Needle in a Haystack): simple retrieval.
RULER: NIAH + multi-hop + aggregation.
LongBench, ∞Bench, LV-Eval: real long-doc tasks.
LooGLE, ZeroSCROLLS.

★ 2026 SOTA update — Long-context benchmarks (2025)

HELMET: 7 application-centric long-context categories (RAG, re-ranking, ICL, QA, summarization) with controllable lengths to 128k, model-based metrics, few-shot support; NIAH replacement.
LongBench v2: 503 hard 4-choice questions over 8k–2M-word contexts across 6 realistic task types; experts ~53.7% in 15 min; strong reasoning models (o1-preview) ~57.7%.

19. Multi-Aspect / Holistic Frameworks

19.1 HELM (Stanford)

Holistic Evaluation of Language Models. 16 core scenarios + 7 metrics dimensions (accuracy, calibration, robustness, fairness, bias, toxicity, efficiency). Mature open framework.

19.2 HELM Lite

Slim version for faster, frequent eval.

19.3 Big-Bench / Big-Bench Hard

Large collection of $\sim$ 200 tasks contributed by community. BBH = 23-task hard subset.

19.4 MMLU-Pro / GPQA Diamond / Humanity's Last Exam

Frontier-level eval suites.

19.5 Evolve (HuggingFace) / Open LLM Leaderboard v2

Curated leaderboard with rigorous methodology. Common reference for open models.

19.6 Per-task breakdowns

Always required:

Aggregate hides cross-task variance.
Per-task winning model varies.
Need per-task to identify regressions.

19.7 Composite scoring

Average rank across tasks (more robust than mean accuracy).
Pareto front (no dominated points).
User-weighted: weight by use-case importance.

20. Eval at Different Stages

20.1 Pretraining tracking

Loss curves per shard.
Held-out perplexity on diverse domains.
Cheap downstream proxy (MMLU subset).
Spike detection.

20.2 SFT / instruction-tuning

AlpacaEval / MT-Bench.
Per-capability suite (math, code, etc.).
Format adherence.
Safety regression.

20.3 RLHF / DPO eval

Held-out human / RM preference rate.
KL drift from base.
Reward overoptimization curve.
Capability regression on standard suite.

20.4 Reasoning RL eval

Verifiable-reward task accuracy.
Response length over training.
Format adherence.
Cross-task generalization.

20.5 Production eval

Online A/B + interleave.
User CSAT / retention.
Safety incident rate.
Latency / cost SLOs.

20.6 Cadence

Per-step: loss only.
Per-1k-steps: cheap proxy.
Per-1k-checkpoints: full benchmark suite.
Pre-release: full human eval.
Post-release: continuous monitoring.

21. Reporting and Communication

21.1 What a good eval report looks like

Per-task numbers + confidence intervals.
Methodology (prompt format, few-shot, sampling).
Compute spent.
Comparison to relevant baselines.
Per-segment / per-slice breakdowns.
Known limitations.

21.2 The bar for "new SOTA" claim

Significance vs prior SOTA.
Across multiple benchmarks.
Same / less compute.
Reproducible (open weights or methodology).
No contamination.
Per-task wins, not just aggregate.

21.3 Dashboards

Wandb / TensorBoard / MLflow.
Internal Streamlit / Grafana for production.
Public leaderboards (HuggingFace, lmarena).

21.4 Warning signs in eval

Aggregate up, per-task mixed.
One task carries everything.
Sample-efficiency unclear.
No CI.
No baseline comparison.
Prompt-format ablation missing.

21.5 Cherry-picking defenses

Pre-register benchmarks.
Report all in fixed suite.
Provide reproduction code.
Submit to public leaderboards.

22. Frontier 2025–2026

22.1 Trends

Open-ended evals via judge / arena dominate.
Verifiable-reward benchmarks for reasoning.
Continuously updated benchmarks (LiveBench, LiveCodeBench, current-year AIME).
Hidden test sets (FrontierMath, HLE).
Per-capability multi-suite reporting.
Style control to reduce length / format bias.

22.2 Saturation crisis

Most legacy benchmarks (≥95% on MMLU, GSM8K, HumanEval).
Need ever-harder ones.
Frontier maintained via private / continually updated.

22.3 New benchmark categories

Long-horizon agent (HAL, MLE-bench).
Multimodal reasoning (MMMU-Pro, MathVista).
Real-world agent tasks (WebVoyager, OSWorld).
Physical commonsense (PhyGenBench, Cosmos Reason).
Robotics (LIBERO, SimplerEnv, RoboCasa).

22.4 Open research

Eval that doesn't saturate as models improve.
Process-level reasoning eval.
Multi-agent system eval.
Long-horizon (multi-day) agent eval.
Cost-quality Pareto reporting.

23. Production Stack 2026

Use case	Default approach	Notes
LLM general quality	MT-Bench / AlpacaEval 2 / Arena	Elo + per-task
LLM frontier eval	MMLU-Pro / GPQA / HLE / Math	Frontier / uncontaminated
LLM reasoning	AIME / Putnam / LiveCodeBench / ARC-AGI	verifiable rewards
LLM code	LiveCodeBench / SWE-bench-Verified / Codeforces	post-cutoff
LLM safety	HarmBench / JailbreakBench / XSTest	+ red-team
VLM / multimodal	MMMU-Pro / MathVista / RealWorldQA / BLINK	+ Vision Arena
Long-context	RULER / LongBench / ∞-Bench / LV-Eval	multi-aspect
Image gen	FID / CLIP-Score / HPSv3 / Image Arena	VQAScore + human
Video gen	FVD / VBench-2 / VideoScore / Video Arena	+ human
Audio gen	FAD / CLAP-Score + MOS	+ human
Detection / segmentation	COCO mAP / Cityscapes mIoU / PQ	per-class
3D / NeRF / 3DGS	PSNR / SSIM / LPIPS + Chamfer	per-scene
Agent (general)	GAIA / AgentBench / WebArena / OSWorld	trajectory eval
Agent (SWE)	SWE-bench-Verified + LiveCodeBench	contamination-resistant
Robotics	SimplerEnv / LIBERO / RoboCasa demos	+ real closed-loop
Eval harness	lm-eval-harness / HELM / Open LLM Leaderboard v2	open standard
Online production	A/B + interleave + guardrail drift	metrics + continuous