AI Models
A Benchmark Is a Map, Not the Territory
A model that wins a reasoning test can still fail to fill in four JSON fields. Build the eval your product actually needs.
A benchmark is a measurement taken in a laboratory. Your product is the weather. The two are related, and confusing them is how teams ship a model that scores 92% on a reasoning test and cannot reliably fill in a JSON object with four fields.
What a public benchmark actually tells you
- It tells you the model can do a class of task under the harness the benchmark used: a particular prompt format, temperature, tool set and scoring script.
- It tells you roughly where a model sits relative to its peers at the moment of publication.
- It tells you almost nothing about latency, cost, refusal behaviour, or how it handles your domain vocabulary.
Two more problems are structural. Contamination: evaluation sets leak into training data, so a high score can mean memorisation. Saturation: once every model scores above 90, the benchmark stops separating them. A leaderboard with a tie at the top is not a ranking, it is a ceiling.
Build the eval you actually need
The eval that matters fits on one screen: thirty real inputs from your product, a pass condition a human can judge, and the cost and latency of each run. Thirty is enough to catch a regression and small enough to re-run after lunch.
{
"task": "ticket-classification",
"cases": [
{ "input": "Refund not received after 7 days", "expect": "billing" },
{ "input": "Cannot log in on Safari only", "expect": "account" }
],
"pass": "label matches expect",
"record": ["model", "latency_ms", "input_tokens", "output_tokens", "cost_usd"]
}Keep the outputs. A score without the transcript is a rumour. When a model fails, you want to see the exact response, not a percentage that went down by four points.
Score the failure modes separately
One number hides the thing you need to know. Split the eval into the ways a model can fail: wrong answer, malformed output, refusal, timeout, hallucinated tool call. A model with 5% malformed output and 0% wrong answers is easy to fix with a validator. A model with 0% malformed output and 5% confident wrong answers is a liability.
Accuracy is one axis. Reliability is the axis that decides whether you sleep.
How to read a leaderboard without being fooled
- Check the date. A six-month-old result is a historical document.
- Check the harness. A score with tools is not comparable to a score without them.
- Check the error bars. If two models are within a point, treat them as tied.
- Check who paid for the evaluation, then check whether the eval is independently reproducible.