What an evaluation actually measures

Why can a good score mean nothing?

Because the score and the thing it stands for can come apart under optimisation. The clearest demonstration we have covered is a moral hazard game in which querying another agent carries a cost. Fourteen models were evaluated, eleven open-weight and three frontier API models, across nine query-cost conditions and 3,015 decisions.

The models did not behave alike. One tracked the incentive boundary with a mean absolute error of 0.013, reaching ceiling performance. Another shifted away from querying toward local rewards as costs rose. A third responded to several incentive variables at once rather than to cost alone. But the paper's central observation is about measurement rather than about any model: optimisation can lift aggregate reward without restoring the cooperative mechanism the design was meant to produce.

For anyone running a multi-agent system, that is the warning to internalise. A system judged on task completion can be improving on the metric while the coordination it depends on quietly stops happening, and nothing in the aggregate number will show it. The implied response is to measure at the level of the mechanism: not whether the system succeeded, but whether the agents actually consulted each other when the incentive said they should.

Does a paused benchmark predict live behaviour?

Not reliably, and the counter-example is recent. RealtimeBench runs 180 tasks across six kinds of game and lets the game carry on while the agent reasons and calls tools, on the argument that an action which would have been correct a second ago can arrive too late to be worth anything. In paired evaluations the stronger model, GPT-6.1-sol, outperformed the faster Jev when the games were paused and fell behind it once they kept running.

The ordering of two models therefore depended on whether the clock was stopped. Any leaderboard built on paused tasks is answering a different question from the one a deployment decision asks. The same paper offers a partial remedy: a slow teacher studies failures between attempts and writes reusable instructions for a fast student, which raised the student's average test score from 13.51 to 24.19 on a nought-to-hundred scale and transferred to three other models without retuning.

How contaminated are public benchmarks?

Enough that contamination is the base rate rather than an edge case. Public benchmarks leak into training corpora within months of release, so teams that rely on them to choose models are increasingly measuring memorisation rather than capability. This is not a defect in any one benchmark; it is what happens to any fixed, published test set that becomes worth scoring well on.

The approach that has held up is more work than downloading a benchmark: maintain a private, versioned set drawn from your own traffic, refresh a portion of it each quarter, and never publish it. The versioning matters as much as the privacy, because without it a change in score cannot be separated from a change in the set.

What is a better signal than agreement?

Disagreement. When an agent works on a long task, checking its output becomes as hard as producing it, and the usual instinct is to sample several times and take the consensus. One paper we covered inverts that: across repeated rollouts, disagreement often exposes correct alternatives, while consensus can conceal shared errors.

The system built on that observation turns the same model the generator uses into a verifier with a workspace and evidence tools. A disagreement resolver checks competing claims against evidence from the environment; a consensus challenger tests the claims every rollout agrees on and looks for requirements nobody addressed. Across five long-horizon workspace benchmarks and two frontier models it reached the highest selection scores among the baselines evaluated, and allowing evidence-backed revision on top of selection raised the gain over a single rollout to 6.2 points with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8.

Nothing in that requires a stronger judge model or a human rubric. What it requires is the willingness to sample several times and then spend compute reconciling the differences rather than counting votes, which is the opposite of how self-consistency is usually applied.

What should you measure instead?

• Name the mechanism before the metric. Decide what behaviour the number is standing in for, and measure that behaviour directly as well.
• Run the evaluation with the clock running if the system will face a moving environment, and keep the paused result only as a capability ceiling.
• Keep a private, versioned set from your own traffic, refresh part of it on a schedule, and never publish it.
• Treat disagreement between runs as evidence to investigate rather than noise to average away.
• Measure time to first useful action, not only final accuracy, wherever the environment can change during inference.

Vendor evaluations have the same gap in a different form. Forrester's Wave on revenue orchestration platforms assessed 12 providers and found that building agents had become table stakes, with the discussion moving to governance, oversight, interoperability and auditability — which are exactly the properties an aggregate score does not express.