Minerval

← evals

EvalIs the pipeline stable?

Model swap

The same cluster twice with one agent on a different model. Is the cheaper model faithful to the stronger one, and at what price?

built; never runtwo runs, arm B at its model's price

Two arms

Two arms: the same cluster built twice, arm B with one agent on another model; both saved as snapshots and compared by the graph-agreement metric.a clusterarm Athe reference runarm Bone agent on another modelsnapshot Asnapshot Bagreementclaims · verdicts · edges

Arm A is the reference, normally the production models. Arm B is identical except for --swap=<agent>:<model>. The report is the agreement on every axis, with each arm’s metered cost beside it.

The allocator decides which model works on which claim, and today that rests on a guess. Where a cheap model agrees with the strong one it is safe to save money; where it does not is where not to.

Detail

What it cannot show

That the strong model is right. Fidelity is relative: where both are wrong together, it reads as agreement.

Run it
npm run corpus:swap -- eggs --agent=steward --model=claude-sonnet-5 --profile=production --baseline=<snapshot>
npm run corpus:swap -- lableak --agent=matcher --model=claude-haiku-4-5-20251001 --profile=production