Minerval

← evals

Background

Models and cost

What each agent runs on in production, what has been measured, and what a run should cost.

What the agents run on

agentmodel$ / M input$ / M output
stewardClaude Fable 5.1 claude-fable-5-110.0050.00
solverClaude Fable 5.1 claude-fable-5-110.0050.00
steward strongClaude Fable 5.1 claude-fable-5-110.0050.00
curatorClaude Fable 5.1 claude-fable-5-110.0050.00
auditClaude Fable 5.1 claude-fable-5-110.0050.00
arbitrationClaude Fable 5.1 claude-fable-5-110.0050.00
extractorClaude Fable 5.1 claude-fable-5-110.0050.00
matcherGLM 5.3 Flash z-ai/glm-5.3-flashprovider-pricedprovider-priced
taggerGLM 5.3 Flash z-ai/glm-5.3-flashprovider-pricedprovider-priced
judge (evals only)Claude Sonnet 5 claude-sonnet-53.0015.00

List rates from pricing.ts, the table the system bills through. Pins from api-stack.ts. --profile=production applies exactly these.

Detail

What has been measured
whatsetupcost
golden pairs, 30 decisionsDeepSeek V4 Flash$0.06
blackholes ingest, 11 Steward runs, cappedSonnet Steward, 12 iterations, Curator off$11.29
judge, 13 verdictsClaude Sonnet 5$0.60
What a run should costestimate
clusterdocumentsSteward runsper run
blackholes4≈28$80 – 200
eggs3≈20$60 – 140
lableak5≈35$100 – 250
lethalities11100 – 150$400 – 900

Basis, as of 2026-09-04: the Steward is nearly the whole cost, one per extracted claim (about seven per document), each a tool-using loop over a prompt that carries the entire constitution. A Sonnet Steward capped at twelve iterations cost about $1 a run; production uses Claude Fable 5.1 at 3.3× the rate with a 200-iteration budget, so $2 – 6 per Steward run, plus Curator sweeps. The first metered production run replaces this table.

Spending limits

Set in .env; 0 means unlimited. The production profile sets models only, not these.

  • STEWARD_MAX_RUNS: total Steward runs in a run. The main cap.
  • STEWARD_MAX_ITERATIONS: tool calls within one Steward run. 200 by default; lower it only for smoke tests.
  • CURATOR_MAX_RUNS, CURATOR_SWEEP_RATE: Curator sweeps; rate 0 disables.
  • LLM_DAILY_TOKEN_LIMIT, LLM_HOURLY_TOKEN_LIMIT: the circuit breaker; the run stops cleanly when hit.

Every run prints its exact metered cost at the end.