Corpus runs
Build a claim graph from a fixed set of documents, with the live pipeline, in a separate database.
What happens in a run
The clusterA fixed set of documents on one topic, committed to the repository, that a test run reads. There are four.evals guide →’s documents go through the same route the live site uses. The ExtractorThe agent that reads a document and lists the claims it makes.agent → lists each document’s claims. The MatcherThe agent that decides whether a claim is already in the graph, possibly worded differently or as its denial.agent → checks whether each is already in the graph. A StewardThe agent that owns a claim: breaks it into the claims it depends on, weighs the evidence, and records a verdict.agent → takes each new claim, breaks it into the claims it depends on, and records a verdictOne of six statuses: verified, supported, inconclusive, doubtful, contradicted, unknown.constitution →. The CuratorThe agent that tends the links between claims: merges duplicates, splits conflations.agent → tends the links. The run ends when no work is queued.
Everything else in this guide measures the graph a run produces. The agents’ full instructions are under agents.
Detail
What a run records
Before its first model call, the run writes its fingerprintThe record of what built a graph: the code version, which model each agent ran on, and the spending limits in force.evals guide →: code version, the model each agent is set to, spending caps. Afterwards it adds the models actually seen in the usage log and the metered cost.
With --profile=production the agents run on the models the live site uses (steward: Claude Opus 5.5, solver: Claude Opus 5.5, steward strong: Claude Opus 5.5, curator: Claude Opus 5.5, audit: Claude Opus 5.5, arbitration: Claude Opus 5.5, extractor: Claude Opus 5.5, matcher: GPT-6 Luna, tagger: GPT-6 Luna, lookout: GPT-6 Luna, consistency: GPT-6 Luna). Without it, cheaper development defaults.
Output: a report to read against the rubric, the graph as JSON, and the fingerprint. See what a run leaves behind.
Cost
Almost all of it is Stewards: one per extracted claim, each a long tool-using loop with web search. Measured once: $11.29 for a capped run with eleven Steward runs on Sonnet.
On the production models, an estimated $80 – 200 for a small cluster and $400 – 900 for the large one. Details and the estimate’s basis: models and cost.
What it cannot show
Whether the graph is right. There is no answer key for a contested question. And one run is one sample: nothing seen once is a property of the pipeline (noise band).
Results11 runs on record
epoch 2026-09-domain-skills commit 4a7d757 profile none: development defaults, not production matcher z-ai/glm-5.3-flash steward z-ai/glm-5.3-flash curator z-ai/glm-5.3-flash judge z-ai/glm-5.3-flash extractor z-ai/glm-5.3-flash models from registry caps curatorMaxRuns=2 stewardMaxRuns=5 curatorSweepRate=1 llmDailyTokenLimit=0 llmHourlyTokenLimit=0 stewardMaxIterations=25 scored 2026-09-18 judge cost none
The file: blackholes/2026-09-18T21-34-02-354Z.json. Its numbers are on the structural and judged pages.
Run it
docker compose up -d # Postgres 16 + pgvector
npm run corpus:reset # the separate database
npm run corpus:run -- blackholes --profile=production --limit=1 # read the printed cost first
npm run corpus:run -- blackholes --profile=production --score