Canonical-form goldens
Twenty-six source excerpts with the one neutral wording each should become. The regression net for the Extractor's wording.
What it checks
The ExtractorThe agent that reads a document and lists the claims it makes.agent → proposes a canonical formThe one neutral wording of a claim that every source's version maps to. About fifteen words.constitution → for every claim it finds, and since the 2026-08-11 finding that the MatcherThe agent that decides whether a claim is already in the graph, possibly worded differently or as its denial.agent → rewrote that proposal on most new claims, both agents’ wording is under test. A prompt or model change that starts rewriting good wording, sharpening a vague proposition with numbers nobody committed to, or writing the form in the source’s direction rather than the direction the discourse debates, would show here first.
Each case pins a verbatim two-to-four sentence excerpt from a committed corpus document and the form its central proposition should take. Six categories: direction (the source argues against the proposition; the form must still state the affirmative), neutrality, scope, survive (the excerpt is already near-canonical and must not be rewritten in substance), hedging, specificity.
Detail
How it is run, step by step
- Extract. the real Extractor, its production prompt unchanged, runs once on the excerpt as a source of type excerpt, capped at three claims.
- Pick. of the proposals it made, the one closest to the expected form by embedding similarity is the one judged; the rest are recorded.
- Judge. a pair judge on JUDGE_MODEL answers three narrow questions about the proposed form against the expected one: same proposition, same direction, neutral with no invented specificity. Pass is all three yes.
- Report. per case the proposals, the one judged, its similarity and the judge's verdict and note; per category the pass rate; the exact metered cost. The report is filed under corpus/scorecards/golden-canonical/ and registered as kind golden-canonical; --min-pass makes it a gate.
The judge's promptverbatim
What the judge is sent per case, placeholders where the case’s fields go. Source: golden-canonical-lib.ts.
You are comparing two candidate canonical forms for the central proposition of a source excerpt, against the standard below. One is the pinned EXPECTED form; the other was PROPOSED by an extraction agent. Answer three narrow questions about the proposed form relative to the expected one. This is a comparison of substance, not style: different words for the same proposition in the same direction with no added specificity is a full pass. Canonical form (Minerval constitution §3): the shortest neutral statement of the proposition as it is actually debated, about fifteen words and rarely more than twenty-five, stated at the precision the discourse debates it. It strips the frame — the author's name, the document's dialectical setup, document-relative references — and is worded so that anyone discussing the proposition, whichever answer they give, would accept it as a fair statement of what is in dispute. It is written in the direction the discourse poses the question (the affirmative form of what is argued, as a debate motion or neutral headline would put it) even when the source argues against it; the source's stance is recorded on the instance, never by bending the form. It is not sharpened with parameters, thresholds or dates the author never committed to, and a vague proposition stays vague. Qualifications and hedges belong to the source's wording, not to the form. ## Source excerpt <the source excerpt, verbatim> ## Expected canonical form <the pinned canonical form> ## Proposed canonical form <the form the Extractor proposed> Answer: same proposition? same direction? neutral and no invented specificity? Then one sentence on the most important difference.
The response schema:
| field | answers | question |
|---|---|---|
same_proposition | boolean | Do the two forms state the SAME proposition — would the same evidence and reasons bear on both, and would a source that affirms one thereby affirm the other? Wording may differ freely; substance may not. |
same_direction | boolean | Are the two forms stated in the same direction — does the proposed form affirm what the expected form affirms, rather than its negation or denial? A form that states the contrary proposition ("X does not…" against "X does…") is the wrong direction even though it is the same node. |
neutral_no_invented_specificity | boolean | Is the proposed form neutral (no author framing, no dialectical setup, no document-relative references, no hedges carried into the form) AND free of specificity the expected form does not carry (no added numbers, thresholds, dates, named studies, mechanisms, or qualifiers that narrow or sharpen the proposition)? |
note | string | One sentence: the single most important difference, or 'equivalent' when there is none. |
The 26 casesthe fixture, in full
Canonical-form golden cases (#334 S1, addendum of 2026-08-11). Each case pins a verbatim 2–4 sentence excerpt from a committed corpus post and the §3 canonical form its central proposition should take. The runner extracts from the excerpt with the real Extractor (EXTRACTION_MAX_CLAIMS 1–3), takes the proposal closest to the expected form by embedding, and asks a pair judge on JUDGE_MODEL three questions — same proposition? same direction? neutral and no invented specificity? — pass = all yes. Categories: direction (the source argues AGAINST the proposition; the form must still state the affirmative the discourse debates), neutrality (author/dialectical framing stripped), scope (population or condition kept), survive (the excerpt is already near-canonical and must not be rewritten in substance), hedging (the proposition, not the hedge), specificity (no parameters the author never committed to). Grading is a judgment on prose, so this is near-deterministic rather than exact-match; it exists to catch a prompt or model change that starts rewriting good wording.
“Even on the extremely conservative assumption that some microscopic black holes turned out to be stable, they would still pose no danger. A stable microscopic black hole would be far too small — on the order of the Planck length — to accrete ordinary matter at any meaningful rate.”
Expected: Microscopic black holes produced at the LHC could grow to endanger the Earth. · The safety case denies the proposition; the discourse poses it as the danger question ('will the LHC create a black hole that destroys the Earth?'). The form states that affirmative; CERN's 'no' is the instance stance. A proposal reading 'pose no danger' has been bent toward the author.
“A large prospective study following nearly 40,000 men and over 80,000 women found that up to one egg per day is not associated with increased heart disease risk in healthy individuals. So for a generally healthy person, an egg a day appears to be fine.”
Expected: Eating up to one egg a day raises heart disease risk in healthy adults. · The debated question is whether eggs raise heart-disease risk; this source answers no. The form keeps the affirmative and the 'healthy adults' scope; the cohort sizes are evidence, not the proposition.
“To engineer SARS-CoV-2, the WIV would have needed a close progenitor. Its closest published virus, RaTG13, is far too distant (~4% divergent — decades of evolution) to be the backbone.”
Expected: The Wuhan Institute of Virology possessed a virus close enough to serve as a progenitor of SARS-CoV-2. · The zoonosis case denies the lab-capability crux; the crux is posed in the affirmative (did the WIV have a close-enough progenitor?). 'RaTG13 is too distant' is a reason, and a possible second claim, but not the proposition this passage turns on.
“In particular, existing AI training strategies don’t need to handle a “drastic” distribution shift from low levels of intelligence to high levels of intelligence. There’s nothing in the foreseeable ways of building AI that would call for a big transfer like this, rather than continuously training as intelligence gradually increases.”
Expected: Aligning AI requires generalizing across a drastic distribution shift from low to high intelligence. · Christiano is denying lethality #12. The proposition as the debate poses it is Yudkowsky's affirmative; this instance denies it. A proposal that states 'does not need to handle' is the same node in the wrong direction.
“Eliezer strongly expects sharp capability gains, based on a combination of arguments that I think don’t make sense and an analogy with primate evolution which I think is being applied poorly. We’ve talked about this before, and I still think Eliezer’s position is probably wrong and clearly overconfident.”
Expected: AI capabilities will increase sharply rather than gradually. · A denial wrapped in attribution ('Eliezer expects…', 'I think'). Both frames go; the form is the takeoff proposition in the direction the debate poses it, and the source denies.
“The pandemic began in Wuhan, home to the WIV, the largest collection of bat coronaviruses in Asia and a center for exactly the kind of research most likely to produce a SARS-like virus. Of all the cities in China — and all the rural locations near the bat caves where these viruses circulate naturally — the outbreak surfaced in the one city hosting the leading lab studying them. Proponents argue this proximity is a strong piece of evidence that conventional zoonosis struggles to match, since wildlife spillovers usually originate in rural areas far from major virology institutes.”
Expected: The outbreak's origin in Wuhan, home of the WIV, is strong evidence for a laboratory origin of SARS-CoV-2. · 'Proponents argue' and the rhetorical 'of all the cities' setup are frame. The proposition is the geographic-coincidence argument, stated so a zoonosis advocate would accept it as what is in dispute.
“The white-dwarf and neutron-star arguments assume the dangerous black holes would be stopped and retained by those bodies. Plaga argues that in the metastable regime the relevant black holes would either not be retained, or would themselves evaporate before destroying the star, so the survival of white dwarfs and neutron stars does *not* constrain the metastable scenario.”
Expected: The survival of white dwarfs and neutron stars excludes dangerous metastable black holes from collider collisions. · Strip 'Plaga argues'. The proposition is the astrophysical bound's reach, which the safety consensus affirms and Plaga denies; the form states it in the consensus' affirmative direction, and this instance denies.
“**#6 (pivotal act is necessary).** We think it's necessary to end the acute risk period, but don't agree with the "pivotal act" framing that assumes that the risk period is ended through a discrete unilateral action by a small number of actors.”
Expected: Ending the acute AI risk period requires a discrete pivotal act by a small number of actors. · 'We think' and the list-item label are frame. The disputed proposition is the pivotal-act framing itself; the team denies it, Yudkowsky affirms it.
“The food scientist Harold McGee argues that the cholesterol in the yolk is not the main problem, because saturated fat raises blood cholesterol much more than dietary cholesterol does.”
Expected: Saturated fat raises blood cholesterol far more than dietary cholesterol does. · A source attribution: the claim is what McGee asserts, stated plainly (extractor prompt: 'Smith asserts X' → X). The attribution is not itself disputed.
“The evidence diverges sharply for people with diabetes. In the Nurses' Health Study and the Health Professionals Follow-up Study, heart disease risk was increased among men and women with diabetes who ate one or more eggs a day.”
Expected: Daily egg consumption raises heart disease risk in people with diabetes. · The population is the point: the general-population claim is a different node. Study names are evidence; 'one or more a day' is the daily scope.
“According to the Standard Model of particle physics, the LHC's energies are far too low to create black holes at all. Some speculative theories that posit extra spatial dimensions suggest that microscopic black holes could be produced — at a rate of perhaps one per second.”
Expected: The LHC can produce microscopic black holes only if extra spatial dimensions exist. · The condition (extra dimensions) is the proposition's scope, not a hedge to strip. 'One per second' is a detail no side commits to.
“Molecular-clock analyses (Pekar et al. 2021/2022; Pipes et al.) give high odds that the two lineages reflect at least two separate introductions of an animal virus into humans at the market within days of each other — a pattern naturally explained by multiple infected animals, and hard to explain by a single laboratory escape.”
Expected: The two early SARS-CoV-2 lineages reflect at least two separate introductions into humans at the Huanan market. · 'At least two' and 'at the market' are the proposition's scope, contested as such (the lab-leak side reads the lineages as a puzzle for a single market origin). The analyses cited are evidence.
“They consistently show that adding eggs raises LDL relative to a no-egg diet — but raising a biomarker is not the same as causing disease, and the effect shrinks or vanishes when eggs replace other foods rather than being added on top.”
Expected: Adding eggs to a diet raises LDL cholesterol relative to a no-egg diet. · A biomarker claim with a specific comparison, not a disease claim. Generalizing to 'eggs raise cholesterol' or 'eggs cause heart disease' changes the proposition.
“**The best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we'd rather the AI not solve**; you can't build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.”
Expected: The algorithms that best solve the problems we want an AI to solve readily generalize to problems we would rather it not solve. · The bolded sentence is already the proposition in near-canonical form; the red/blue cars are illustration. A survive case: substance must not move.
“On my model, one of the most central technical challenges of alignment—and one that every viable alignment plan will probably need to grapple with—is the issue that capabilities generalize better than alignment.”
Expected: AI capabilities generalize further than alignment. · The Extractor's own prompt lists 'Capabilities generalize further than alignment' as a good claim. The 'on my model' and 'most central challenge' framing goes; the proposition survives.
“SARS-CoV-2 has a furin cleavage site absent from its closest known relatives, encoded partly with an arginine codon (CGG) that is uncommon in this viral family. One side sees a fingerprint of engineering; the other sees ordinary natural recombination and an insertion too "suboptimal" to be designed.”
Expected: The furin cleavage site of SARS-CoV-2 is evidence of laboratory engineering. · The overview already states the crux neutrally with both sides; the form is the debated motion. Composition details (CGG codon) are evidence either side cites.
“Combining the strong theoretical expectation that any micro black hole evaporates essentially instantly via Hawking radiation with the empirical, model-independent bound from the observed stability of white dwarfs and neutron stars, we find that there is no risk of any significance whatsoever from black holes that might be produced at the LHC. The astrophysical bound stands even if Hawking radiation is assumed not to operate at all.”
Expected: The observed stability of white dwarfs and neutron stars excludes dangerous LHC black holes even without Hawking radiation. · The load-bearing claim of the paper, stated in the paper's own affirmative direction; 'even without Hawking radiation' is the proposition's strength as the authors commit to it, not invented specificity.
“Powerful AI systems have a good chance of deliberately and irreversibly disempowering humanity. This is a much more likely failure mode than humanity killing ourselves with destructive physical technologies.”
Expected: Powerful AI systems have a substantial chance of deliberately and irreversibly disempowering humanity. · Already canonical. A proposal that sharpens 'good chance' into a probability, or folds in the comparison to physical technologies, has rewritten it.
“The DEFUSE proposal shows that, eighteen months before the pandemic, researchers at exactly the institution at the outbreak's location had proposed to build viruses with precisely the features SARS-CoV-2 turned out to have. That coincidence of capability, intent, and location is the strongest form of the lab-leak argument: not proof, but a documented motive and means.”
Expected: The 2018 DEFUSE proposal documents an intent to engineer the features that distinguish SARS-CoV-2. · 'Not proof, but…' is the author's hedge and belongs on the instance. The proposition is what the proposal documents; the stronger 'proves a lab origin' would overstate it.
“The LHC Safety Assessment Group (LSAG) notes that "there is broad consensus among physicists on the reality of Hawking radiation, but so far no experiment has had the sensitivity required to find direct evidence for it."”
Expected: Black holes emit Hawking radiation. · The proposition is the physics, not the sociology: 'there is consensus that X' is a hedge around X. A proposal about consensus states a different proposition — the miss this case exists to catch.
“In the 2015–2020 Dietary Guidelines for Americans, that numerical limit was dropped. The advisory committee concluded that "cholesterol is not a nutrient of concern for overconsumption," reflecting evidence that dietary cholesterol has a weak and variable effect on blood cholesterol for most people.”
Expected: Dietary cholesterol has only a weak effect on blood cholesterol in most people. · The guideline change is the event; the proposition it reflects is the weak-effect claim. 'The committee concluded' is a hedge, not the claim.
“Physicists noted they cannot demonstrate a zero probability for catastrophic events — they can only calculate upper limits. Martin Rees indicated the risk of a RHIC doomsday scenario as being at most a 1 in 50 million chance.”
Expected: The probability of a catastrophic outcome from collider experiments cannot be shown to be zero, only bounded. · The attribution to 'physicists' and Rees's illustrative number are frame; the proposition is that collider risk is bounded above, not excluded.
“**We can't just "decide not to build AGI"** because GPUs are everywhere, and knowledge of algorithms is constantly being improved and published; 2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.”
Expected: Humanity cannot coordinate to refrain from building AGI. · '2 years' and '5 other actors' are illustrative, not a commitment; a form that carries them has been sharpened past what the author debates.
“For SARS-CoV-2, despite the testing of tens of thousands of animals, no infected intermediate host has ever been found. The environmental market samples show viral RNA co-located with animal stalls but do not establish that an infected animal — rather than an infected human vendor or shopper — deposited it. The absence of a found animal host, years on, is offered as evidence against the market-spillover story.”
Expected: The failure to find an infected intermediate host is evidence against a market spillover origin of SARS-CoV-2. · 'Tens of thousands of animals' and 'years on' quantify the evidence; the proposition is the inference from absence. Do not import the numbers.
“People who eat many eggs may differ systematically from those who don't — in smoking, exercise, overall diet, or socioeconomic status. A classic pattern is that, in some populations and eras, frequent egg eaters also ate more processed meat and fewer vegetables, so the egg "effect" may be the effect of an unhealthy dietary pattern.”
Expected: Observed associations between egg consumption and heart disease are largely explained by confounding. · The listed confounders are examples; a form naming 'processed meat' or 'socioeconomic status' has committed the author to a mechanism the passage only illustrates.
“But the probability of producing a strangelet *decreases* as collision energy increases, so the high-energy LHC is even less likely to produce one than lower-energy machines that ran for years without incident.”
Expected: The probability of producing a strangelet decreases as collision energy increases. · The general proposition; the LHC-vs-predecessors comparison is its application. A form pinning energies or machines has added specificity.
The file: canonical-forms.json.
Results1 run
| run | passed | by category | cost |
|---|---|---|---|
2026-09-18T17-42-30-813Z.json | 10/26 | direction 0/5 · neutrality 3/4 · scope 1/4 · survive 2/5 · hedging 2/4 · specificity 2/4 | $0.10 |
What it cannot show
- Grading is a judge’s opinion on three narrow questions, not an exact match. A disagreement on a direction case is a fixture judgment to discuss, not a model failure.
- Only the proposal closest to the expected form is judged; a case can pass on a neighbouring claim the Extractor happened to state.
- The judge warns, but does not refuse, when it is the same model as the Extractor.
Run it
npm run corpus:golden-canonical -- --profile=production
npm run corpus:golden-canonical -- --category=direction --min-pass=0.9