EvalIs a graph well built?
Structural scorecard
Counts taken from a run's graph, no model involved. Each one serves a section of the rubric.
What it checks
Counts a program takes from a run’s graph, with no model involved: how many claims came out of how many words, how long their wording is, how often the Matcher merged, how deep the decomposition goes, how many verdicts have reasoning behind them. Each count serves one section of the rubric and is a symptom of a failure that section names.
The numbers only mean something against earlier runs: a count that moves beyond the noise bandHow much a number varies between runs when nothing changed. A difference smaller than that is not a result.evals guide → after a change is a lead, and its direction says which agent to look at.
Detail
The metricshover a name for its standard
Letters are rubric sections. Hover a metric for the standard it serves; click for the section. Judge rows come from the judged scorecard.
blackholes
| metric | latest (2026-09-18) | across runs |
|---|---|---|
| A · claims per 1k wordsExtract all substantive claims, faithfully and charitably, across every claim type. Questions, commands, meta-text, stipulative definitions, and hedged non-assertions are not claims.rubric → | 22.05 | n=4 · 21.35 ± 9.26 |
| B · canonical form, p90 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 24 | n=4 · 23 ± 4.08 |
| B · share over 25 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 6% | n=4 · 7% ± 5% |
| C · dedup ratioTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 1.94 | n=4 · 1.88 ± 0.52 |
| D · max depthNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 6 | n=4 · 5 ± 2.38 |
| D · atomic shareNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 81% | n=4 · 78% ± 15% |
| E · shared subclaimsRedundancy should collapse and structure should be shared: claims that recur across posts become one node, common dependencies become shared subclaims, and related/opposing positions are connected, not siloed. The admin thinks at graph level — upstream, downstream, lateral.rubric → | 4 | n=4 · 3 ± 2.5 |
| F · assessments with a traceUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 100% | n=4 · 100% ± 0% |
| §21 · coherence violations | 0 | n=4 · 0 ± 0 |
| B · Matcher rewrite rateThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 18% | n=3 · 17% ± 17% |
| B · rewrite magnitudeThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0.61 | n=2 · 0.45 ± 0.22 |
| C · matched denies shareTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 0% | n=3 · 0% ± 0% |
| importance · meanUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 0.36 | n=4 · 0.32 ± 0.04 |
| judge · claim-bar pass rate | n/a | not on this run |
| judge · importance overrated share | n/a | not on this run |
| judge · readability (1–5) | n/a | not on this run |
| judge · reasoning fit (1–5) | n/a | not on this run |
| judge · impartiality (1–5) | n/a | not on this run |
| judge · sycophancy share | n/a | not on this run |
| judge · overhedged share | n/a | not on this run |
| judge · overconfident share | n/a | not on this run |
| judge · canonical-form miss share | n/a | not on this run |
| judge · political bias share | n/a | not on this run |
eggs
| metric | latest (2026-09-18) | across runs |
|---|---|---|
| A · claims per 1k wordsExtract all substantive claims, faithfully and charitably, across every claim type. Questions, commands, meta-text, stipulative definitions, and hedged non-assertions are not claims.rubric → | 3.88 | n=3 · 16.04 ± 13.55 |
| B · canonical form, p90 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 23 | n=3 · 22 ± 0.58 |
| B · share over 25 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 6% | n=3 · 3% ± 3% |
| C · dedup ratioTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 2.06 | n=3 · 1.44 ± 0.56 |
| D · max depthNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 7 | n=3 · 4 ± 2.65 |
| D · atomic shareNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 75% | n=3 · 75% ± 7% |
| E · shared subclaimsRedundancy should collapse and structure should be shared: claims that recur across posts become one node, common dependencies become shared subclaims, and related/opposing positions are connected, not siloed. The admin thinks at graph level — upstream, downstream, lateral.rubric → | 15 | n=3 · 7 ± 7.64 |
| F · assessments with a traceUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 100% | n=3 · 100% ± 0% |
| §21 · coherence violations | 0 | n=3 · 0 ± 0 |
| B · Matcher rewrite rateThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 13% | n=3 · 19% ± 10% |
| B · rewrite magnitudeThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0.06 | n=3 · 0.17 ± 0.1 |
| C · matched denies shareTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | n/a | not on this run |
| importance · meanUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 0.41 | n=3 · 0.41 ± 0.02 |
| judge · claim-bar pass rate | 100% | n=3 · 94% ± 10% |
| judge · importance overrated share | 0% | n=3 · 0% ± 0% |
| judge · readability (1–5) | 4.3 | n=2 · 4.31 ± 0.02 |
| judge · reasoning fit (1–5) | 4.4 | n=2 · 4.29 ± 0.16 |
| judge · impartiality (1–5) | 4.6 | n=2 · 4.39 ± 0.3 |
| judge · sycophancy share | 0% | n=3 · 6% ± 10% |
| judge · overhedged share | 0% | n=3 · 0% ± 0% |
| judge · overconfident share | 0% | n=3 · 0% ± 0% |
| judge · canonical-form miss share | 10% | n=3 · 31% ± 31% |
| judge · political bias share | 0% | n=3 · 6% ± 10% |
lableak
| metric | latest (2026-09-18) | across runs |
|---|---|---|
| A · claims per 1k wordsExtract all substantive claims, faithfully and charitably, across every claim type. Questions, commands, meta-text, stipulative definitions, and hedged non-assertions are not claims.rubric → | 37.19 | n=2 · 27.36 ± 13.91 |
| B · canonical form, p90 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 21 | n=2 · 20 ± 2.12 |
| B · share over 25 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0% | n=2 · 0% ± 0% |
| C · dedup ratioTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 3.43 | n=2 · 2.38 ± 1.48 |
| D · max depthNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 7 | n=2 · 5 ± 2.83 |
| D · atomic shareNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 79% | n=2 · 87% ± 10% |
| E · shared subclaimsRedundancy should collapse and structure should be shared: claims that recur across posts become one node, common dependencies become shared subclaims, and related/opposing positions are connected, not siloed. The admin thinks at graph level — upstream, downstream, lateral.rubric → | 12 | n=2 · 8 ± 5.66 |
| F · assessments with a traceUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 100% | n=2 · 100% ± 0% |
| §21 · coherence violations | 0 | n=2 · 0 ± 0 |
| B · Matcher rewrite rateThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 27% | n=2 · 18% ± 13% |
| B · rewrite magnitudeThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0.23 | n=2 · 0.2 ± 0.04 |
| C · matched denies shareTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 17% | n=2 · 8% ± 12% |
| importance · meanUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 0.38 | n=2 · 0.41 ± 0.05 |
| judge · claim-bar pass rate | 100% | n=2 · 100% ± 0% |
| judge · importance overrated share | 0% | n=2 · 0% ± 0% |
| judge · readability (1–5) | 4.73 | n=2 · 4.53 ± 0.28 |
| judge · reasoning fit (1–5) | 4.36 | n=2 · 4.18 ± 0.25 |
| judge · impartiality (1–5) | 4.64 | n=2 · 4.48 ± 0.22 |
| judge · sycophancy share | 0% | n=2 · 0% ± 0% |
| judge · overhedged share | 0% | n=2 · 0% ± 0% |
| judge · overconfident share | 9% | n=2 · 21% ± 17% |
| judge · canonical-form miss share | 9% | n=2 · 5% ± 6% |
| judge · political bias share | 0% | n=2 · 0% ± 0% |
lethalities
| metric | latest (2026-09-18) | across runs |
|---|---|---|
| A · claims per 1k wordsExtract all substantive claims, faithfully and charitably, across every claim type. Questions, commands, meta-text, stipulative definitions, and hedged non-assertions are not claims.rubric → | 2.59 | n=2 · 2.26 ± 0.47 |
| B · canonical form, p90 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 19 | n=2 · 18 ± 1.41 |
| B · share over 25 wordsThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0% | n=2 · 1% ± 2% |
| C · dedup ratioTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 1.75 | n=2 · 1.4 ± 0.49 |
| D · max depthNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 5 | n=2 · 3 ± 3.54 |
| D · atomic shareNeutral decomposition: identify what a claim depends on, don't evaluate it. Decompose until reaching genuine bedrock (bedrock fact / contested empirical / value premise).rubric → | 89% | n=2 · 94% ± 8% |
| E · shared subclaimsRedundancy should collapse and structure should be shared: claims that recur across posts become one node, common dependencies become shared subclaims, and related/opposing positions are connected, not siloed. The admin thinks at graph level — upstream, downstream, lateral.rubric → | 17 | n=2 · 9 ± 12.02 |
| F · assessments with a traceUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 100% | n=2 · 50% ± 71% |
| §21 · coherence violations | 0 | n=2 · 0 ± 0 |
| B · Matcher rewrite rateThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 40% | n=2 · 26% ± 20% |
| B · rewrite magnitudeThe canonical form is the shortest neutral statement of the proposition as it is actually debated: about fifteen words, rarely more than twenty-five, terse and frame-independent, self-contained, and meaning-preserving. State the proposition at the precision the discourse debates it; do not sharpen it with parameters the author never committed to, and do not mint placeholders for missing ones — the vague proposition is the claim.rubric → | 0.38 | n=2 · 0.39 ± 0.01 |
| C · matched denies shareTwo formulations are the same claim when they turn on the same considerations: nothing could count as evidence or argument bearing on one without bearing equally on the other. Identical decomposition is a useful diagnostic, not the definition.rubric → | 33% | n=2 · 28% ± 8% |
| importance · meanUse the six statuses honestly (Verified, Supported, Inconclusive, Doubtful, Contradicted, Unknown); a status describes the evidence, not the debate, so a disputed claim gets the status its evidence supports and Inconclusive is reserved for evidence that, examined, comes down on neither side; never round an inconclusive claim up to verified or down to contradicted; every assessment carries a substantive reasoning trace; assessment is holistic judgment, not mechanical aggregation.rubric → | 0.52 | n=2 · 0.53 ± 0.02 |
| judge · claim-bar pass rate | 100% | n=2 · 50% ± 71% |
| judge · importance overrated share | 0% | n=2 · 0% ± 0% |
| judge · readability (1–5) | 4.4 | n=2 · 2.2 ± 3.11 |
| judge · reasoning fit (1–5) | 3.9 | n=2 · 1.95 ± 2.76 |
| judge · impartiality (1–5) | 4.3 | n=2 · 2.15 ± 3.04 |
| judge · sycophancy share | 0% | n=2 · 0% ± 0% |
| judge · overhedged share | 0% | n=2 · 0% ± 0% |
| judge · overconfident share | 0% | n=2 · 0% ± 0% |
| judge · canonical-form miss share | 0% | n=2 · 0% ± 0% |
| judge · political bias share | 0% | n=2 · 0% ± 0% |
How the numbers are computed
A pure function over the graph, unit tested: metrics.ts. Shared subclaims are counted once. The two coherence rules are checked mechanically: a claim cannot be verified while a claim it requires is contradicted, and two claims joined by a contradiction edge cannot both be verified.
Authorship metrics (how often the Matcher reworded the Extractor’s proposed canonical formThe one neutral wording of a claim that every source's version maps to. About fifteen words.constitution →) need the proposal stored on the instance, which graphs built before September 2026 lack: those rows read n/a.
What it cannot show
Whether any claim is well stated, well decomposed, or fairly judged. The counts say where to look; a wording of fifteen words can still be wrong in every way §3 cares about.
Run it
npm run corpus:score -- blackholes --no-judge # after a run; writes scorecard.json