Assessment history
Two things a graph's own record of verdicts says about the Steward, with no answer key and no second run.
What it checks
Evidence monotonicity. An accepted supporting contribution should not lower a claim’s credenceThe Steward's probability that a claim is true.constitution →, and an accepted challenge should not raise it. The sign of the move is checked, not its size. Each accepted support, source instance or challenge is linked to the reassessment that followed its review, and every wrong-signed move is listed by claim and contribution.
Overturn-rate discrimination. Bin every claim by the credence its first assessment gave it. For each bin, how often was the claim later materially changed, and how often reversed (credence crossed 0.5, or the status flipped between the supported and the contradicted side)? If claims at 0.9 reverse as often as claims at 0.6, the credences are not telling anything apart. This is the only falsification signal available for the contested core of the graph without an outside referent.
Detail
How it is computed, step by step
- Read the record. the assessment history of every claim, ordered by time, and the accepted contributions with their review time and type.
- Link contribution to reassessment. the first assessment after the review, preferring one whose trigger is contribution_accepted; a reassessment that integrated contributions pulling both ways is skipped as ambiguous.
- Check the sign. support and add_instance expect a non-negative credence move; challenge expects a non-positive one.
- Bin and count. first credence in 0 to 0.2, 0.2 to 0.4, and so on, plus bins by distance from 0.5; per bin, the share later materially changed and the share reversed; bins under --min-bin are flagged as too small.
- Write and register. history.json in the run directory and a row of kind history.
What it cannot show
- A graph straight out of one ingest has nothing to reverse; run it after a contribution scenario, a staleness sweep or the fixpoint property.
- The contribution-to-reassessment link is by time, so a reassessment with another cause in the same window can be attributed to the contribution.
- An added instance is treated as affirming; the contribution row carries no stance.
- Inconclusive and unknown never count as a polarity flip.
Run it
npm run corpus:contributions -- blackholes # or corpus:property -- fixpoint blackholes
npm run corpus:history -- db --min-bin=5