Epoch gate
Before a prompt or model change ships, its scored runs against the committed baseline, with the noise band applied to every headline metric.
What it checks
The comparison rule, made into a verdict. It reads the committed scorecards of a cluster, takes a baseline group and a candidate group, applies the band to every headline metric in the direction that counts as better for it, and exits non-zero on a regression of a gated metric: the claim-bar pass rate, coherence violations, the dedup ratio, and the share of assessments with a trace.
It refuses, printing the deltas for reading, when a side has fewer than two runs or the sides differ in profile or epoch: a delta across configurations is a different graph, not a regression. A passed gate means not shown to regress, never shown equal.
Detail
How it chooses the sides
- Baseline. --baseline=<files>, else corpus/scorecards/<cluster>/baselines.json, else the earliest runs sharing the earliest run's epoch and profile.
- Candidate. --candidate=<files|latest>, else the newest runs outside the baseline sharing the newest run's fingerprint.
- Band. for each metric the mean and sample spread per side; a delta counts only beyond the combined spread.
- Verdict. regressed, improved, within band, moved (for undirected metrics such as max depth), or no verdict.
What it cannot show
Anything with one run on a side. Undirected metrics are reported as moved and never gated. It is file-only: it reads no database and registers nothing.
Run it
npm run corpus:gate -- blackholes
npm run corpus:gate -- blackholes --candidate=latest --min-n=2 --gated=claim-bar,coherence