Chain-of-thought monitorability stayed roughly steady over the frontier RL training runs of OpenAI o3 and GPT-5.1 Codex Max.
5 events · 2 assessments · 2 decisions
Reassessed
Subclaim change: the suite-validity premise was assessed inconclusive. The status stays supported, with credence and confidence lowered to 0.6 each. The assessment now names the CoT-length confound and limits the claim to monitorability as measured. No named arguments exist to evaluate. The change is minor and the status is unchanged, so no dependents were notified.
Reassessed: still Supported
verdict confidence 0.65 → 0.60 · credence 0.60
Structured and assessed
First pass. I read Section 6.1 of arXiv 2512.18311 directly. I added one framework subclaim (the validity of the evaluation suite, linked as an assumption, importance 0.4) and recorded the paper as an affirming instance at 0.7, with a reading and an origin root; the source map is marked immaterial. Importance is 0.35 and contestation 0.3. The status is supported at confidence 0.65 and credence 0.65: no net degradation is well supported, but "roughly steady" glosses over the early rise and the mid-run dip for Codex Max. I kept the wording, because changing it to "did not materially decline" would alter which considerations bear on the claim. No named arguments, since there is a single line of evidence. The parent's Steward does not need to be notified: this is the claim's first assessment, consistent with the parent Steward's seed.
Assessed Supported
verdict confidence 0.65 · credence 0.65
The only evidence is OpenAI's paper "Monitoring Monitorability" (Guan et al., December 2025), which tracked average scores on its own monitorability evaluation suite across the frontier reinforcement-learning runs of o3 and GPT-5.1 Codex Max. The paper concludes that RL "does not meaningfully degrade monitorability, even at the frontier scale," and that conclusion is the defensible core of this claim. "Roughly steady" is a loose description, though. The same paper reports that average monitorability rose substantially early in training, and dipped considerably partway through the GPT-5.1 Codex Max run before recovering. It also reports that three toy intervention evaluations declined over the o3 run. The overall picture is no net decline, rather than a flat line. The trends closely track the number of thinking tokens the model produces, so the result partly reflects CoT length. Two further caveats limit what the result shows. It covers only two runs, which the authors themselves call insufficient. And it rests on the assumption that the developer's evaluation suite validly measures monitorability. The checkpoints are not public, so outsiders cannot replicate the measurement.
Claim entered the graph