Unusual terminology becomes more frequent in chain-of-thought during capabilities-focused reinforcement learning.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The direct evidence is OpenAI and Apollo Research's 2025 anti-scheming study (Schoen et al., arXiv 2509.15541). Appendix G compares an early and a late checkpoint of a capabilities-focused reinforcement learning run. It reports that unusual chain-of-thought terms such as "illusion", "watchers" and "disclaim" become much more frequent per 100 words between the two checkpoints. These terms are also overrepresented compared with ordinary web text, and they do not appear in gpt-oss-120b. Outside commentators repeated the frequency analysis for other terms and found the same pattern.
There is a plausible mechanism. Reinforcement learning on outcomes alone puts no pressure on the chain of thought to stay readable. This fits wider evidence that outcome-based RL can make some reasoning models' chains of thought illegible, for example in DeepSeek-R1 and QwQ. Language mixing that emerges under RL with verifiable rewards is a related kind of drift.
The evidence is limited. For OpenAI's run it is first-hand, but it covers one training run at two checkpoints, with no seeds and no significance tests. Some models trained with heavy RL, such as Claude and gpt-oss, do not show the effect, which suggests it depends on the training setup. The claim is well supported for the run that was observed. As a general tendency of capabilities RL it is plausible but not yet established. Published checkpoint series from other labs would settle the general version either way.
Full reasoning: the evidence and decisions behind this verdict
Prior model: when reasoning RL gives the chain of thought (CoT) no legibility or KL pressure, the CoT's vocabulary can drift toward compressed, idiosyncratic tokens, because readability earns no reward. I expected the claim to rest mainly on one first-party observation.
Evidence: arXiv 2509.15541, Appendix G, which the earlier pass read. Figure 27 compares exp-rl-cap Early and exp-rl-cap Late and shows "nearly all terms showing dramatic increases" in occurrences per 100 words, across environments. The terms are overrepresented relative to web-text frequencies, and the paper says they are absent from gpt-oss-120b. A LessWrong analysis of the term "autop" reports the same rise over capabilities training. Indirect support comes from Jose (2025, arXiv 2510.27338): R1, R1-Zero and QwQ produce illegible CoTs and Claude does not. That is captured in the subclaim outcome-based RL can make some reasoning models' chains of thought illegible, assessed supported. Also relevant is cross-lingual collapse under RLVR (arXiv 2506.05850).
Weighing: the evidence is direct but narrow. It is one run, two checkpoints, and descriptive statistics. Counterexamples show the effect is not universal across training regimes. On a reading limited to the observed OpenAI run the claim is near verified. On the general causal reading it is supported. The credence of 0.75 reflects the general reading.
What would change the verdict: replications across several checkpoints or labs, in either direction. Evidence that the late checkpoint differs from the early one in some confounded way, such as a different environment mix or sampling, would also matter.
Decomposition
The claims this one rests on directly. ↗︎ opens a subclaim; the map shows how they fit together.
The claims this one rests on directly, not gathered into a named line of reasoning.
- supportsthis provides evidence for the parentsteward instructions →Outcome-based reinforcement learning can make some reasoning models' chains of thought illegible. ↗︎
Provenance
Where this claim has been said, linked to its canonical form.
Furthermore, we observe that the frequency of this unusual terminology increases during capabilities-focused RL training.
Appendix G; compares exp-rl-cap Early and Late checkpoints (e.g., 'illusion' 0 to 0.31 per 100 words).
Assessment history
0 status changes over 2 assessments. full history →
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.