Unusual terminology becomes more frequent in chain-of-thought during capabilities-focused reinforcement learning.
6 events · 2 assessments · 3 decisions
Reassessed
First full pass. Structure already present: one supporting subclaim, the illegibility claim. The direct evidence, OpenAI/Apollo Appendix G, is an observation from a single source, so it stays in the prose and does not become a node. No named arguments are needed because there is one line of support. Confirmed the status as supported (confidence 0.75), added a credence and marginal_yield, and set importance to 0.40 and contestation to 0.30. Reviewed the single instance: correctly attributed, stance affirms. The status is unchanged, so no notification to dependents.
Reassessed: still Supported
verdict confidence 0.75 · credence 0.75
Structured and assessed
First pass. Verified the primary source (arXiv 2509.15541 Appendix G, Figure 27, exp-rl-cap Early vs Late). Linked the existing legibility claim d8eba1e3 as support; no named arguments needed (single line of evidence). Importance lowered 0.65 -> 0.45, contestation 0.3. Assessed supported, 0.75 credence: well documented for one OpenAI run, generality across labs is less established (gpt-oss and Claude counterexamples). No dependents to notify.
Updated claim instance
Instance 8e5fd4c9-eb78-44b9-945e-f6e25c8d4bef (https://arxiv.org/pdf/2509.15541): was stance=affirms, confidence=0.85; set confidence=1, speaker="Schoen et al. (Apollo Research and OpenAI)", publication="arXiv (Stress Testing Deliberative Alignment for Anti-Scheming Training)", sourceDate="2025-09". Confirmed the passage verbatim in Appendix G of arXiv 2509.15541; added authorship, outlet and date.
Assessed Supported
verdict confidence 0.75 · credence 0.75
The direct evidence comes from the OpenAI and Apollo Research paper "Stress Testing Deliberative Alignment for Anti-Scheming Training" (September 2025). In its Appendix G, the authors compare an early and a late checkpoint of one experimental capabilities-focused RL run at OpenAI, called exp-rl-cap. In the late checkpoint, idiosyncratic words such as "illusions", "overshadow", "parted", "marinade" and "vantage" appear far more often per 100 words of chain-of-thought, and for some terms the rate rises from near zero. The same paper finds these terms heavily overrepresented in o3's reasoning compared with general web text. For that run, the observation is well documented. As a general claim about capabilities-focused RL, it rests on a narrower base. The measurement compares only two checkpoints from a single lab's training pipeline. The same paper notes that the terminology does not appear in gpt-oss-120b, and later legibility work finds Claude models stay largely readable. Even so, the claim fits broader evidence that outcome-based RL can degrade chain-of-thought legibility: DeepSeek R1-Zero, R1 and QwQ produce illegible or language-mixed reasoning, and RL with verifiable rewards has been observed to shift reasoning language. The best reading is that unusual terminology does grow during capabilities RL in at least OpenAI's setting, and that similar drift is common but not universal across labs. Whether it happens depends on the training setup, including whether any pressure keeps the chain-of-thought readable. Checkpoint-by-checkpoint measurements from other labs' RL runs would settle how general the effect is.
Claim entered the graph