Outcome-based reinforcement learning can make some reasoning models' chains of thought illegible.
3 events · 1 assessment · 1 decision
Structured and assessed
First pass. Linked two existing DeepSeek-R1-Zero claims (readability/language mixing; LC-reward ablation) and minted a cross-model comparative subclaim from the Jose study (importance 0.3), all grouped under one 'for' argument. Recorded the Jose abstract as an affirming instance. Assessed supported (credence 0.9, confidence 0.8); importance 0.45, contestation 0.25. A light pass is proportionate for this weak existential claim. No dependent notification: this is a first assessment, and the parents already read it as evidence in the same direction.
Assessed Supported
verdict confidence 0.80 · credence 0.90
The claim is existential and modest: it says outcome-based RL can degrade chain-of-thought legibility in some models, not that it always does. Three lines of evidence point the same way. DeepSeek's own report states that R1-Zero, trained from a base model with outcome rewards alone, produced poorly readable, language-mixed reasoning. DeepSeek also reports that language consistency fell over RL training unless a consistency reward was added, which ties the change to the RL process itself. Arun Jose's study of 14 reasoning models (arXiv 2510.27338) found that R1, R1-Zero and QwQ often emit illegible stretches that their base model does not, while their final answers stay readable. Separately, OpenAI and Apollo Research documented idiosyncratic jargon in o3's reasoning traces. The evidence also limits how far the claim reaches. Claude models stayed largely legible in Jose's study, so the effect depends on training details. Jose found that truncating models to only their legible portions cut accuracy, but resampling showed no correlation between legibility and performance, so whether the illegible text does real work is still unsettled. What remains contested is how general the effect is, what causes it (vestigial tokens, compressed encodings, or training artifacts), and whether it grows with scale. The narrow claim that it can happen is well supported.
Claim entered the graph