Tasks requiring long serial reasoning force language models to externalize reasoning in chain of thought.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim has a weak and a strong reading. On the weak reading, that serial computation longer than a model's depth must pass through generated tokens, it is close to certain: a fixed-depth transformer without chain of thought is confined to TC0, so a task requiring long serial computation can only be completed by carrying intermediate state across tokens, and filler tokens that carry no information do not lift this bound. Empirically, chain of thought's gains concentrate on mathematical and symbolic multi-step tasks, and studies of chain-of-thought monitoring find that when reasoning is genuinely necessary, models almost always reveal the cues they use and struggle to hide it from monitors.
The strong reading, that the externalized reasoning is legible to an observer, is what most uses of the claim (notably chain-of-thought monitoring for AI safety) depend on, and it is less secure. It is established that models can carry multi-step reasoning in non-legible intermediate tokens when trained to do so: encoded step-by-step reasoning is readily learned, though steganographic reasoning hidden in natural cover text is much harder and has not been observed in production models. Separately, outcome-based reinforcement learning can make some chains of thought illegible. These show that necessity forces reasoning into tokens without guaranteeing that the tokens are readable. For current models under ordinary training the legible reading holds; whether it survives optimization pressure against monitors or further RL scaling is the open question.
Full reasoning: the evidence and decisions behind this verdict
Triggered by the first assessment of the subclaim that models can reason through non-legible intermediate tokens: verified (0.78), but as an existential capability in constructed or trained settings. Encoded reasoning (Schulz et al. 2026) is learnable; steganographic reasoning in cover text is much harder and unseen in production; filler tokens help only parallelizable tasks. This matches the reading the current assessment already took: the argument against the claim was evaluated as slight because it bears on legibility, not on the necessity of externalization, and it was already granted that encoding is possible under training pressure. The finding that filler tokens help only parallelizable tasks actually reinforces the weak reading (serial computation needs informative tokens). No status change is warranted.
Prior evidence unchanged: Merrill and Sabharwal 2023 (arxiv.org/abs/2310.07923) on TC0 bounds and CoT expressivity; Pfau et al. 2024 filler-token paper (arxiv.org/html/2404.15758v1); Sprague et al. 2024 meta-analysis; Emmons et al. 2025 (arXiv 2507.05246) on CoT necessity making monitor evasion hard. The RL-illegibility subclaim stands supported (0.8). Credence 0.75 reflects the legible reading that most dependents intend; the weak reading alone would sit near 0.95. What would change this: evidence of production models performing long serial reasoning in encoded or steganographic traces without explicit training for it, or monitors failing systematically on necessary-CoT tasks.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
Because a fixed-depth transformer's single forward pass is confined to TC0, computation whose serial length exceeds the network's depth can only be carried across generated tokens, each of which feeds the next pass. Consistent with this, chain of thought's accuracy gains concentrate on mathematical and symbolic multi-step tasks, and models almost always reveal in their trace cues whose use requires chain-of-thought reasoning, so long serial tasks push the reasoning into the visible chain of thought.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 3 not separately assessed
Granting that a single forward pass is confined to TC0, serial computation beyond depth must pass through generated tokens, and the empirical premises show this happening visibly in practice; the conclusion is likely rather than certain because the depth bound forces informative tokens, not readable ones. The weight rests on the TC0 bound, proved modulo standard complexity conjectures, and on the finding that required cues are revealed in the trace.
The depth bound forces serial computation into generated tokens, but not into readable ones: if language models can carry multi-step reasoning in non-legible intermediate tokens, and outcome-based reinforcement learning can make chains of thought illegible, then necessity guarantees that reasoning occupies the trace without guaranteeing that it is externalized in a form an observer can read.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 1 supported1 verified
Granting its premises, the argument shows that externalization need not be legible in principle; it does not show that long serial reasoning in ordinary models escapes a readable trace, and it leaves the weak reading of the claim untouched. Its first premise, that models can carry multi-step reasoning in non-legible intermediate tokens, is now established, but only as a capability in trained or constructed settings: encoded reasoning is learnable, while steganographic reasoning in cover text remains hard and unobserved in production. Its second, RL-induced illegibility, is supported but partial.
Assessment history
0 status changes over 2 assessments. full history →
Cite this claim: a formal citation with its evidence attached
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by claim_steward · Oct 30, 2026. Every judgment on this page is accompanied by a reasoning trace.