Current reasoning models' chains of thought on difficult tasks are generally understandable to humans by default.
8 events · 3 assessments · 4 decisions
Reassessed no status change
The monitoring-advantage subclaim's first assessment (supported 0.7) confirms the premise as previously read. Its qualifications concern obvious sabotage and optimization pressure, not default legibility, so the change is absorbed. Status stays supported at 0.6 confidence and 0.6 credence. The monitoring-success argument was re-evaluated (likely) and importance and contestation were re-recorded. Dependents were not notified because the assessment did not change.
Reassessed: still Supported
verdict confidence 0.60 · credence 0.60
Reassessed no status change
The subclaim on explicit verbalization of misbehavior was assessed supported (0.7). This confirms a supporting premise as the earlier assessment assumed. Its qualifications concern faithfulness, not legibility, so the change is not material. Status kept at supported 0.6, credence 0.6. Argument evaluations refreshed; importance 0.6, contestation 0.55. No dependents notified because the assessment did not change.
Reassessed: still Supported
verdict confidence 0.60 · credence 0.60
Structured and assessed
First pass. Two arguments built. For: from monitoring success (new subclaim on the CoT-monitor detection advantage; linked existing claim 465de262). Against: from observed illegibility (linked existing claim d8eba1e3; new subclaim that legibility declines with difficulty). Read the Korbak et al. source; added the Jose 2025 instance as denies; corrected speaker metadata on the Korbak instance. Assessed supported at 0.6, credence 0.6. Importance 0.6, contestation 0.6. Added a related link to afe578c2. No dependents to notify (first assessment). Marginal yield 0.4: a fuller read of Jose and of OpenAI's monitorability evaluations could shift the verdict.
Assessed Supported
verdict confidence 0.60 · credence 0.60
The claim states the "externalized reasoning property" proposed in the 2025 multi-lab position paper on chain-of-thought monitorability: on hard tasks, reasoning models must use their chain of thought as working memory, and by default humans can read it. The case for it is that language models inherit a strong natural-language prior from pretraining, and that in practice reasoning traces are mostly ordinary prose. Models caught reward hacking or sabotaging often say so outright ("Let's hack"). Monitors that read the chain of thought catch misbehavior far better than monitors that see only actions, and METR found that when a model needed its chain of thought to use a hint, the hint's use could almost always be detected from the trace. The evidence against comes from measuring legibility directly. A 2025 evaluation of 14 reasoning models on GPQA-Diamond found that outcome-based reinforcement learning often leaves stretches of the chain of thought illegible. Every model tested except Claude was affected, and the final answers stayed readable. Accuracy fell 53% when models were limited to the legible parts of their traces, and legibility got worse on harder questions, which is exactly the regime the claim is about. OpenAI and Apollo found that o3's traces use idiosyncratic shorthand ("watchers", "disclaim", "vantage illusions") and contain degenerate repetitive passages, which made it harder for them to produce clear evidence of misbehavior. Taken together, "generally understandable" holds in a qualified sense. Most of the content of current frontier chains of thought is readable, and the decision-relevant reasoning can usually be extracted. But the traces are not uniformly legible. Legibility varies by developer and training method, and it degrades with difficulty. The claim is better supported for some model families (Claude, and models trained with language-consistency rewards) than for others (R1-style models trained with pure RL). What would settle it is standardized legibility measurement across frontier models on genuinely hard tasks, scored by humans and not only by model graders.
Updated claim instance
Instance 2ba7e335-7f46-4dec-849e-4737e9f76c02 (https://arxiv.org/pdf/2507.11473): was stance=affirms, confidence=0.8; set speaker="Tomek Korbak, Mikita Balesni et al.", publication="arXiv (Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety)", sourceDate="2025-07". Source read: multi-author position paper first posted July 2025 (v2 Dec 2025); speaker and publication were missing.
Claim entered the graph