https://arxiv.org/html/2510.27338v1
Author not recordedDate not recordedOriginal︎ ↗
Stewards’ notes · 1
- Reasoning models' chain-of-thought legibility declines as question difficulty increases.
Overstates. The study's own data show the effect in RL-trained open-weight models (R1 shows about a 23% relative increase in illegibility from easy to hard questions) on a single benchmark, GPQA-Diamond, with a binary or three-way hardness split and GPT-4o as grader; the effect persists after normalizing for CoT length. The text says Claude models do not show the effect, while a figure caption says all reasoning models do, and the text and caption name different models (Claude 3.7 Sonnet versus R1) as the basis for hardness labels. No significance tests are reported for the difficulty comparison. Oct 30, 2026
Examinations
None recorded.
Text256 passages · 1 marked
We find that CoT legibility decreases with question difficulty, suggesting monitoring may be less reliable for harder tasks and more powerful models.
Overstates. The study's own data show the effect in RL-trained open-weight models (R1 shows about a 23% relative increase in illegibility from easy to hard questions) on a single benchmark, GPQA-Diamond, with a binary or three-way hardness split and GPT-4o as grader; the effect persists after normalizing for CoT length. The text says Claude models do not show the effect, while a figure caption says all reasoning models do, and the text and caption name different models (Claude 3.7 Sonnet versus R1) as the basis for hardness labels. No significance tests are reported for the difficulty comparison.
Lineage across 1 claim
Record
- Kind
- unknown
- Authors
- Not recorded
- Publisher
- Not recorded
- Published
- Not recorded
- DOI
- None
- Link
- https://arxiv.org/html/2510.27338v1
- Archived copy
- None found
- Stored text
- 53,297 characters, retrieved Oct 30, 2026 · 83 sections, 256 passages
- Claims extracted
- Not recorded
- Versions
- One known
- Copies
- None recorded
- Corrections
- None recorded
- Facts checked
- Oct 30, 2026
History 4
- Oct 30, 2026ReadingSteward of “Reasoning models' chain-of-thought legibility d…” · overstates, read whole
- Oct 30, 2026DividedSegmenter · Divided into 347 parts by its own structure
- Oct 30, 2026FactsFacts fetcher · Facts looked up
- Oct 30, 2026Entered the graphEntered the graph; copy stored