Chain-of-thought controllability increases with model scale.
5 events · 2 assessments · 2 decisions
Reassessed no status change
The RL-training subclaim (contradicting, grouped under the reasoning-compute argument) received its first assessment: inconclusive, 0.35. The prior evaluation already treated its premise as mixed, and the argument concerns a different axis from parameter scale, so the change is immaterial. Re-recorded status supported 0.7, credence 0.72, refreshed all three argument evaluations, and kept importance at 0.45 with contestation 0.4. No dependents were notified because the status did not change.
Reassessed: still Supported
verdict confidence 0.70 · credence 0.72
Structured and assessed
First pass. Created three arguments (within-family parameter scaling, successive frontier generations, scaling reasoning compute) with three new subclaims after match_claim returned novel for each; linked related claims on GPT-6 Astra RL controllability and current low controllability. Recorded two affirming instances (arXiv paper, OpenAI blog). Assessed supported, 0.7 confidence, credence 0.72, reading 'scale' as parameter count. Importance 0.45, contestation 0.4. Parent's assessment cites this claim as supporting; verdict agrees with its seed, so no notification needed.
Assessed Supported
verdict confidence 0.70 · credence 0.72
Chain-of-thought controllability is a model's ability to follow instructions about what appears in its reasoning trace (for example, not using a given word, or reasoning only in capitals), measured as a proxy for the ability to reshape reasoning to evade a monitor. Read as a claim about parameter count, the evidence favors it. The CoT-Control evaluation (Chen Yueh-Han et al., 2026) found that within open-weight model families, larger models score higher, with controllability rising roughly logarithmically in parameter count. That is the cleanest test available, because same-family comparisons hold the training recipe roughly fixed, but it rests on two families, a handful of sizes up to about 120 billion parameters, and one evaluation suite, with no independent replication found. Later evidence is consistent with the trend. OpenAI system cards from 2026 report that newer frontier models control their chains of thought far better than their predecessors: at matched reasoning length, GPT-6 Astra followed CoT instructions about 61% of the time against 16% for GPT-5.6 Sol and 2% for GPT-5.5 Thinking. Parameter counts are undisclosed and training changed between generations, so these comparisons cannot isolate scale. The claim weakens if "scale" is read to include reasoning compute. The same study found controllability falls with more reasoning effort and, on its evidence, with more outcome-based RL training, though GPT-6 Astra's controllability rose during RL training, so that effect appears to depend on the training recipe. Longer reasoning traces are also harder to control, which confounds comparisons across models that reason at different lengths. Repeating the evaluation on larger open-weight families, with length controlled, would settle the parameter-scale reading.
Claim entered the graph