AI systems are improving at choosing next steps in AI research.
Structured and assessed
First pass. Kept atomic with one named argument (rising scores on next-step/experiment-choice evaluations); placed the unplaced Anthropic passage on it; added TasteVal preprint as an independent instance and argument instance; recorded a provenance reading of the Anthropic report as evidence not shown. Assessed supported, credence 0.85. Importance 0.55, contestation 0.45.
Assessed Supported
verdict confidence 0.75 · credence 0.85
The claim holds that AI systems are getting better at deciding what to do next in AI research: which experiment to run, which lead to follow, when to change course. That is a narrower and harder skill than writing code or running a fixed experiment, and it is the one most often named as the remaining human advantage. Two kinds of evidence, from different hands and by different methods, point the same way. Anthropic reported in 2026 that on 129 moments taken from its researchers' real debugging and investigation sessions, the share of cases where a model's proposed next step was judged better than the researcher's rose from 51% for Claude Opus 4.5 (November 2025) to 64% for Claude Mythos Preview (April 2026). The test used unpublished data, was judged by another Claude model that knew how each session ended, and deliberately picked moments where the human choice was poor, so it shows improvement between models rather than parity with human judgment, and it comes from a developer with a stake in its models looking capable. The external TasteVal benchmark (Jaffe and Sherburn, October 2026), in which the evaluated model chooses experiments while a fixed coding agent runs them, found the efficiency of frontier models relative to human experts rising across 20 models from 2023 to 2026, with the trend steepening since late 2025 and the best model exceeding the expert baseline. Several other 2025-2026 studies of forecasting which research ideas will work report similar gains. The evidence favors the claim, but it is indirect: each test captures a narrow slice of research judgment, the tasks are mostly unreleased, and model-graded judging can drift. Neither test reaches the broadest form of the skill, choosing which problems are worth working on at all, where even the developers reporting gains say humans still lead.
Claim entered the graph