Weak research judgment in AI systems will slow automation of the remaining AI R&D tasks below current-trend extrapolations.

The claim is a forecast: that AI systems' weak research judgment (choosing goals and next steps, redesigning after flaws, backtracking, interpreting results) will hold back automation of the AI R&D work not yet automated, so that it proceeds more slowly than simple extrapolations of current trends suggest. Those extrapolations, built mostly on company-reported metrics, put most AI R&D in AI hands by 2027 or 2028.

The case for it is substantial. By late 2026 AI writes most code and does several agent-workdays per human workday at the leading labs, yet humans still supply most research judgment↗︎, and Anthropic itself reports large gaps in its models choosing goals. Independent "shadow evaluations" in which agents tackled the question of an unpublished paper found recurring judgment failures↗︎: underpowered results presented as findings, caveats added instead of redesign, early abandonment of ambitious directions; the original authors rejected all four AI-written papers. On open-ended reinvention tasks agents recover a small fraction of a published innovation's gain. When fast-moving components are automated, the slowest remaining component tends to govern overall pace.

That bottleneck argument needs the residual work to be mostly judgment, and the evidence only leans that way. That most remaining tasks require research judgment↗︎ is supported but weakly: OpenAI's agents completed only 35% of 8 to 16 hour tasks without human help, no measured part of Anthropic's R&D is fully autonomous, and Anthropic places its judgment gaps "in both engineering and research". Long-horizon engineering is therefore a real share of what remains. That share matters in two ways. Where the residual is engineering, its pace is the pace the trend extrapolations already track, so the judgment bottleneck governs only part of it. And where long engineering tasks stall on judgment inside them, the trend data on task length already reflect how fast that kind of judgment improves. The claim holds only if broad, open-ended judgment is a distinct component that improves more slowly than what current trends measure.

The case against is that measured judgment is improving quickly↗︎: Anthropic's internal next-step test rose from 51% to 64% preference over researchers between two model generations, and the external TasteVal benchmark shows experiment-selection efficiency improving on a steepening trend, with the best model above an expert baseline. These measures test narrow judgment inside a framed problem, whereas the failure reports test broad judgment about what is worth pursuing and when a design is broken. Narrow judgment is plainly improving; whether broad judgment follows at the same pace has not been measured.

The question therefore turns on an unobserved rate, and on how large the judgment-heavy share of the residual is. Some slowing relative to naive extrapolation remains slightly more likely than not, but the evidence does not settle it. Repeated open-ended research evaluations across model generations, and a breakdown of human interventions on unfinished agent tasks by kind, would.