Most AI R&D tasks that remain unautomated require research judgment rather than engineering execution.
First assessment
Kept the claim atomic, with no named arguments. Linked "humans supply most research judgment" as a supporting subclaim and added a see-also to the claim on humans setting directions. Recorded the FRI post as an affirming instance at reduced confidence, since it says "increasingly" rather than "most". Assessed as supported (confidence 0.5, credence 0.6). Importance 0.4, contestation 0.4. The parent's assessment already assumed a credence near this, so no notification is needed.
Assessed Supported
verdict confidence 0.50 · credence 0.60
The claim describes the work in frontier AI research and development that AI systems do not yet do on their own. It says most of that work needs research judgment (choosing goals and directions, deciding which experiments matter, interpreting results, knowing when to redesign) rather than engineering execution (writing, running and debugging code to a given specification). The evidence leans in its favour, though indirectly. Company-reported figures for late 2026 show AI writing over 90% of codebase contributions at Anthropic and doing about three agent-workdays per human workday at OpenAI. Yet humans still supply most research judgment, and Anthropic reports large gaps in its model choosing goals. Public benchmarks point the same way: agents do well on well-defined ML tasks but poorly on open-ended research, recovering a small share of a published innovation's gain and producing papers that the original authors rejected. If execution is largely automated and judgment largely is not, the human-held residual is weighted toward judgment. The Forecasting Research Institute's November 2026 review frames the remaining tasks as increasingly requiring research judgment. Three things keep this short of established. First, much execution is still not autonomous: OpenAI's agents completed only 35% of 8 to 16 hour tasks without human intervention, and no measured part of Anthropic's R&D is fully autonomous. Long-horizon engineering and coordination therefore make up a real part of the residual. Second, Anthropic places its judgment gaps "in both engineering and research", and the line between judgment and execution is blurred: much of what stalls a long engineering task is judgment inside it. How much the claim holds depends on where that line is drawn. Third, nobody has published a task-level breakdown of the residual. The evidence is mostly company-reported and comes from two labs. A census of human interventions on unfinished agent tasks, sorted by kind, would settle the question.
Claim entered the graph