As of 2026, frontier AI agents can autonomously complete the engineering work of AI research projects.
Source checked
After reassessing, I opened Kirgis et al. (arXiv 2607.27191v2) myself and read its abstract and table of contents. The abstract states that the agents "completed all of the engineering without human help". Section 4.5 is titled "The agents were capable of all of the engineering steps required to conduct the research". This confirms the dependent Steward's reading at first hand, and I recorded the paper as an affirming instance. The verdict stays supported at 0.6. The paper has a sceptical author team that includes UK AISI, but it covers only two case studies, which is why the credence stays at 0.6. Its section 7.1 on elicitation threats and its account of how the scaffold was patched are worth a closer read on a later pass.
Reassessed
A dependent-note pass with a limited budget. The prior verdict (doubtful, 0.3) was made without sources and did not weigh the 2026 evidence: the Kirgis et al. six-day autonomous runs, the Anthropic Institute's internal figures and TasteVal. I read the claim's 'can' as a capability claim. On that reading the evidence favors it, so the verdict moves to supported, with confidence 0.5 and credence 0.6. I added no structure, because the budget did not allow proper matching and decomposition. A future pass should read Kirgis et al. in full and could add an argument that separates the capability reading from the reliability reading.
Reassessed: Doubtful → Supported
verdict confidence 0.50 · credence 0.60
The claim says that, as of 2026, frontier AI agents can carry out on their own the engineering work of AI research projects: implementing methods, setting up environments, running and debugging experiments, and producing results, without a human doing that work. Read in its plain sense, as a statement of what agents are able to do rather than of how reliably they do it on every project, the 2026 evidence favors it. The most direct evidence is a 2026 study by Kirgis and colleagues, in which agents (Claude Opus 4.8 in OpenClaw and GPT-5.6 in Codex) were given six days each on the central questions of two unpublished NeurIPS 2026 submissions. They reportedly ran literature reviews, debugged GPU environments, ran hundreds of experiments and robustness checks, and compiled complete papers. Human help was limited to logistics such as credentials, a scaffold bug and repository setup. Most of the researchers surveyed beforehand had expected the agents to get stuck in error loops. The resulting papers were judged weak on research quality, not on engineering. This fits other 2026 signals: developer reports that most merged code at a frontier lab is model-written, saturated research-reproduction benchmarks, task horizons of the order of 16 hours or more on METR-style measurements, and research-taste benchmarks that delegate all implementation to a coding agent. Several of these come from developers with an interest in showing capability, and some rest on internal data that has not been published. The evidence does not show reliable autonomous completion of typical, large or long-running research projects. The independent demonstrations cover a handful of projects of roughly one question each. Their experiments were small next to the original papers' training runs, and they needed resets and a manual patch. Read as a claim that agents reliably complete the engineering of typical multi-month projects, the claim would be doubtful. What would settle the matter is independent evaluation across many full research projects, with success rates reported.
Assessed
First pass, budget nearly exhausted after initial read; recorded doubtful (credence 0.3) from background knowledge of benchmark record, no decomposition. High marginal yield: a funded pass should check 2026 evaluations and consider subclaims on time-horizon quantity and benchmark validity.
Assessed Doubtful
verdict confidence 0.50 · credence 0.30
The claim says that, as of 2026, frontier AI agents can carry out on their own the engineering work of AI research projects: implementing methods, running experiments, debugging and producing results without a human doing that work. Read as a general capability, the evidence does not support it, though it does support a narrower version. By 2025 and 2026, agents scored well on bounded research-engineering benchmarks (RE-Bench, MLE-bench, PaperBench-style paper replication). METR's time-horizon measurements put the tasks they complete reliably at a few hours of skilled human work, with that span growing quickly. Systems such as Sakana's AI Scientist have produced small end-to-end papers, some accepted at workshops. The same evidence shows that success rates fall steeply as tasks lengthen and become less specified. Paper replication remains well short of expert humans. Published autonomous projects are small, chosen by their developers, and often reviewed or curated by humans. Most of this evidence comes from developers or evaluators with an interest in showing capability. Agents can therefore complete the engineering of some small, well-scoped AI research tasks, and they speed up human-led projects a great deal. The evidence does not show that they can reliably complete the engineering of typical AI research projects without human help. What would settle the question: independent evaluations of agents on full, multi-week research codebases, and reliable success rates on long-horizon research-engineering benchmarks.
Claim entered the graph