As of 2026, frontier AI agents can perform research engineering but struggle with critical stages of the AI research lifecycle.
Since this assessment: 1 premise judged again, 2 sources added.
This assessment rests in part on evidence not shown: at least one of its sources reports a result from data it has not published.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim has two halves. The first is that the best AI agents of 2026 can do the engineering work of AI research: implementing methods, running and debugging experiments, and producing a manuscript. The second is that they struggle at stages a research project cannot do without, such as choosing what to investigate, deciding what evidence would settle a question, and abandoning an approach that is failing. The evidence favours both halves, the first more firmly than the second.
The most direct test is a mid-2026 study. Agents were given the central questions of two unpublished NeurIPS submissions, six days, and several thousand dollars of compute, and the papers' own authors graded the results. Across three runs on two model families, the agents did the engineering with only logistical human help: literature review, GPU debugging, hundreds of experiments, a compiled paper. The graph's stronger claim that agents can autonomously complete the engineering work of AI research projects is judged supported on a capability reading, though not on a reading that requires reliable completion across typical projects. The first half of this claim asks less than that one, because "can perform research engineering" does not require whole-project autonomy or reliability. So it holds with more confidence. Both papers were scored as clear rejections, and the logs show recurring failures: underpowered experiments on hand-picked or synthetic data, negative results presented as findings, no project-level change of course after the first day, and less than half the budget used. The study is small. Its graders knew the work was AI-generated, and it was run by a team that says it is known for doubting rapid AI self-improvement.
It does not stand alone. Anthropic has every commercial reason to describe its models as capable. It reports that they match or beat skilled humans at executing well-specified experiments, while large gaps persist in judgment about which goals to pursue. An independent October 2026 study of four open-ended AI research tasks found agents stronger, and improving faster, on engineering-heavy tasks than on conceptual ones. Benchmarks of judging research directions put models well below experienced researchers.
Two lines of evidence point the other way. First, a few AI-generated papers have passed peer review, and some system papers claim expert-level autonomous research. But acceptance at a noisy venue is weak evidence, the acceptances are few and selected, and the strongest recent claims rest on automated reviewers. In the direct study, such reviewers accepted drafts that the human experts rejected. More serious is the pace of change. A benchmark published in October 2026 found the newest model exceeding recruited experts at designing and interpreting experiments on a fixed problem, which is one judgment-dependent stage. That does not show research taste comparable to top researchers overall, and no open-ended evaluation of the models released in the last months of 2026 has been published. The claim is therefore well supported for the frontier through about mid-2026 and less certain for the newest systems. Direct evaluations of those systems on unpublished research questions, which the study's authors say they plan, would settle it.
Full reasoning: the evidence and decisions behind this verdict
This pass was prompted by the reassessment of the stronger neighbouring claim that agents can autonomously complete the engineering work of AI research projects, from doubtful (0.3) to supported (0.6). The earlier assessment here was written before that change. It said the neighbour stood as doubtful on a pass that had not weighed the Kirgis et al. evidence. That description is now out of date and has been corrected. The verdict does not change.
Reconciling the two judgments. The neighbour reads "can" as demonstrated capability and rests mainly on the same study (Kirgis, Kapoor, Narayanan et al., arXiv 2607.27191v2, section 4.5): in all three runs the agents resolved environment problems themselves and carried out the experiments, and human interventions were logistical. The neighbour notes that on a reliability reading (typical projects, consistent completion) it would still be doubtful. This claim's first half, "can perform research engineering," is weaker than the neighbour on both axes. It does not require whole-project autonomy, and it does not require reliability. So evidence for the neighbour carries over here, and doubts about the neighbour's reliability reading do not. The 0.6 on the neighbour and roughly 0.93 on this first half are jointly coherent, because this half is implied by the neighbour and also has independent support: saturated reproduction benchmarks, the developer's reports on well-specified experiment execution, and the engineering-heavy side of the independent four-task study. What the successful runs establish is the occasional, demonstrated completion of engineering on real research questions. They do not establish broad, reliable capability across projects, and this claim does not assert that.
The second half is unaffected by the neighbour's change. It rests on the expert-graded rejections, the recurring failure modes, the developer's admission of judgment gaps, and third-party taste and direction benchmarks, against the October 2026 TasteVal result for the newest model on fixed problems. Credence in the second half, read as covering late 2026, is about 0.87. The joint credence is about 0.82, as before.
Sources were as in the previous pass (Kirgis et al. read in full in sections 4 to 8; the Anthropic Institute report read in part; Jackson et al., October 2026, summary and methods; abstracts of TasteVal, rTaste, Taste-Bench and ScientistTwo). No new sources were opened in this pass, because the change was in the graph's judgment of a premise, not in the evidence.
What would change the verdict: expert-graded, open-ended evaluations of late-2026 models that produce acceptable research, or evidence that the engineering success in the case studies depended on unreported human help.
Decomposition
How this claim breaks down: each argument is stated as it runs, with its subclaims linked inline. ↗︎ opens a subclaim; the map shows how they fit together.
In shadow evaluations run in mid-2026, frontier agents were given the central research questions of two unpublished NeurIPS submissions, six days and several thousand dollars of compute, and the papers' own authors graded the output. The agents finished the engineering with only logistical human help, which is evidence that agents can complete the engineering work of AI research projects. Both papers were rejected, and the agents failed in recurring ways at research judgment, redesign, backtracking, resource awareness and instruction following. The models and scaffolds tested were among the strongest available and the failures were not artifacts of the setup or of the graders' knowledge that the work was AI-generated. So frontier agents can do research engineering but struggle with stages open-ended research depends on.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 2 supported
Granting its premises, the conclusion is much more likely than not, because an expert-graded test on fresh research questions is the most direct evidence available on both halves of the claim. The weight rests on the step from three runs on two papers to frontier agents in general. The study's own checks support that step for mid-2026 models: a second model family and scaffold, generous budgets, and failures that did not track scaffold crashes. But the graders were not blind, and models released later in the year were not tested. Autonomous completion of research engineering is now judged supported as a statement of demonstrated capability, though not of reliable completion across typical projects. The argument needs only the weaker point, that agents did the engineering in these projects, so that premise holds for its purposes. The recurring failure modes have not yet been assessed separately, and the second half of the conclusion leans on them.
Automated systems have produced papers end to end that human reviewers accepted: one AI Scientist-v2 paper cleared the acceptance threshold at an ICLR 2025 workshop, and Intology reported a Zochi paper accepted at ACL 2025. Producing an acceptable paper takes every stage of the research lifecycle, from choosing a question to interpreting results. Acceptance by peer reviewers is good evidence that the research was done competently and without human help, contrary to the view that such acceptance is weak evidence of autonomous research ability. So frontier agents do not struggle with critical stages of the lifecycle.
- Does it follow?
- slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
- Do the premises hold?
- 1 not separately assessed
Even granting that acceptance is good evidence of competent unaided research, a handful of acceptances shows that agents sometimes carry a small project through. It does not show that they do not struggle. Only one of three submissions cleared a workshop threshold, the main-conference case involved humans in preparation and rebuttal, and the number of rejected attempts is not reported. The argument also depends on denying that such acceptance is weak evidence of autonomous research ability, which has not yet been assessed separately. Reviewer agreement at these venues is known to be low, and in the direct evaluations an automated reviewer accepted drafts that subject experts rejected.
Current AI systems have research taste comparable to top human AI researchers: on TasteVal, an October 2026 benchmark of experiment design and interpretation on a fixed problem, the best model exceeded the best of 24 recruited experts. The stages at which earlier evaluations found agents failing, such as choosing experiments, judging evidence and changing course, are exercises of research taste. Those evaluations tested models released earlier in the year. So the frontier agents of late 2026 do not struggle with those stages, and the earlier failures no longer describe the frontier.
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 doubtful
Granting its premises, the conclusion would probably follow, since an agent with the research taste of a top researcher would not be expected to make the errors of judgment the earlier evaluations recorded. The argument stands or falls with parity in research taste, which the evidence currently weighs against. The benchmark it cites measures only experiment design and interpretation on a fixed, scored problem, with a separate agent doing the coding, against recruited experts. Measures of choosing and judging research directions still show models well below experienced researchers. It remains the most serious consideration against the claim, because it concerns models newer than those tested in open-ended evaluations.
Provenance
Where this claim has been said, linked to its canonical form.
The clearest statement of the claim comes from one study of two research questions, graded by the original papers' authors, from a team that discloses its scepticism about rapid AI self-improvement; it releases reviews and logs for one of the two papers. A developer's report says much the same about its own models from internal data it has not published, and its admission of a gap in judgment runs against its interest. The one recorded denial is a system paper by the system's developers that judges its output with automated reviewers.
None of its 5 sources has been traced to an origin, or to one another.
In engineering, Claude can be handed an underspecified problem and figure out how to solve it; humans supply the goal, but they no longer need to supply the method. In research, Claude can already match or outperform skilled humans at executing a well-specified experiment. However, large performance gaps persist when it comes to Claude exercising judgement in choosing goals in both engineering and research.
A developer's account of how far its own models have come toward automating AI development, drawing on internal usage data. It reports strong engineering and experiment-execution performance and names judgment in choosing goals as the remaining gap.
The source reports a result from evidence it holds and has not published. The account rests on internal usage data, staff polls and session outcomes scored by the developer's own model, none of it published in a form others can check. The strong engineering figures serve the developer's interest. The statement that large gaps persist in judgment about which goals to pursue runs against the impression the rest of the report gives, and deserves weight for that reason. The first 14,000 characters of the report were read.
Our results provide early evidence that today's agents can do the engineering of AI research, but struggle with critical parts of the research lifecycle.
Concluding sentence of the abstract, after reporting shadow evaluations in which frontier agents attempted the central research questions of two unpublished NeurIPS 2026 submissions.
The source's own evidence bears what it asserts. The statement is hedged as early evidence and the paper's own results bear it at that strength. Three six-day runs on two research questions (two with Opus 4.8 in an open-source scaffold, one with GPT-5.6 Sol in its vendor scaffold) completed the engineering with only logistical human help, and the original papers' authors scored the output 2 of 6 and 1 of 6. The graders knew the work was AI-generated, the strongest Anthropic model was not tested, and the core team states that it is known for doubting imminent recursive self-improvement. Reviews, repositories and logs are released for one of the two papers. Worth reading closely: Sections 4.5, 6, 7 and 8 settle how far the engineering finding and the failure modes can be generalised beyond the two papers and the models tested.
The results show that current agents operate more like engineering optimizers than fully autonomous researchers
Abstract summarizing a systematic evaluation of seven frontier models on 36 long-horizon AI R&D tasks (arXiv, August 2026).
As a result, ScientistTwo autonomously generates expert-level, publishable papers and fully verified, executable codebases. Its solutions consistently outperform human state-of-the-art models, and achieve higher average review ratings than human-authored papers under automated AI review agents. These results show that ScientistTwo is not merely an assistive tool but an autonomous scientific pioneer capable of pushing the frontiers of human discovery.
Abstract of a system paper presenting a multi-agent framework said to run the full research cycle from a problem statement. Paper quality is judged by automated AI review agents, not by human experts.
Only the abstract was read. By its own account the paper judges the quality of generated papers with automated AI review agents, not human experts. In the shadow evaluations, one automated reviewer recommended acceptance of agent-written drafts that the human experts went on to reject, so ratings of this kind are a lenient test. Worth reading closely: Whether any human expert assessed the generated papers, and how the research problems were chosen, would decide how much the paper counts against the claim.
However, AI agents struggle with open-ended research.
Section 'Public evidence points the same way', contrasted with good performance on well-defined ML benchmarks.
Assessment history
0 status changes over 2 assessments. full history →
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="81eeb61b-4ce4-4d0b-891d-11e970d8f669"><a href="https://minerval.ai/claims/81eeb61b-4ce4-4d0b-891d-11e970d8f669">As of 2026, frontier AI agents can perform research engineering but struggle with critical stages of the AI research lifecycle.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/81eeb61b-4ce4-4d0b-891d-11e970d8f669" title="Minerval: As of 2026, frontier AI agents can perform research engineering but struggle with critica…" width="480" height="225" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/81eeb61b-4ce4-4d0b-891d-11e970d8f669)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Stated views None yet. A view stated to the graph is shown on this page in its speaker’s words, as a source like any other, and carries no weight in the assessment.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Nov 11, 2026. Every judgment on this page is accompanied by a reasoning trace.