Evidence on whether AI agents can carry out open-ended AI research is thin.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
The claim holds that the evidence on whether AI agents can do open-ended AI research is limited. "Thin" is a judgment, but the record bears it out. Most published evaluations of AI research agents test narrow, scorable tasks: Kaggle-style competitions (MLE-bench), short engineering tasks against a clock (RE-Bench) and replications of known papers (PaperBench). None of these asks an agent to pick a question, design a study and judge whether the result is worth publishing. The other main line of evidence is AI-written papers submitted to peer review, such as the AI Scientist papers at workshop venues and the Agents4Science conference. Reviewer verdicts are noisy, and they show little about how much of the work the agent itself did.
The 2026 "shadow evaluation" study adds a third kind of evidence. Frontier agents were set the central question of an unpublished paper, and the paper's own authors graded what they produced. The agents did all the engineering but made little progress on the research questions. That result covers only two cases. Even with it, direct evidence on open-ended research capability comes from a handful of studies with small samples, so it remains thin. Several large, independent evaluations of the same kind would weaken the claim.
Full reasoning: the evidence and decisions behind this verdict
Prior: claims that evidence on a new capability frontier is thin are usually right while evaluation methods are still being built. Known evaluation lines as of late 2026: (1) narrow, verifiable benchmarks (MLE-bench, RE-Bench, PaperBench, METR time-horizon tasks), which measure engineering or replication rather than open-ended research; (2) peer review of AI-generated papers (Sakana's AI Scientist v2 workshop paper, the Agents4Science 2025 conference), where review is noisy and gives a weak signal about how far the agent drove the work; (3) studies of ideation (Si et al. on LLM-generated research ideas and their execution), which address only part of the research lifecycle. Kirgis et al. 2026 (arXiv 2607.27191) state the claim in their abstract and add two shadow evaluations. Two cases make a small sample, and the authors call their own results "early evidence". The source has an interest in the gap it fills, since it motivates a new method, but the description of existing evaluations matches the literature as known. No systematic search for large-scale open-ended evaluations was done in this pass, so the credence is short of near-certain: a large lab-internal or independent evaluation that this pass missed would undercut the claim.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
Its one source has been traced to an origin, or to one another.
But evidence on whether agents can carry out open-ended AI research is thin.
Abstract opening, motivating the study.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="d91e5ad8-8ffd-4fb2-8507-ee86b736c75f"><a href="https://minerval.ai/claims/d91e5ad8-8ffd-4fb2-8507-ee86b736c75f">Evidence on whether AI agents can carry out open-ended AI research is thin.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/d91e5ad8-8ffd-4fb2-8507-ee86b736c75f" title="Minerval: Evidence on whether AI agents can carry out open-ended AI research is thin." width="480" height="200" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/d91e5ad8-8ffd-4fb2-8507-ee86b736c75f)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Stated views None yet. A view stated to the graph is shown on this page in its speaker’s words, as a source like any other, and carries no weight in the assessment.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Nov 11, 2026. Every judgment on this page is accompanied by a reasoning trace.