Current AI systems have research taste comparable to top human AI researchers.
Assessed
First pass under a tight budget. Left atomic: no named arguments, because the evidence is a small set of benchmarks that can be weighed directly. Recorded three instances (TasteVal affirms partially; TASTE and IdeaTaste deny). Assessed doubtful, credence 0.2: the top model is at expert level on the experimental component, but there are large gaps on choosing and judging research directions. A future pass could add subclaims separating experimental taste from direction-selection taste, and read the full papers.
Assessed Doubtful
verdict confidence 0.60 · credence 0.20
Research taste is usually taken to have several parts: choosing which problems are worth pursuing, designing experiments, and interpreting their results. The claim holds that current AI systems match top human AI researchers across this whole capacity. As of late 2026 the evidence is split by component, and on the components that most define "taste" in the usual sense, it weighs against the claim. The strongest evidence for the claim is TasteVal (Jaffe and Sherburn, October 2026). It measures only experimental taste: how efficiently a model designs and interprets experiments on a research problem it is given, with a fixed coding agent doing the implementation. On its eight unreleased frontier-R&D tasks, the best model (Opus 5.5) beat the best of 24 recruited human experts, reaching the same score with about 2.3 times less serial compute (95% interval 1.15 to 4.37). The paper also reports that this compute multiplier has been doubling roughly every three months since December 2025. So on the experimental component, the leading model plausibly matches or exceeds strong experts. Two caveats apply: the sample of tasks is small, and the baseline experts are not necessarily the top researchers at frontier labs. On judging and choosing research directions, the evidence goes the other way. On Anthropic's TASTE benchmark (August 2026), leading models predicted experienced AI safety researchers' preferences between research proposals at near chance. On IdeaTaste (November 2026), LLM agents given a 2025 literature snapshot reached only about 18% of the human level at anticipating the ideas research communities actually published in 2026. Earlier work found that LLM ideas judged novel before execution lost more ground than expert ideas once they were carried out. Read as a claim about research taste in its full sense, comparable to the best human researchers, it is therefore doubtful. Measurement of taste is still immature, and the experimental component is improving fast, so this verdict could change within months. The most direct test would be blinded comparisons of how frontier models and top lab researchers choose research directions, scored by the outcomes that follow.
Claim entered the graph