Step 8: tool results

read_page

{
  "url": "https://arxiv.org/abs/2610.06824",
  "max_chars": 4000
}

{"ok":true,"url":"https://arxiv.org/abs/2610.06824","text":"[2610.06824] TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search &middot; Advanced search\n-->\nComputer Science > Artificial Intelligence\narXiv:2610.06824 (cs)\n[Submitted on 5 Oct 2026 ( v1 ), last revised 6 Oct 2026 (this version, v2)]\nTitle: TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts\nAuthors: Oliver Jaffe , Dane Sherburn\nView a PDF of the paper titled TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts, by Oliver Jaffe and 1 other authors\nView PDF\nHTML (experimental)\nAbstract: We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models. We define research taste as the ability to pick interesting problems to solve, design experiments, and interpret experimental results. TasteVal measures the experimental component of research taste; given a fixed research problem, we measure how well a model iteratively designs experiments and draws conclusions from their outcomes. We operationalize experimental research taste as compute efficiency; a Researcher who reaches the same score as an expert human using half the serial experimental compute has twice the experimental taste. Experimental taste thus acts as a multiplier on experimental compute, making it a key input to forecasts of AI progress. TasteVal consists of 8 novel, challenging, open-ended tasks representative of frontier AI R&D. To isolate taste from coding ability, the model under evaluation acts as a Researcher that iteratively designs experiments while a fixed Coder agent implements them and reports their results. The Researcher executes until either the 40 H100 hour or 120 wall-clock hour budgets are exhausted. We recruit 24 human experts, at least 2 per task, and take the best expert attempt per task as the expert baseline. We evaluate 20 models released between 2023 and 2026. The best-performing model, Opus 5.5, exceeds our expert baseline, with a compute multiplier of 2.3x (95% CI 1.15-4.37), at roughly 1/30 of our baseliners' average per-run cost. On TasteVal, the compute multiplier of frontier models has doubled approximately every 3.0 months since December 2025 (95% CI 1.7-5.0), up from every 14 months between 2023 and December 2025. Measured by final normalized performance, frontier models show no trend break, doubling every 14.6 months. To keep TasteVal uncontaminated, we do not release the tasks.\nComments:\n38 pages, 21 figures\nSubjects:\nArtificial Intelligence (cs.AI)\nCite as:\narXiv:2610.06824 [cs.AI]\n(or\narXiv:2610.06824v2 [cs.AI] for this version)\nhttps://doi.org/10.48550/arXiv.2610.06824\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Oliver Jaffe [ view email ] [v1]\nMon, 5 Oct 2026 17:57:15 UTC (1,084 KB)\n[v2]\nTue, 6 Oct 2026 07:00:00 UTC (1,085 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts, by Oliver Jaffe and 1 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nAdditional Features\nAudio Summary\nCurrent browse context:\ncs.AI\n< prev\n|\nnext >\nnew\n|\nrecent\n| 2026-10\nChange to browse by:\ncs\nReferences & Citations\nSciX\nGoogle Scholar\nSemantic Scholar\nexport BibTeX citation\nLoading...\nBibTeX formatted citation\n&times;\nloading...\nData provided by:\nBookmark\nBibliographic Tools\nBibliographic and Citation Tools\nBibliographic Explorer Toggle\nBibliographic Explorer ( What is the Explorer? )\nConnected Papers Toggle\nConnected Papers ( What is Connected Papers? )\nLitmaps Toggle\nLitmaps ( What is Litmaps? )\nscite.ai Toggle\nscite Smart Citations ( What are Smart Citations? )\nCode, Data, Media\nCode, Data and Media Associated with this Article\nalphaXiv Toggle\nalphaXiv ( What is alphaXiv? )\nLinks to Code Toggle\nCatalyzeX Code Finder for Papers ( What is CatalyzeX? )\nDagsHub Toggle\nDagsHub ( What i","chars":4000,"total_chars":5425,"offset":0,"truncated":true,"next_offset":4000}