Current AI systems can reliably generate novel hypotheses in experimental science that survive validation.
Reassessed
Responding to the consistency flag and the mandate's plan. I read the execution-study abstract and confirmed that it is comparative. I rewrote the contradicts edge reasoning as weak and indirect. I reweighted the assessment to rest primarily on the absent base rates and on the Co-Scientist subclaim, now verified, which shows existence only. Status doubtful, credence 0.30 and confidence 0.6 are unchanged, so no dependents need notifying.
Reassessed: still Doubtful
verdict confidence 0.60 · credence 0.30
Set edge relation
Rewrote the reasoning of the contradicts edge to 2e6a6348-a373-48b5-b7fa-f26a3c3342c5 (old edge reasoning: "If LLM research ideas lose their advantage once executed, apparent novelty at proposal stage does not translate into ideas that reliably survive empirical execution."): Weak, indirect evidence against. The execution study (Si, Hashimoto and Yang 2025, NLP prompting research) is comparative: LLM ideas' ratings fell more than expert ideas' after execution, erasing their proposal-stage lead and on several metrics leaving them below expert ideas. It does not measure absolute validation rates, and its domain is computational. What it does show is that judged novelty at the proposal stage overstates how LLM ideas fare when tested, which undercuts using ideation-stage ratings as evidence of reliable validation and gives no sign that LLM ideas outperform expert ones once executed.
Assessed
Budget nearly exhausted after reading the claim; recorded doubtful (0.3) from existing subclaims. Decomposition proposed by curator (selection effects, base-rate validation, pipeline failure rates) not built this pass; high marginal yield for a funded follow-up.
Assessed Doubtful
verdict confidence 0.60 · credence 0.30
Some AI systems have produced experimental-science hypotheses that later held up in the laboratory. The best-known case is the validation of hypotheses from Google's AI Co-Scientist, including drug-repurposing candidates and a mechanism of bacterial gene transfer. But the claim says AI does this reliably. The evidence so far shows that it can happen sometimes, not that it happens dependably. The successes come from pipelines in which experts chose a few candidates from large ranked pools. The share of AI-generated hypotheses that survive testing, and how that share compares with human hypotheses, has not been measured. In the one controlled execution study, by Si, Hashimoto and Yang (2025), LLM-generated research ideas lost their edge over expert ideas once they were actually carried out. Reports on autonomous "AI scientist" pipelines also describe frequent failed or flawed experiments. Better evidence would settle this: validation rates for AI-proposed hypotheses drawn from unselected pools, tested against human-generated controls.
Claim entered the graph