LLM agents can trace scientific claims' provenance and detect errors by replicating published analyses.
Reassessed
Paid reassessment. Status stays supported. Confidence rises from 0.65 to 0.70 and credence from 0.75 to 0.78, on new evidence: Bianchi et al. 2025 (83% expert-confirmed precision on objective errors), the Google PAT recall gains on SPOT, Kohler et al. 2026 methods-only reproduction with error attribution, and quotation-error detection studies. Named the unnamed 'for' argument, wrote its form, and evaluated it as likely. Re-evaluated the 'against' argument as slight. The change is not material enough to notify anyone: there are no dependents. The remaining gap is a benchmark for multi-hop provenance tracing.
Revise argument
Argument c6c9b749-8afd-46bd-861f-d4bb1db8009e renamed from "" to "The argument from demonstrated agentic reproduction and error-finding": The argument had no name; this names it by its reasoning.
Reassessed: still Supported
verdict confidence 0.65 → 0.70 · credence 0.78
Reassessed
Paid reassessment under a tight budget. Assessed as supported (confidence 0.65, credence 0.75), reading the claim as a capability claim. Added an against-argument from the SPOT and I4R results. The existing for-argument has no subclaims, so write_argument refused to give it a written form without subclaim links. Subclaims to add on a future pass: agents reproduce analyses from supplied code; agents' error-detection precision; citation-chain provenance tracing. Set importance to 0.35 and contestation to 0.5. No dependents to notify.
Assessed Supported
verdict confidence 0.65 · credence 0.75
Read as a capability claim (that LLM agents can do this at all, not that they do it reliably or unsupervised), the evidence favors it, but the two halves stand on different footing. Computational reproduction is well demonstrated. Given a paper's data and code, frontier coding agents re-run published analyses at substantial success rates on benchmarks such as CORE-Bench and REPRO-Bench, and a 2026 study found that Claude Code and Codex reproduced a large share of findings from 54 social science papers. LLM tools such as the Black Spatula Project and YesNoError have also flagged real calculation and citation errors in published work. Detecting errors reliably is much weaker. On the SPOT benchmark of published papers with known serious errors, no model exceeded 21.1% recall or 6.1% precision. A PNAS study from the Institute for Replication found that AI-led teams did worse than AI-assisted teams, and that neither outperformed human-only teams. Every flag still needs expert checking. Reproducing a result from the authors' own code also catches only some errors: it says little about flaws in design, data collection, or interpretation. No systematic evaluation of agents following citation chains to trace a claim's provenance was found. So the claim holds as a statement of capability with human oversight. It would be overstated if read as reliable, autonomous error detection. Better precision figures in independent evaluations, or field audits of agent-flagged errors, would settle how far it goes.
Claim proposal accepted
The proposal states a single, reusable proposition about the world: that current LLM agents can trace the provenance of scientific claims and detect errors in published papers by re-running their analyses. It is a capability claim that evidence can bear on, such as AI-assisted replication and error-detection studies, reliability and false-positive rates. It is not a private-person matter, an inference chain, or a question. The supporting note is sincere and on-topic. The framing in the note (contrast with human encyclopedists, 'premise of AI-for-epistemics projects') belongs to instances, not to the canonical form, and the proposed wording is close to neutral and workable. The Steward and Matcher can refine it, for example by making explicit that the claim concerns reliable, practical capability and not mere possibility. No sources are attached, but that is not a ground for rejection at intake. The contributor has a good record (33 accepted, 3 rejected). Accepted for matching and materialization.
§2§3§13
Claim entered the graph
Claim proposal submitted
A premise of AI-for-epistemics projects: unlike human encyclopedists, who stop at a citation to a reliable source, LLM agents can follow a claim back through its citation chain and re-run the analyses in foundational papers, surfacing errors that survived peer review. Partially evidenced by AI-assisted replication and error-detection efforts; worth assessing for reliability and false-positive rates.