Step 6: tool results

{
  "status": "supported",
  "claim_id": "142793f4-48e0-427e-9662-866a34fa2dcb",
  "assessment": "The claim joins two propositions: that a large share of frontier AI progress comes from iterative experimentation rather than rare conceptual breakthroughs, and that this experimentation is automatable, with its pace set mainly by tools and resources (compute, infrastructure, engineering throughput).\n\nThe first half is reasonably well supported. Analyses of AI progress attribute most measured capability gains to scaling of training compute and data, with algorithmic improvements as the other major contributor, and several studies of algorithmic progress find that much of it is scale-dependent and discovered through large numbers of empirical runs rather than through a few insights. Practitioners widely describe frontier work as a loop of scaling, observing failures, fixing them and retrying, and access to experimental compute is commonly reported as the binding constraint inside labs.\n\nThe second half is less established. That experiment throughput is bounded by compute is plausible, but whether the choice of which experiments to run (research taste and direction-setting) is itself automatable, and how much it contributes, is the live dispute. Evidence of AI systems autonomously carrying out frontier research remains limited and comes largely from developers with an interest in it, including the AI company report that states the claim. \"Much\" is also vague, so the claim is easier to support than to test. On balance the evidence favors the claim in its moderate reading, while the strong reading, that research progress is mostly a function of tools and resources and direction-setting matters little, remains unsettled.",
  "confidence": 0.55,
  "claim_credence": 0.6,
  "marginal_yield": 0.6,
  "reasoning_trace": "Prior model: if the claim were true, one would expect (a) progress to track compute closely, (b) algorithmic gains to be scale-dependent and found by many empirical runs, (c) labs to report compute for experiments as the bottleneck, (d) AI agents to measurably accelerate research engineering. (a)-(c) hold in the public record (scaling-law literature; Epoch AI estimates of compute growth and algorithmic efficiency gains; work arguing much algorithmic progress is scale-dependent; practitioner accounts). (d) is only partly shown: agent benchmarks on ML engineering tasks show rising capability, but evidence that agents set fruitful research directions is thin, and economists modelling an automated-research feedback loop note that complementarity between compute and research ideas could cap acceleration.\n\nThe sole recorded instance is Anthropic's essay \"When AI builds itself\" (https://www.anthropic.com/institute/recursive-self-improvement), which asserts the claim in reply to the objection that human direction-setting matters most. Anthropic has a commercial interest in its models being seen as capable research agents; the essay states the claim rather than presenting new evidence for it. The verdict therefore rests on the broader literature, assessed here from general knowledge in a budget-limited pass; no subclaims have yet been mapped. A fuller pass should separate \"progress is mainly experiment-driven and compute-bound\" from \"the experimentation is automatable\", and examine quantitative estimates of the share of progress attributable to compute versus ideas."
}

{
  "claim_id": "142793f4-48e0-427e-9662-866a34fa2dcb",
  "reasoning": "A key premise in debates on recursive self-improvement and AI timelines; actively argued (compute bottleneck vs research taste), but narrower than the central AI-risk claims.",
  "importance": 0.65,
  "contestation": 0.65
}

{
  "name": "The argument from incremental, experiment-driven progress",
  "stance": "for",
  "claim_id": "142793f4-48e0-427e-9662-866a34fa2dcb",
  "description": "Breakthroughs are rare; most progress is iterative scaling-and-fixing, which AI agents can perform."
}