Step 14: tool results

{
  "content": "Lines of code and agent-hours measure volume of output, not contribution to research progress. The decisive inputs to AI R&D are choosing goals, designing experiments and judging results, and labs themselves report that these remain largely human: Anthropic says no measured part of its AI R&D is fully autonomous and that large gaps persist in Claude's judgment in choosing goals, and AI agents fail open-ended research evaluations. If share is weighted by contribution to progress rather than volume, the AI share is smaller than output metrics suggest, and estimated overall acceleration is about 1.5x to 2x rather than the many-fold that a 90% share would imply.",
  "argument_id": "f69b6c3e-02da-4ab6-8476-611de086a79a"
}

{
  "status": "supported",
  "claim_id": "a3a7a95b-f3fb-4cf8-a605-cbe3bee834db",
  "assessment": "On the measures frontier labs publish, AI systems do a large and fast-growing share of the work of AI research and development. Anthropic reports that Claude wrote over 90% of contributions to its codebase in September 2026, and OpenAI reports that AI agents did about 3.1 workdays of work per human workday in August 2026, up from about 0.5 in May; OpenAI also reports rising rates of tasks completed without human intervention (57% of 2 to 4 hour tasks in July 2026, up from 28% in January). A November 2026 review by the Forecasting Research Institute summarizes this company data as showing a large and fast-growing AI share.\n\nTwo qualifications limit how far this goes. First, the figures are reported by the companies themselves, which have an interest in the narrative of automated research, and outsiders cannot audit them. Second, they count volume of output (code, agent-hours), not contribution to research progress. Labs report that research judgment, such as choosing goals and directions, remains mostly human; no measured part of Anthropic's AI R&D is fully autonomous, and AI agents still fail open-ended research evaluations. Estimates of overall acceleration (about 1.5x to 2x at Anthropic, per METR) are modest compared with what a 90% labour share would suggest. The claim therefore holds on a volume reading of \"share\" and is weaker on a reading weighted by contribution to progress. Independent, contribution-weighted measurement would settle the gap.",
  "confidence": 0.7,
  "claim_credence": 0.75,
  "marginal_yield": 0.4,
  "reasoning_trace": "Prior: by late 2026, with coding agents widely deployed, it is plausible that most code at frontier labs is AI-written, while autonomous research remains limited. The FRI post (Kučinskas and Rosenberg, 10 Nov 2026, https://forecastingresearch.substack.com/p/how-close-is-recursive-self-improvement) confirms both halves: company-reported metrics (Anthropic >90% codebase contributions, Sept 2026; OpenAI 3.1 AI workdays per human workday, Aug 2026, up from ~0.5 in May) support \"large and rapidly growing\"; the same post says humans still supply most research judgment and cites Anthropic's own statement of persistent gaps in goal choice, InnovationEval and CRUX 2 failures, and METR acceleration estimates of ~1.5x (later ~2x). The canonical wording says \"share of AI research and development\", which a reader would most naturally take as share of the work done; on that reading the evidence favours the claim. The credence (0.75) discounts for self-reporting by interested parties, the narrowness to two labs, and the alternative contribution-weighted reading on which the claim is less clearly true. Not verified because evidence is not independently auditable. The search in this pass was limited to the one recorded source; a further pass could look for independent measurements (e.g. METR uplift studies) and Google DeepMind data."
}