Step 18: tool results

{
  "status": "supported",
  "claim_id": "3d8d4a7b-39ca-46f6-9d47-bf7a3240b8ac",
  "assessment": "The evidence favors this claim for one developer, Anthropic, but the causal half of it rests on judgment and on evidence the public cannot see.\n\nThe claim has two parts: that capability progress at some frontier developer has sped up by an amount that shows in measurements, and that the developer's use of AI in its own research is what sped it up. The first part is fairly well evidenced. METR, an outside evaluator given eleven weeks of access to Anthropic's internal data, estimated in November 2026 that [[claim:ec9c31d9-8ba5-4107-86f2-25ee089e89a1|Anthropic's benchmark index has grown at about 1.9 times its earlier rate since October 2025]] (95% interval 1.5 to 2.8, with robustness checks from 1.1 to 2.9). Anthropic's own system cards reported a similar figure, and an outside analyst reports recovering most of the trend break from public scores of released Claude models. Whether the figure is \"about twice\" is less certain than whether some real speed-up occurred, and this claim needs only the latter.\n\nThe second part is weaker. METR is confident that faster scaling of training compute does not explain the change, and guesses that perhaps 80% of the acceleration as of October 2026 is due to AI use. That is a subjective estimate drawing on staff self-reports and on evidence Anthropic redacted. METR also says non-AI factors probably played a large role in late 2025 and that Anthropic staff had no consensus on attribution, and whether [[claim:e4cbf66e-fd61-4ae6-8247-19b9b2fa87c9|AI use caused most of the acceleration since October 2025]] is close to even odds. This claim does not need \"most\": a substantial, detectable share is enough. Anthropic's August 2026 risk report attributed the acceleration up to mid-July to factors other than AI, while an essay from Anthropic's own institute asserts that AI is already accelerating its development. That essay's evidence (the share of code written by Claude, code merged per engineer up about eightfold, a staff poll with a median fourfold self-reported uplift) shows faster research work, not a measured faster rate of model improvement, and Amdahl-style bottlenecks in compute and human direction mean the two need not move together. Rival explanations remain open: catch-up from a below-frontier earlier rate, growth in experimental compute, and fitting to well-specified benchmark tasks.\n\nThe comparison cases cut against a general effect. Google DeepMind's October 2026 assessment found [[claim:eab21612-23b6-4d07-8fa2-49d80f34494f|no statistically significant acceleration of its own research]], though its point estimate was a 30% rise in slope over a short window, and [[claim:52d0a82b-d764-4b85-80b7-9d930729f50c|the frontier of released models on the Epoch Capabilities Index]] has held at about 14 points a year. Neither refutes a claim that needs only one developer, but both make a cause particular to Anthropic more plausible.\n\nOn balance it is more likely than not that AI use has produced a detectable share of a real acceleration at Anthropic. The claim would be firmer if the redacted attribution evidence were published or examined by a second party, if the acceleration persisted in released models, or if a second developer reported the same pattern. It would weaken if Anthropic's trend reverted toward the industry rate once it reached the frontier.",
  "confidence": 0.72,
  "claim_credence": 0.7,
  "marginal_yield": 0.45,
  "reasoning_trace": "This pass re-judged the claim after both Anthropic subclaims received their first assessments and an Anthropic Institute essay was added.\n\nSubclaims. The trend-break subclaim is supported at credence 0.65 for \"about twice\" (read as 1.6 to 2.4x); its Steward notes that credence in some real speed-up is higher. This parent needs only a measurable acceleration, so the earlier estimate of about 0.85 that a real acceleration of roughly 1.5x or more occurred still stands. The attribution subclaim is inconclusive at 0.5 for \"most\" on a cumulative reading from October 2025; its reasoning matches what this assessment already weighed (METR's ~80% guess, non-AI factors in late 2025, redaction, no staff consensus). The parent needs a material, detectable AI share, not a majority of the cumulative change, so the earlier conditional estimate of about 0.8 is lowered only slightly to about 0.78. Net: 0.85 × 0.78 ≈ 0.66, plus a small chance the claim holds at another developer, about 0.7. No change in status or credence.\n\nNew source. \"When AI builds itself\" (https://www.anthropic.com/institute/recursive-self-improvement), by Marina Favaro and Jack Clark of the Anthropic Institute, read in full. It asserts that AI is already accelerating AI development. Its internal evidence: more than 80% of merged code authored by Claude by May 2026; lines merged per engineer per day about 8x the 2024 level in Q2 2026, which it calls almost certainly an overstatement of productivity; a March 2026 poll of 130 research staff with a median 4x self-reported uplift, which it expects is too high; rising Claude-judged task success; a kernel-optimization test rising from ~3x to ~52x; and an experiment on next-step research choices. It also notes human code review becoming a bottleneck (Amdahl's law). None of this measures the rate of capability improvement, and the public trends it cites (time-horizon doubling every ~4 months) are industry-wide. The developer has an interest, recorded on the source. Weight: modest corroboration that AI use substantially speeds research work at Anthropic, which makes the causal mechanism more plausible, but it does not independently establish the capability acceleration or its attribution. It is placed as a new argument from internal productivity, rated slight-to-likely because productivity gains do not translate one for one into faster capability progress. It also sits in tension with Anthropic's August risk report attributing the earlier acceleration to non-AI factors, showing the developer's own statements differ by document and purpose.\n\nUnchanged from the previous pass: METR report read through the main sections; METR's review of the August risk report seen in summary; Tadepalli's public-data reproduction (~1.64x) seen in excerpts; Google DeepMind's non-detection via its author's thread; FRI/Epoch on the flat public frontier. The open request for a public-data refit of Anthropic's trend against the frontier remains the most valuable next step."
}