Step 22: tool results

{
  "status": "supported",
  "claim_id": "0655bddb-4381-4431-bc7e-c04ae510a5d6",
  "assessment": "The claim contrasts two readings of Anthropic's capability data since late 2025: a single step up in capability (around the Claude Mythos Preview model) followed by a return to the old growth rate, or a lasting increase in the growth rate itself. Anthropic's August 2026 Risk Report described weak evidence for the one-time jump. METR's November 2026 independent assessment, made with extensive non-public access, concluded the opposite: newer internal models, including later snapshots of an unreleased model, kept pulling further above the old trend, so that [[claim:ec9c31d9-8ba5-4107-86f2-25ee089e89a1|benchmark scores have grown roughly 1.9 times the pre-October 2025 rate]] (95% interval 1.5 to 2.8), and a slope-change fit outpredicts a level-shift fit. [[claim:d4cfffee-9259-4d52-92b1-3b8979884218|Internal leading indicators]] and redacted pipeline metrics also point to elevated, possibly increasing, growth.\n\nThe evidence favours sustained acceleration but does not establish it. It rests almost entirely on one assessment whose most informative evidence was redacted, and on an index built from few frontier models, the newest of which are near saturation on most benchmarks. When non-Anthropic models are included the estimated acceleration falls to about 1.4x, leaving open that part of the change was a one-off catch-up from a subfrontier position. METR itself said the August-era data could not distinguish the hypotheses and that further AECI measurements would. A continuation of above-trend gains in models released over the next several months would strengthen the claim; a return to the pre-2025 slope would undercut it.",
  "confidence": 0.65,
  "claim_credence": 0.68,
  "marginal_yield": 0.3,
  "reasoning_trace": "Sources read: METR, \"Independent Assessment of AI R&D Acceleration at Anthropic in October 2026\" (https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/) and METR's review of the Risks from automated R&D section of Anthropic's August 2026 Risk Report (https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review/).\n\nFor: AECI acceleration of 1.9x [1.5-2.8] since October 2025, robust across regularisation and saturation variants (1.9-2.1x), with robustness range 1.1x-2.9x; METR states \"New internal models make a permanent slope change look more plausible than a one-time jump at Mythos Preview\" and the review notes later Model 2 snapshots improved on AECI by about as much as Mythos 5 over Mythos Preview, contradicting the reading in the public Risk Report. Leading indicators show accelerations between 1.3 and 2.8. Redacted internal metrics reportedly indicate acceleration possibly above 2x and increasing. Interviews (11 coded) mildly support continuing acceleration.\n\nAgainst / limits: Anthropic's August report read the data as weak evidence for a one-time jump; METR concedes August-era AECI data were \"only mildly in favor of a jump\" and could not rule out either. Including all frontier developers yields ~1.4x, with the historical trend dominated by other labs' models, compatible with catch-up. Model 4 exceeds 50% on all 15 AECI benchmarks, so the index is near saturation and sparse; overfitting to benchmark distributions is possible. Most decisive evidence is redacted and unverifiable by the public; METR worked with data largely supplied through two Anthropic staff, though it judged deliberate misleading extremely unlikely. Anthropic has a stake in how its progress is described (in both directions: risk thresholds vs. capability reputation).\n\nWeighing: a sustained slope change is the better-supported reading given multiple roughly independent internal indicators, but evidence is single-assessor, partly hidden, and depends on a short post-break window. Credence about 0.68."
}