Step 10: tool results

read_page

{
  "url": "https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/",
  "max_chars": 12000
}

{"ok":true,"url":"https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/","text":"Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 - METR\nOur Work\nResearch\nNotes\nUpdates\nRisk Assessment\nAbout\nDonate\nCareers\nSearch\n-->\nOur Work\nResearch\nNotes\nUpdates\nRisk Assessment\nAbout\nDonate\nCareers\nMenu\n×\nIndependent Assessment of AI R&D Acceleration at Anthropic in October 2026\nCONTRIBUTORS\nTom Cunningham ,\nNate Rush ,\nParker Whitfill ,\nand\nHjalmar Wijk\nDATE\nNovember 5, 2026\nSHARE\nCopy Link\nCitation\nBibTeX Citation\n×\n@misc { metr-2026-anthropic-ai-rd-acceleration-assessment ,\ntitle = {Independent Assessment of AI R&D Acceleration at Anthropic in October 2026} ,\nauthor = {METR} ,\nhowpublished = {\\url{https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/}} ,\nyear = {2026} ,\nmonth = {11} ,\n}\nCopy\nSubscribe\nRedaction summary: As part of this assessment, Anthropic had rights to redact any non-public information in this post. Per our approach to transparency , we had the right to disclose if Anthropic exercised redaction rights, and whether or not any of our findings depended significantly on redacted information. While we were able to freely share our highest-level conclusions about risk, Anthropic redacted a majority of the evidence we found most informative for these conclusions. Anthropic described the redactions as “aimed at removing confidential information that could accelerate other AI developers.” 1 Some of these redactions are themselves challenging to describe clearly without revealing sensitive information we have been asked to redact. We share a table of important evidence we were and were not able to discuss given the redactions in our process section below. All redactions that were important for our conclusions are covered by our descriptions in that table or other in-line descriptions in the report below.\nMuch of the redacted evidence we viewed had multiple plausible interpretations, and both METR and Anthropic researchers have significant uncertainty about how to synthesize the evidence into a holistic picture. This makes it hard for the public to judge this evidence themselves, reconcile it with evidence from other sources, and come to their own conclusions (i.e. this report does not achieve ‘transparency about evidence’ ). As a consequence, the public has to rely on METR’s judgment much more than would be ideal, given the weight of the questions answered here.\nExecutive summary\nFrom August 12 to October 29, 2026, METR staff performed a high-access evaluation at Anthropic to assess the degree to which AI is accelerating AI R&D. 2 If AI systems started to significantly accelerate further AI development, this could lead to a sudden emergence of extreme and risky capabilities .\nAnthropic offered unprecedented access to internal data on their AI R&D process, including a variety of metrics on pretraining, reinforcement learning (RL), training compute in each domain, benchmark scores for internal models, AI speedup analyses, forecasts of a future model’s metrics, and interviews with relevant staff.\nIn our first section, we share our subjective conclusions about risk . We now believe that:\nAs of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. This estimate is given with high uncertainty; we would not be surprised if the true acceleration was closer to 1.3x or 3x the historical rate.\nA majority of this acceleration today is likely attributable to Anthropic’s extensive use of AI to do R&D, though there is significant uncertainty and other causes likely played a large role in acceleration in late 2025.\nThe lack of observed bottlenecks or strong diminishing returns means there is a significant risk that further capabilities progress could lead to even faster acceleration, though, as we say below, we may be missing important bottlenecks.\nIn our second section we share some of the key empirical findings from our Full Report :\nSince October 2025, benchmark scores (measured via AECI, Anthropic’s version of the Epoch Capabilities Index) for Anthropic’s models have been growing at an estimated 1.9x the pre-October 2025 rate [95% confidence interval: 1.5–2.8x], implying that Anthropic has made roughly two years of historical progress over the last year.\nWe are confident that the acceleration was not due to an acceleration in the scaling of training compute.\nInterviews and surveys are consistent with AI usage causing a doubling in the pace of capabilities progress. Anthropic staff self-report dramatic uplift from use of AI systems in interviews and surveys, although these are challenging to interpret. We were told that R&D is accelerated primarily via AI assistance rather than fully autonomous contributions, but there are some signs that fully autonomous contributions will play a larger role soon.\nWe did not see evidence for significant bottlenecks which would prevent further capabilities progress due to limits on algorithmic progress, training compute, experiment compute, or data in the next 12 months. However, our understanding of the status of different bottlenecks changed several times as we received more detailed information, so we are not confident in our current assessment.\nSome of the key findings that most informed our conclusions about risk were redacted by Anthropic.\nIn our third section, we describe the assessment process :\nThe scope, setup, and timeline of this assessment.\nDetails of the information shared with METR.\nDetails of what information and evidence we were not able to share given Anthropic’s redactions.\nFinally, we describe the biggest limitations of our analysis:\nWe could be underestimating capability acceleration if we did not assign enough importance to (1) evidence that some growth rates have further increased recently; (2) further automation of AI R&D tasks leading to a further acceleration in progress; (3) evidence that AECI growth has sometimes understated growth in real-world impact.\nWe could be overestimating capability acceleration if we did not assign enough importance to (1) non-AI factors in Anthropic’s research progress; (2) AECI’s focus on verifiable tasks; (3) reasons why Anthropic’s early growth rate (2023–2025) was unusually low.\nThe remainder of the report goes into detailed discussion of each of the key components. We include appendices with supplemental material.\nOur subjective conclusions on risk\nOur conclusions on risk.\nIndependently of this assessment, METR published an October 2026 article proposing a set of high-level questions about AI R&D acceleration we believe the public needs to understand to be able to judge loss-of-control risks from frontier AI development. In this section, we describe how the evidence we reviewed in this assessment changed our subjective conclusions on these high-level questions.\nPace of capabilities progress.\nA significant acceleration of Anthropic’s overall rate of progress would be surprising, since diminishing returns and dependence on compute scaling mean this would likely require remarkable scaling in other inputs. 3 After reviewing the internal evidence, we still have significant uncertainty and internal disagreements on the team, but most METR staff who worked on this project now believe that, as of October 2026, Anthropic is likely improving the capabilities of their models on well-specified tasks at around twice the rate they did between March 2024 and October 2025. We would not be surprised if the current rate of growth was closer to 1.3x or 3x the historical rate. We have additional uncertainty in how this acceleration translates to broader and less well-specified tasks.\nThe 1.9x acceleration [95% confidence interval: 1.5–2.8x] in the AECI trend since October 2025 described below significantly informed this view, but could be explained by overfitting to the benchmark distribution or otherwise be an artifact of the AECI methodology. On the other hand, we are not confident that AECI would reliably let us estimate recent R&D acceleration, since the data points are lagging acceleration in research and are somewhat sparse.\nBeyond the AECI evidence, we also saw redacted evidence that significantly informed our view on this question, and made us take this apparent ≈2x acceleration more seriously. It also gave us some indication that acceleration could be above 2x and increasing over time, though we would be surprised if it was significantly higher.\nAttribution to AI.\nWe looked at a variety of factors that could explain the acceleration we observed, including compute scaling, hiring, data improvements, algorithmic discoveries, and resolving other bottlenecks.\nAs we describe below, we were able to confidently determine that the acceleration was not due to an acceleration in the rate of growth of training compute. Beyond that, there was not a consensus among Anthropic’s own staff and we did not find decisive evidence. Our overall subjective estimate is that:\nThe acceleration in late 2025 was likely influenced by non-AI factors. We investigated a variety of factors which could have contributed to the observed acceleration in late 2025, but details of many of these factors have been redacted.\nWe guess that a majority (perhaps ≈80%) of the current (October 2026) acceleration is due to AI. This judgment is given with substantial uncertainty, but is based on a variety of lines of evidence on contributions from AI and other causes. We believe AI’s contribution has grown relatively more important over time, and do not think the other factors we investigated would explain the sustained acceleration. The evidence of speedup from the usage of AI we describe below also seems compatible with a large contribution.\nWe illustrate these points below:\nOur more detailed estimates for the cumulative and instantaneous effects of AI on Anthropic’s rate of progress in AECI, and at various stages of the model training pipeline, were redacted.\nBottlenecks to further acceleration.\nA key concern is that acceleration from the use of AI could start to compound, as the development of more capable models leads to further acceleration. As we describe below, we did not identify any clear bottlenecks (limited inputs that would prevent further growth) or steep diminishing returns to further scaling that would get in the way of such compounding acceleration, though our understanding evolved several times over the course of this assessment, and we have significant uncertainty. Because of this, we believe there is a significant risk that further capabilities progress could lead to even faster acceleration over the coming year.\nKey findings\nIn this section, we summarize the key empirical findings from our Full Report that we can describe without revealing redacted information (see the process section for more details on what was redacted).\nTerminology: We describe growth as annual multiples and acceleration as a ratio of growth rates.\nWe use “acceleration” to refer to an increase in the rate of growth, such that a 2x acceleration means that 2 years of historical progress is made in 1 year.\nIf AECI’s historical trend was 15 points/year, then increasing to 30 points/year implies an acceleration rate of 2 (it takes 1 year to achieve 2 years’ progress at the old rate). For an exponentially growing variable, the acceleration rate is the ratio of growth rates. If a variable is growing 3x/year, then to make 2 years of progress in 1 year the growth rate must be (3 2 )x/yr = 9x/yr (and acceleration can be written as ln(9)/ln(3) = 2).\nSince October 2025, benchmark scores for internal models have been growing at around twice their historical rate.\nBenchmark scores of Anthropic’s models, as measured by AECI (Anthropic’s version of the Epoch Capabilities Index ), have accelerated 1.9x [95% confidence interval: 1.5–2.8x] since October 2025. Put another way, Anthropic appears to have made as much progress on benchmarks in the past year as they did between October 2023","chars":12000,"total_chars":78371,"offset":0,"truncated":true,"next_offset":12000}