Review of the "Risks from automated R&D" section in Anthropic's Risk Report (August 2026)
Author not recordedMETR BlogPublished Nov 5, 2026Original︎ ↗Archived copy
Anthropic's public August 2026 Risk Report understates the acceleration of AI research and development at Anthropic.. What this source says is evidence, and each claim’s Steward weighs it knowing the stake.
Stewards’ notes · 1
- Anthropic's public August 2026 Risk Report understates the acceleration of AI research and development at Anthropic.
Evidence not shown. The review sets passages of the public report beside passages of the private, unredacted version Anthropic shared, and shows three places where the public text is more confident than the private one: a leading-indicator speedup the private figures put at about 1.97 is described publicly as less than a factor of 2; the private report calls it plausible that AI-attributable speedup exceeds 2x while the public one implies more confidence; and the public report omits Anthropic's private expectation that later Model 2 snapshots would score higher on its capability index, which they did. The private quotations are partly redacted and the 1.97 figure is the reviewers' own calculation, so readers cannot check these numbers; Anthropic approved the published text and its co-founder said many of the conclusions match his own. Nov 14, 2026
Examinations
None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.
Text116 passages · 7 marked
Overall, we agree with Anthropic that by July 15th, internally deployed models posed low risk of directly automating R&D in key domains or causing acute catastrophic risks from acceleration of capabilities progress. However, the report’s claim that they had not crossed their early warning thresholds relies on contestable judgment calls that aren’t communicated clearly in the public report. While both METR’s researchers on this project and Anthropic’s relevant staff lean toward it not having been crossed, we think it is plausible that it had been, and we believe the public report overall understates the level of acceleration that was occurring.
Evidence not shown. The review sets passages of the public report beside passages of the private, unredacted version Anthropic shared, and shows three places where the public text is more confident than the private one: a leading-indicator speedup the private figures put at about 1.97 is described publicly as less than a factor of 2; the private report calls it plausible that AI-attributable speedup exceeds 2x while the public one implies more confidence; and the public report omits Anthropic's private expectation that later Model 2 snapshots would score higher on its capability index, which they did. The private quotations are partly redacted and the 1.97 figure is the reviewers' own calculation, so readers cannot check these numbers; Anthropic approved the published text and its co-founder said many of the conclusions match his own.
The Risk Report argues the 2x Acceleration Threshold has not been crossed as capability metrics and leading indicators show less than a 2x acceleration, with factors other than AI driving that acceleration.
Not yet read by this claim’s Steward.
We find the Risk Report’s argument that the Full Substitution Threshold had not been crossed broadly compelling.
Not yet read by this claim’s Steward.
Similar to Anthropic, our confidence in this claim is much lower than it was in the past due to the lack of high-confidence unsaturated benchmarks demonstrating capability limitations that seem critical for automating AI R&D. The lack of such benchmark evidence makes us more worried that alignment, tooling, oversight or elicitation issues are holding back automation despite the models having obtained or being very close to obtaining the required capabilities.
Not yet read by this claim’s Steward.
METR’s own analysis of these leading indicators, which was redacted from this report, finds a range of reasonable estimates, between 1.3 and 2.8. We have substantial methodological uncertainty about the correct way to measure these values, which we discussed in the full (unredacted) version of the holistic report. As discussed in that report, Anthropic redacted this discussion.
Not yet read by this claim’s Steward.
Based on interviews with research leads and informal surveys, we would guess that the overall AI capabilities progress is still slightly less than twice as fast as it would be without AI assistance. We have enough uncertainty it seems plausible we do exceed this factor of 2 overall.”
Not yet read by this claim’s Steward.
“Given current AECI data, we cannot confidently distinguish between a one-time jump in capabilities and a continued acceleration. However, given the acceleration in the leading indicators mentioned above, a sustained acceleration seems more parsimonious.
Not yet read by this claim’s Steward.
Not found in the text 4
We also do not believe models at the time could fully automate AI R&D or R&D in other key domains.
Not yet read by this claim’s Steward.
Overall, we agree with Anthropic that by July 15th, internally deployed models posed low risk of directly automating R&D in key domains or causing acute catastrophic risks from acceleration of capabilities progress.
Similar to Anthropic, our confidence in this claim is much lower than it was in the past due to the lack of high-confidence unsaturated benchmarks demonstrating capability limitations that seem critical for automating AI R&D.
Not yet read by this claim’s Steward.
we expect AI’s augmentation of R&D to continue to grow, and we cannot rule out a substantial further acceleration in the pace of capabilities progress over the next 6 months. This could rapidly lead to “high” risks in these areas.
Not yet read by this claim’s Steward.
Lineage across 11 claims
Record
- Kind
- report
- Authors
- Not recorded
- Publisher
- METR Blog
- Published
- Nov 5, 2026
- DOI
- None
- Link
- https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review/
- Archived copy
- https://web.archive.org/web/20261108013244/https://metr.org/blog/2026-11-05-rd-section-anthropic-risk-report-august-2026-review/
- Stored text
- 22,963 characters, retrieved Nov 11, 2026 · 58 sections, 116 passages
- Claims extracted
- For
What the review finds about Anthropic systems' ability to automate AI research and development, and the limits of the evidence it saw
· 10 claims (8 new) · Nov 13, 2026 for Whether Anthropic or OpenAI will have AI able to carry out all frontier AI research work except agenda-setting by the end of 2027 - For
What the review finds about Anthropic systems' ability to automate AI research and development, and the limits of the evidence it saw
· 8 claims (2 new) · Nov 13, 2026 for Whether Anthropic or OpenAI will have AI able to carry out all frontier AI research work except agenda-setting by the end of 2027
- For
- Versions
- One known
- Copies
- None recorded
- Corrections
- None recorded
- Facts checked
- Nov 11, 2026
History 4
- Nov 14, 2026ReadingSteward of “Anthropic's public August 2026 Risk Report unde…” · evidence not shown, read whole
- Nov 11, 2026DividedSegmenter · Divided into 174 parts by its own structure
- Nov 11, 2026FactsFacts fetcher · Facts looked up
- Nov 11, 2026Entered the graphEntered the graph; copy stored