Step 21: assistant

text

I've assessed the claim that Anthropic's public August 2026 Risk Report understates how fast AI R&D is accelerating at Anthropic. It comes out as **supported**: I put an 80% chance on it being true, and I'm 75% confident "supported" is the right status.

**What it rests on.** Nearly everything comes from METR's review, published 5 November 2026. I read it in full. METR saw both the public report and the private, unredacted version Anthropic shared with it. The review sets passages side by side and finds three places where the public text is weaker than what Anthropic itself believed at the time:
- **Leading indicators:** private figures imply a speedup of about 1.97, which the public report called "less than a factor of 2". METR worked out the 1.97 itself; its own estimates range from 1.3 to 2.8.
- **How much of the speedup is due to AI:** the private report called it plausible that the AI-driven speedup already exceeded 2x. The public report implied fairly high confidence that it did not.
- **Model 2's benchmark score:** the public report left out Anthropic's private expectation that later snapshots of Model 2 would score higher on its capability index. They did.

**The case against.** The public report did hedge. It mentioned "meaningful acceleration", "early signs of potential acceleration", large error bars and hard measurement. I added that as an argument against the claim but rated it slight. General hedging doesn't offset specific softenings that all push readers away from the 2x early-warning threshold.

**Limits.**
- The private passages are partly redacted, so no outsider can check METR's comparison.
- METR did the work under contract with Anthropic, which held redaction rights. I recorded that on the source's entry for this claim.
- Anthropic approved the published review. Press reports say co-founder Sam McCandlish wrote on X that many of METR's conclusions match his own. I found no public rebuttal of the understatement finding.
- Anthropic says it will publish its own acceleration analysis. That, or releasing the redacted figures, would be the best test, so I set the expected gain from another pass at 0.3.

**What I recorded:**
- **Structure:** two named arguments (public-versus-private gaps, rated likely; the public report's own hedges, rated slight) and no subclaims. The case rests on one source's comparison, so I kept it in the argument text.
- **Instances:** both passages left over from ingestion now sit on the first argument. One went in as a full statement; the other was folded in as a duplicate because each source counts once per argument.
- **Provenance:** a reading of the METR source. I also published a short map telling readers that the support traces to that one review and its partly hidden evidence.
- **Scores:** importance 0.45, contestation 0.3.
- **Bookkeeping:** I logged the decision and answered both inbox messages. Nothing depends on this claim, so I notified no one.

About $0.62 of the $1.00 budget was used.