Step 11: assistant

thinking

text

I rated the claim **supported**: I put the probability that it's true at 0.95, and my confidence that "supported" is the right label at 0.85.

**What I relied on.** I read METR's November 5, 2026 review of the automated-R&D section of Anthropic's August 2026 Risk Report. I didn't read the Risk Report itself; I relied on the passages METR quotes from it.
- **The claim's basis:** it matches the "Full Substitution Threshold" in Anthropic's scaling policy. That threshold is a model able to replace the entire set of research scientists and research engineers at no more than five times the cost.
- **Anthropic's case:** it has strong reasons to swap staff for models wherever it can, yet its researchers were still needed.
- **METR's view:** METR had access to Anthropic's non-public data. It agreed the bar is high and that Anthropic would likely have noticed if its July 15 models could clear it.

**Main reservation.** METR noted that, without strong benchmark evidence, practical obstacles might have held back automation the models were close to managing. Those obstacles are problems with tooling, oversight, alignment, or getting the models to perform at full ability. That caveat matters most for whether capability was close. It matters little for full replacement of the whole research staff, which would be hard to miss. The evidence is mostly Anthropic's own observation, partly unpublished, checked by one outside reviewer. The real dispute in METR's review is about a different threshold, whether AI progress had sped up twofold.

**Structure.** I left the claim as a single proposition with no subclaims. I added one argument for it, "the argument from observed non-substitution under strong incentives," and rated it "likely," not decisive. I set importance to 0.45 and contestation to 0.2, and answered both inbox messages.

**One gap.** Three passages from the METR review were waiting to be filed under arguments. The system accepts only one passage per source on an argument, so the first was recorded and the other two were dismissed as duplicates. One of the two dismissed was METR's criticism, so the page doesn't show it as a recorded criticism of the argument. Its substance is covered in the assessment and the argument's evaluation.