Step 9: assistant

text

I've assessed the claim as **supported**, with a credence of about 0.72 that it's true and a confidence of 0.65 that "supported" is the right reading. The main evidence is METR's November 2026 assessment of AI R&D at Anthropic.

**What the 2026 evidence shows so far**
- **Organisation-level measurement:** METR's report (5 November 2026) found that Anthropic's models have improved about 1.9 times faster since October 2025 than before (95% interval 1.5 to 2.8). This was measured on Anthropic's own version of the Epoch Capabilities Index, a benchmark composite.
- **Cause:** METR ruled out faster growth in training compute as the cause. It judged that a majority of the speedup is likely due to Anthropic's use of AI in its R&D.
- **Partial rather than full automation:** METR was told the speedup comes mainly from AI assisting staff, not from AI working autonomously. That is direct evidence of a substantial speedup before full automation.
- **Bottlenecks:** METR saw no major bottleneck likely to block further progress in the next 12 months. It says it isn't confident in that view, because its picture changed several times as more detail came in.

**What limits the evidence**
- It comes from one company.
- Anthropic redacted most of the evidence METR found most informative, so the public has to take METR's judgment on trust.
- The benchmark measure may overstate progress on less well-specified tasks, or reflect quirks of the benchmark itself.
- How much of the speedup is due to AI rather than other factors is uncertain.
- Controlled studies of individual developers point the other way. METR's 2025 trial found experienced open-source developers about 19% slower with AI tools. Its February 2026 update said a follow-up study was unreliable because of selection effects, and it changed the study design.

The individual-developer studies measure something different from a whole organisation's rate of progress; the organisation-level evidence bears more directly on this claim.

**Bearing on a strong feedback effect before full automation**
The METR result supports the idea that partial automation can already produce large gains. Because it rests on one lab, largely hidden evidence and a benchmark measure, it does not settle the question. Replication at other labs and published, unredacted data would move the verdict most.

**What I did on the claim's page**
- Recorded the assessment, set importance to 0.7 and contestation to 0.65, and logged the decision.
- Recorded the METR report as a source affirming the claim.
- Answered both inbox messages.

**Not done yet (marginal yield 0.5)**
To keep within the $0.75 budget, I didn't break the claim into subclaims or add named arguments. The pieces to separate out next are:
- whether the speedup is real and caused by AI;
- whether it holds beyond one lab and beyond well-specified tasks;
- whether bottlenecks will emerge.

One passage from Forethought's essay, which argues that human researchers would focus on the tasks AI can't yet do, is still unplaced.