Step 9: assistant

text

I assessed the claim as **supported**, with 70% credence that it is true and 0.6 confidence that "supported" is the right status. It is a description of where AI development is now and over the next few years, not a settled fact about where it ends up.

**What holds up.** The first half, that AI development is becoming substantially automated, rests mostly on public evidence. METR's measurements of how long a task models can complete on their own show that length doubling every few months. Coding and research-reproduction benchmarks such as SWE-bench and CORE-Bench have saturated.

**Where the evidence is weaker.** The second half, that humans still set research directions and judge results, rests mainly on Anthropic's own report, "When AI builds itself." It says that by May 2026 more than 80% of code merged into its codebase was written by Claude. It also says Claude now matches skilled humans at running well-specified experiments but still falls well short at choosing goals. That admission cuts against the interest of a company selling capable models, which gives it some weight. But Anthropic has a commercial stake in how its models are seen, and nobody outside the company has examined its internal data. The report itself says its 8x code-per-engineer figure overstates the real productivity gain.

**What stays open.** The question is how long humans keep the direction-setting role. Some forecasters expect models to match top researchers at picking problems within a few years; others expect automation of AI research to stall.

**Changes to the graph:**
- **No subclaims or arguments.** I added no subclaims or named arguments. I only linked two related claims: one that partial automation speeds AI progress before full automation, and one that progress toward automating AI research is likely to stall.
- **Importance and contestation.** I lowered the importance from 0.75 to 0.6 and the contestation from 0.65 to 0.55.
- **Reading of the source.** I recorded a reading of the Anthropic report noting that its internal data are unpublished and the company has a stake in the result.
- **Inbox.** I logged the decision and answered both inbox messages.

I set the expected value of another pass at 0.4. Independent measurement of how much of AI research work is automated, or evidence that models now choose research directions themselves, would most change this verdict. The run used $0.63 of its $1.00 budget.