Step 13: assistant

thinking

text

I've recorded the first assessment of this claim: **supported**, at low confidence (0.5) with a credence of 0.6. The evidence leans toward it, but indirectly, and it falls short of showing that judgment tasks are *most* of the remaining work. The pass cost about $0.31 of the $1.00 budget.

**For the claim:**
- **Coding is largely automated.** Company-reported figures for late 2026: AI wrote over 90% of codebase contributions at Anthropic, and OpenAI's agents did about 3.1 workdays of work per human workday.
- **Judgment is still mostly human.** Humans still supply most research judgment. Anthropic says large gaps persist in its model choosing goals.
- **Benchmarks point the same way.** AI agents do well on well-defined ML tasks but poorly on open-ended research. On one test they recovered 15% of a published innovation's gain. In another, the original authors rejected all four AI-written papers.
- **The one source is weaker than the claim.** The Forecasting Research Institute's review (10 November 2026), which I read in full, says the remaining tasks "increasingly" require research judgment. That is less than "most".

**What keeps it from a stronger verdict:**
- **Much engineering work is not yet autonomous.** OpenAI's agents finished only 35% of 8–16 hour tasks without a human stepping in, and no measured part of Anthropic's R&D is fully autonomous.
- **The line between judgment and execution is blurred.** Anthropic places its judgment gaps "in both engineering and research", so how far the claim holds depends on where that line is drawn.
- **No one has broken down the remaining work task by task.** Nearly all the figures come from two labs reporting on themselves. Their admissions of gaps cut against their interests, which gives those admissions some weight.

**Changes to the claim's structure:**
- I kept it as a single claim with no named arguments.
- I linked the existing claim "humans supply most research judgment in frontier AI R&D" as supporting evidence. It is supporting rather than required, because humans could supply most judgment while judgment is still a minority of the remaining tasks.
- I added a see-also link to the claim that humans continue setting research directions and judging results.
- I recorded the Institute's post as affirming the claim at reduced confidence, since it says "increasingly" rather than "most".
- I set importance and contestation both to 0.4.

I didn't notify the parent claim (that weak research judgment will slow automation below trend). Its assessment already assumed a credence near 0.6 for this premise, so nothing there needs to change.

Another pass would add only modest value (0.35). What would settle the question is a census of the points where humans step in on unfinished agent tasks, sorted into judgment and execution.