Step 13: assistant

thinking

text

I assessed the claim as **inconclusive**, with a credence of 0.45 that it is true, after a fairly brief search. I'm 0.65 confident that "inconclusive" is the right reading of the evidence. The pass spent about $0.37 of the $1.00 budget.

The claim comes from the 2018 Iterated Amplification paper by Christiano, Shlegeris and Amodei. It predicts that the big supervision savings they saw on toy tasks will also hold for realistic ones.

- **The direct evidence is weak for realistic tasks.** On five algorithmic tasks, the model learned from tens of thousands of decomposition examples, where ordinary supervised learning needed tens of millions of full solutions. But those tasks were chosen because they split cleanly into small steps, so they say little about messier work.
- **The paper's general argument is thin.** It says that breaking a task down, with help on the pieces, is strictly easier than solving it whole. That shows decomposition is easier to *do*. It doesn't show it needs *much* less supervision to learn, or that people can cheaply demonstrate good decompositions.
- **Evidence from realistic tasks points both ways.**
  - Ought's human-only experiments in 2019–2020 found real reading and argument-evaluation tasks hard to break into checkable steps. Participants often disagreed about whether a step was valid.
  - OpenAI's 2021 book-summarisation work showed decomposition can make human feedback workable on a long, realistic task. It didn't compare supervision against learning the task directly.
- **No study I found compares supervision counts on a realistic task.** Today most capability comes from pretraining, which changes what "learning directly" costs, so the original comparison is largely untested.

**What I added to the page:**
- **Three named arguments**, each with a written form:
  - For, from the toy-task experiments: rated likely, but only if realistic tasks decompose as cleanly.
  - For, from "decomposing is strictly easier than solving": rated slight.
  - Against, from Ought's findings that realistic tasks are hard to decompose: rated slight, since it measured human reasoning rather than ML supervision.
- **Sources:** I placed the paper's two supporting passages on the arguments they make. I also recorded Ought's 2020 write-up as a source for the argument against.
- **Importance and contestation:** I set importance to 0.35 and contestation to 0.4. It is a supporting point in the scalable-oversight debate, and pretraining-based methods have largely overtaken it. I created no subclaims, since none seemed worth a node of its own at that importance.
- **Housekeeping:** I logged the decision and answered both inbox messages.

I put the chance that another pass would change the verdict at 0.3. A wider search of the process-supervision and task-decomposition literature might move it a little. What would actually settle it is a controlled count of labels needed on a realistic task.