Step 7: assistant

thinking

text

I've recorded the claim as **supported**, with credence 0.8 and confidence 0.7. The claim is that no published work shows iterated amplification training an AI system on a complex real-world task that has no other training signal. The search was short and turned up no counterexample, so the absence counts only moderately as evidence.

**What the search found:**
- **The original 2018 paper** (Christiano, Shlegeris and Amodei) tested the method on five toy algorithmic tasks with known solutions. An algorithm stood in for the human.
- **OpenAI's 2021 book summarization** is the closest real-world case. But the way the task was split up was fixed by hand, humans gave feedback at every level, and the whole task could still be judged by humans directly, just slowly. So it doesn't meet the claim's bar.
- **Ought's factored-cognition studies** used human participants, not trained models.
- **Deep Cogito's 2025 models** are described by the company as trained with "iterated distillation and amplification". No detailed method has been published, and the models are tested on standard benchmarks that have ordinary training signals.

**What could weaken it:** the doubt that remains is about little-noticed or recent work, and about how strictly "iterated amplification" and "lacks other training signals" are read. Anyone who counts the book-summarization work as amplification would see the claim as weaker.

**Other things I recorded:**
- The claim has no sub-claims or named arguments. It is a claim that something is missing from the literature, and it doesn't break down naturally.
- Importance is 0.3 and contestation 0.2.
- I logged the decision and answered both inbox messages.

**Gaps:** I recorded no source passages on the claim page. The OpenAI blog post refused access (error 403), and I stopped early because the per-turn cost was high against the budget. I spent $0.22 of the $1.20.