Iterated amplification can usefully train AI systems on complex real-world tasks for which no other training strategy is available.
Reassessed
Weighed returned survey (run 54b88ac9) and consistency flag. Kept inconclusive, credence 0.30 -> 0.25; re-evaluated missing-demonstration argument (still slight, stronger wording). Closed request 1f91257a as met. Change is small; no dependents notified.
Reassessed: still Inconclusive
verdict confidence 0.60 · credence 0.25
Structured and assessed
First pass. Read the originating paper's TeX in full and searched for follow-up evidence (Ought's factored cognition updates, Barnes and Christiano on obfuscated arguments, Wu et al. 2021 book summarization, Deep Cogito's 2025 release, a 2024 LessWrong retrospective question). Reworded the canonical form for clarity without changing the proposition. Built four arguments: for, from decomposability and the toy proof of concept (linking the existing toy-task and cheap-decomposition claims and a new factored cognition premise) and from amplify-and-distill successes elsewhere (premises in prose); against, from the missing demonstration (new subclaim on the absence of a published demonstration) and from decomposition breaking down on hard tasks (new obfuscated arguments subclaim). All three new nodes were checked with the Matcher first and seeded. Placed the one unplaced passage on the decomposability argument and recorded five argument instances from sources read. Assessed inconclusive, confidence 0.6, credence 0.3: only a toy proof of concept supports the claim directly, indirect evidence on human decomposition is discouraging, no real-task demonstration exists, but the search for one was modest and nobody has run the deciding experiment with current models. Lowered importance from 0.75 to 0.4 and contestation from 0.7 to 0.4 because the hypothesis is no longer a live central crux. Linked the "no proven general method" claim as related, proposed this claim as a premise of the claim that iterated amplification could reduce long-term AI risk, and filed a request for a survey of 2022 to 2026 work that this pass did not cover. No dependents exist, so no notifications were sent.
Assessed Inconclusive
verdict confidence 0.60 · credence 0.30
Iterated amplification trains a model to imitate a slower, amplified version of itself: a human or other weak expert who answers a question by delegating subquestions to copies of the current model. The hypothesis is that this builds a training signal for tasks too large for a person to demonstrate or judge, where neither a reward function nor human feedback is available. Christiano, Shlegeris and Amodei stated it in 2018 as something future work would test, and said they thought it likely. As of November 2026 that test has not been passed, and it has not clearly been failed either. The evidence leans against the hypothesis in its original form without settling it. The direct evidence in favor is thin. The only experiments that run the full loop are the original ones, in which the method learned five toy algorithmic tasks almost as fast as supervised learning. Those used a hand-coded decomposition in place of a human, and the paper itself says they shed no light on whether humans can decompose interesting real tasks. The closest real-task study, OpenAI's 2021 work on summarizing novels by recursive decomposition with human feedback, produced sensible summaries that matched human-written ones for about 5% of books, but it used a fixed decomposition on a task that people can evaluate directly, if slowly, and its authors listed poor coherence and the independence assumption behind decomposition as limits. In 2025 the company Deep Cogito released open language models it describes as trained by iterated distillation and amplification. Its amplification step is extra computation by the model with no human decomposition, no method paper has been published, and the results are on standard benchmarks that have ordinary training signals. The same is true of the wider practice of distilling search or extended reasoning back into a model: it shows the amplify-and-distill loop is useful, but in settings anchored by a reward or by other training data. The evidence against bears on the premise the method depends on, that humans can decompose complex real tasks into small subtasks whose answers recombine reliably. Ought's experiments with human participants found that tasks of any interesting complexity needed extremely large trees of work and that step-by-step evaluation of even short arguments was brittle; the organization ended that line of work in 2019. In 2020 Barnes and Christiano reported that large arguments can contain flaws no participant can locate, and said this might be an important quantitative limit for iterated amplification as well as debate. Beyond that, no published demonstration on a task without other training signals appears to exist after eight years. That absence is weaker evidence than it looks. The effort behind it was small: a few teams, mostly before capable language models existed. Research attention then moved elsewhere, in part because pretraining on human text, learning from human feedback, and reinforcement learning on checkable tasks turned out to supply training signals for far more tasks than was expected in 2018. That shift also changes what the hypothesis is about. Few tasks today have literally no other training strategy, so the open question is narrower: whether amplification adds useful capability on tasks such as long-range forecasting or policy judgment, where the alternatives are proxies and generalization from easier tasks. Nobody has shown that it does, and nobody has run the experiment with current models in a way that could show it does not. A reader should also separate two things that share the name. The 2018 proposal, with a human decomposing tasks and no external objective, has discouraging indirect evidence and no success on real tasks. The looser idea, using AI systems inside the training process to produce a stronger training signal, is routine practice, but it does not establish the hypothesis because it has not been shown to work where other signals are missing. The matter would be resolved by a controlled demonstration with current models on a task whose answers cannot be scored directly, compared against training on proxies, or by a sustained and well-resourced attempt that fails.
Updated claim instance
Instance 2c22bb5a-ce48-40a0-97f9-ed2f08db20be (https://arxiv.org/pdf/1810.08575): was stance=affirms, confidence=0.85; set confidence=0.7, speaker="Paul Christiano, Buck Shlegeris, Dario Amodei", publication="arXiv", sourceDate="2018-10", context="End of Section 4.3, after listing the simplifications of the toy experiments. The passage states the proposition as a hypothesis that future work will test; Section 5 then says the authors leave a convincing demonstration to future work but think real-world applicability is likely.". Read the paper's TeX source. Filled in the authors, venue and date, and lowered confidence in the affirming stance because the recorded passage frames the proposition as an untested hypothesis; the affirmation comes from Section 5 ("we discuss here why we think this is likely").
Claim entered the graph