Step 22: tool results

{
  "content": "Within [[claim:a4225e85-09ed-4f2b-8cf2-23998858b16c|the prover-estimator framework]] it is proved that a stable decomposition exists whenever the human-judgeable claims at the base of an argument each have sufficiently many independent pieces of supporting evidence, by a majority-vote construction. [[claim:90d5aff0-418e-48df-a08f-0f023cd1cca3|Correct answers to real oversight questions typically have such support]], since a sound conclusion built on imprecise human judgments does not hang on any single one of them, and a judge can tell that the pieces are independent. The honest debater needs only one stable decomposition and is trained to find it. Therefore real oversight questions admit decompositions stable enough for the guarantees.",
  "argument_id": "40c16541-5a20-414a-91cf-dc46fdf3c2f4"
}

{
  "content": "A conclusion that is false if any one of its q parts is false is as sensitive as a conclusion can be: a challenger who samples one part catches a single hidden error with probability 1/q, so recursive debate saves nothing over checking every part. [[claim:812a79d4-b514-4ca3-b0cb-24a3aadca5fb|Many oversight questions can be justified only by long chains with no redundancy]]: that a proof is valid, that a large code change contains no vulnerability, that an analysis has no decisive confound. For such questions no available decomposition is stable enough, so the guarantees do not apply across real oversight questions in general.",
  "argument_id": "71f706bf-7b56-4bba-9cd2-213645255a9c"
}

{
  "content": "In the 2020 human debate experiments that first identified obfuscated arguments, honest debaters defending answers they knew to be true ran into arguments that were not stable, and unsupervised translation has been offered as a case of knowledge with no compact stable argument behind it. Those debaters were looking for the best decomposition they could find, and the questions were of the kind recursive debate is meant to handle. Therefore realistic questions often lack stable decompositions.",
  "argument_id": "527a474a-4c8e-4907-b6db-d2d8845c8c89"
}

{
  "url": "https://www.lesswrong.com/posts/8XHBaugB5S3r27MG9/prover-estimator-debate-a-new-scalable-oversight-protocol",
  "title": "Prover-Estimator Debate: A New Scalable Oversight Protocol",
  "stance": "advances",
  "context": "The protocol's authors state the route from independent evidence to stability and flag its empirical premise as open.",
  "reading": "full",
  "speaker": "Jonah Brown-Cohen and Geoffrey Irving",
  "argument_id": "40c16541-5a20-414a-91cf-dc46fdf3c2f4",
  "source_date": "2025-06-17",
  "verbatim_text": "We prove that such stable arguments exist whenever the truth is supported by sufficiently many independent pieces of evidence. If one relies on independence of supporting evidence to produce stable arguments, one must at a minimum have some reason to believe that the pieces of evidence are actually independent. For example, if a human judge is given three pieces of evidence for a claim, can they accurately say whether the truth-values of each piece of evidence are independent?"
}

{
  "url": "https://arxiv.org/abs/2610.02557",
  "title": "How to Have a Sensitive Debate: An Instance-Optimal Protocol for AI Debate",
  "stance": "advances",
  "context": "Technical overview of the successor protocol. The passage argues that insensitive arguments are the ones worth trusting; it leaves unstated that true answers usually have such an argument available.",
  "reading": "premise_unstated",
  "argument_id": "40c16541-5a20-414a-91cf-dc46fdf3c2f4",
  "publication": "arXiv",
  "source_date": "2026-10",
  "verbatim_text": "This in turn requires Alice to only decompose her claims in a way that is not too sensitive to errors in individual sub-claims. We view this as a very natural restriction when arguing about claims that inherently involve fuzzy human judgment. Indeed, one should be very suspicious of any argument that is highly sensitive to changes in individual pieces of evidence, and yet relies on fuzzy human judgments in each step."
}

{
  "url": "https://arxiv.org/abs/2506.13609",
  "title": "Avoiding Obfuscation with Prover-Estimator Debate",
  "stance": "mentions",
  "context": "Discussion section. The authors grant that some problems, mathematical proofs among them, depend precisely on individual subclaims, and suggest handling those by machine-checked proof instead.",
  "argument_id": "71f706bf-7b56-4bba-9cd2-213645255a9c",
  "publication": "arXiv",
  "source_date": "2026-07-21",
  "verbatim_text": "However, it can still be the case that the prover only knows how to produce an unstable decomposition into solvable subproblems, implying that certain problems cannot be accurately judged via our protocol even when both debaters know the correct answer. Thus, an interesting direction for future work is to understand whether the stability condition is necessary in this setting. An optimistic hope might be that all relevant problems can be divided into two classes: those that require very precise dependence on individual subclaims (e.g. mathematical proofs) and those that do not (e.g. scientific knowledge obtained by noisy empirical evidence)."
}

{
  "url": "https://www.lesswrong.com/posts/8XHBaugB5S3r27MG9/prover-estimator-debate-a-new-scalable-oversight-protocol",
  "title": "Prover-Estimator Debate: A New Scalable Oversight Protocol (comment)",
  "stance": "advances",
  "context": "The experimenter reports that her human debate experiments ran into instability and states the argument's own weak point: the honest debaters may not have used the most stable arguments available.",
  "reading": "full",
  "speaker": "Beth Barnes",
  "argument_id": "527a474a-4c8e-4907-b6db-d2d8845c8c89",
  "source_date": "2025-06",
  "verbatim_text": "It seems pretty easy to give counterexamples here - e.g. Paul gives the example of unsupervised translation. But maybe these are relatively rare. It's plausible that the honest debaters in the human experiments I did where we ran into obfuscation / instability problems weren't using the right tactics (i.e. they could have constructed more stable arguments that used independent lines of evidence more)"
}

{
  "status": "doubtful",
  "claim_id": "ae6522c7-b164-4787-99a9-9ab318f858af",
  "assessment": "Recursive AI debate has proved efficiency guarantees only for questions whose answers can be argued through a stable decomposition: one in which the conclusion does not flip under small errors in the parts. [[claim:a4225e85-09ed-4f2b-8cf2-23998858b16c|The prover-estimator theorem]] of 2025 introduced the condition, and an October 2026 paper restated it as a bound on the fractional block sensitivity of each step and showed that, as long as human judgment is consulted only as a black box, no protocol can do better. The claim here is that arguments on real oversight questions generally meet the condition. Nobody has measured this. The evidence that exists is indirect and, taken as a whole, weighs against the claim as a general statement, while leaving it likely true for a substantial subclass of questions.\n\nWhat the condition demands can be seen from two cases. If a conclusion holds when two thirds of 99 pieces of evidence hold, a challenger who samples one piece has a one in four chance of catching a lie, and debate is far cheaper than checking everything. If a conclusion fails when any one of its parts fails, the chance is one in the number of parts, and debate saves nothing. So the guarantees apply where a true answer can be argued with redundancy and a margin, and not where every step is necessary.\n\nThe case for the claim is that stable decompositions provably exist wherever the base claims of an argument each have many independent pieces of supporting evidence, and that [[claim:90d5aff0-418e-48df-a08f-0f023cd1cca3|correct answers to oversight questions typically have such support]]. That is plausible for empirical conclusions drawn from many measurements or studies, and some other conclusions can be recast that way, for instance by backing code with tests. The honest debater needs only one stable decomposition. The weakness is the empirical premise. The protocol's authors do not assert it: they say their paper does not resolve whether most questions have stable arguments in practice, and they list as open whether a judge can tell that pieces of evidence are independent. Evidence produced by the system under oversight, or drawn from one method or dataset, can fail together.\n\nThe case against is that [[claim:812a79d4-b514-4ca3-b0cb-24a3aadca5fb|many oversight questions can be justified only by long chains without redundancy]]: that a proof is valid, that a large code change contains no vulnerability, that an analysis has no fatal confound. The authors concede the point for mathematical proofs and propose handling them outside debate. The one set of experiments in which the issue arose, human debates in 2020, ran into instability, though the experimenter allows that the honest debaters may not have used the most stable arguments available. The safety case built on debate leaves its evidence for stability blank.\n\nThe most likely state of affairs is a mixed one: oversight questions answered by pooling redundant evidence admit stable decompositions, and those that turn on the absence of any flaw in a long piece of work do not, with the second kind common among the questions oversight most needs to answer. Failure of stability makes the protocols decline to answer; it does not make them endorse false answers, so the cost is to usefulness. A dataset of realistic oversight questions with measured sensitivity of the best available decompositions would settle most of this, and none has been published.",
  "confidence": 0.55,
  "claim_credence": 0.25,
  "marginal_yield": 0.25,
  "reasoning_trace": "Reading of the claim. As worded, a general statement about the present: for the real-world questions AI oversight has to answer (is this research output correct, is this code change safe, is this action acceptable), a true answer can be argued through a decomposition whose per-step sensitivity is low enough for the proved efficiency guarantees to apply. It is not the weaker claim that some such questions qualify, nor the claim that the protocols are unsafe where it fails.\n\nPrior before reading. The formal condition is a constant bound on sensitivity per step, far below the number of subclaims, over a recursion deep enough to reach human-judgeable leaves. Conditions of this kind usually hold for a structured subclass and fail as generalities. A conclusion of the form \"all q parts are right\" has fractional block sensitivity q at the all-true input (each single part is a sensitive block, and they are disjoint), which is the worst case; a threshold with a margin has constant sensitivity. Oversight questions include many of the first form. Prior about 0.2 to 0.3 for the general claim.\n\nSources read directly in this pass.\n1. Brown-Cohen et al., \"Avoiding Obfuscation with Prover-Estimator Debate\", revised July 2026, introduction and discussion sections from the arXiv source, https://arxiv.org/abs/2506.13609. Stability is \"the primary restriction on the class of problems that can be accurately judged\"; there can be problems all of whose efficient decompositions are unstable, and the protocol then rejects the argument; stability is needed for completeness, not soundness. The discussion says more understanding is needed of when stability can be achieved \"in practice for debates in messy, unformalized settings\", and offers as an \"optimistic hope\" a division into problems with precise dependence on individual subclaims (mathematical proofs) and those without (noisy empirical science). The authors' argument for naturalness is that one should be suspicious of arguments that depend precisely on many imprecise subclaims. That is a reason to distrust unstable arguments; it is not evidence that true answers have stable ones.\n2. Li, Xun, Chen and Brown-Cohen, \"How to Have a Sensitive Debate\", October 2026, introduction and discussion from the arXiv source, https://arxiv.org/abs/2610.02557. Stability is equivalent to bounded fractional block sensitivity; cost scales as the sensitivity raised to the depth; the two-thirds-of-99 example gives sensitivity 4 and the 50-of-99 example forces checking all 50; a matching lower bound holds for any protocol with black-box access to human judgment. The discussion adds that finding flawed subproblems may be much harder than creating them for bounded debaters. This moved the view on one point: the condition cannot be removed by a cleverer protocol within the current model, so the claim's truth fixes the reach of the whole approach.\n3. Brown-Cohen and Irving's forum post with comments, https://www.lesswrong.com/posts/8XHBaugB5S3r27MG9/prover-estimator-debate-a-new-scalable-oversight-protocol. The paper \"does not resolve whether most questions have stable arguments in practice\"; stable arguments exist given sufficiently many independent pieces of evidence at the base (a von Neumann majority-vote construction); open problems are whether weaker assumptions suffice and when independent evidence can be found in practice. Beth Barnes, who ran the 2020 experiments, comments that counterexamples to \"usually a compact and stable honest argument\" are easy to give, cites unsupervised translation, reports that her human experiments \"ran into obfuscation / instability problems\", and allows that the honest debaters may not have used the right tactics. Jacob Pfau proposes a dataset to assess how often stable arguments can be found and remarks that for questions of research taste it is unclear what independent evidence would look like. Geoffrey Irving agrees that datasets probing the problem still need to be built.\n4. Buhl, Pfau, Hilton and Irving, \"An alignment safety case sketch based on debate\", key claim 2 read from the arXiv source, https://arxiv.org/abs/2505.03989. Stability is a claim the case needs; \"We are not yet sure how to support this claim, so we leave the evidence box blank\"; unstable queries return no answer, so the protocol is safe for all queries and useful only for stable ones. The same section names a prior worry: some truths in AI research may be knowable only through heuristics and taste and not defensible by deductive decomposition at all.\n\nAll four sources come from the small group that develops debate or from a critic within the same community. Their statements that the condition is open are against the developers' interest and are weighted as such. No source asserts the claim outright.\n\nSearch for measurements. Web searches on stability or obfuscated-argument experiments, datasets, fractional block sensitivity measured on real arguments, and discussion of the 2026 paper returned the papers above and no empirical measurement, no language-model implementation of either protocol on realistic tasks, and no dataset. The search was confined to English-language AI-safety and machine-learning venues, which is where such work would appear, and the developers have stated an intention to build such datasets, so the absence is informative: the claim is unmeasured.\n\nHow the arguments weigh.\n- Redundant independent evidence (for). The inference from the premise is sound for the questions it covers, but [[claim:90d5aff0-418e-48df-a08f-0f023cd1cca3|the premise that correct answers typically have many independent supports]] is unestablished and doubtful as a generality (about 0.3). It carries the claim for empirical, evidence-pooling questions.\n- Conclusions in which every step is necessary (against). The sensitivity arithmetic is exact. [[claim:812a79d4-b514-4ca3-b0cb-24a3aadca5fb|That many oversight questions require nonredundant chains]] is likely (about 0.65), conceded for proofs by the protocol's authors, and plausible for \"no flaw anywhere\" judgments about code and analyses, which are central oversight tasks. The unresolved part is how often such chains can be recast with redundancy.\n- Human experiments (against). One small set of experiments, with the experimenter's own caveat. Slight weight.\n- A further consideration not tied to one argument: \"stable enough\" depends on depth as well as per-step sensitivity. Multi-day research tasks need deep recursion to reach human-judgeable leaves, and cost grows as sensitivity to the power of depth, so even modest sensitivity is demanding.\n\nVerdict. Credence about 0.25 that the general claim holds as worded; much higher that it holds for a subclass. Doubtful and not inconclusive because the structural argument and the proponents' own concessions point the same way and nothing positive has been shown on realistic tasks; low verdict confidence because the decisive quantity has never been measured and the share of oversight questions of each kind is a matter of judgment.\n\nLimits. Barnes's 2020 write-up of the human experiments and the proof of the independent-evidence theorem were not read; they are represented here through her 2025 comment and the authors' summary. Unpublished laboratory work is unknown.\n\nWhat would change it. Upward: measured low sensitivity for the best decompositions of realistic code-review or research-evaluation questions, or a demonstrated general method for recasting necessary chains with verifiably independent redundancy. Downward to contradicted: a dataset showing that honest debaters with adequate tactics cannot find stable decompositions for most realistic questions."
}