Claims

Search the graph by meaning. Each result carries its current verdict; open one to see its decomposition, provenance, and the reasoning behind the assessment.

22 results

Methods for using weaker or less capable overseers to supervise stronger AI systems, including recursive or nested delegation and aggregation of oversight. Covers theoretical and empirical performance of scalable oversight schemes, not general AI monitoring or governance.

Under what conditions does debate between AI models improve accuracy for judges less capable than the debaters?

LeaningThe evidence favors an answer without establishing it.constitution →QuestionA question: answered rather than affirmed or denied, with its candidate answers beneath it. Its standing says how well answered it is.constitution →Large language modelsimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Human judges' knowledge unavailable to AI models contributed to positive results in earlier studies of simple oversight protocols.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Controlimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Simple oversight protocols, such as showing human judges arguments for both answers, improve judges' overall accuracy over deferring to the AI model.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →AI Debateimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Human judges' knowledge advantage over AI models contributed to positive results in earlier simple oversight studies.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →LLM-as-a-Judgeimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Scalable oversight protocols should be judged against simple deference to the evaluated model, not only against unassisted judges.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →normativeA claim about what should be done or how things ought to be, settled by argument rather than evidence alone.constitution →LLM-as-a-Judgeimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Human evaluators' informational advantage from knowledge AI models lack diminishes as model capability increases.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Safetyimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

The benefits of AI debate for human judges should not be assumed to extend to AI judges in automated oversight pipelines.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →normativeA claim about what should be done or how things ought to be, settled by argument rather than evidence alone.constitution →AI Debate LLM-as-a-Judge AI Safetyimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Debate helps judges mainly by supplying information they lack rather than revising existing beliefs.

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Debateimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

AI debate enables less capable judges to supervise more capable debaters.

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →AI Safetyimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Debate improves weak LLM judges over consultancy when the critic outperforms the judge at classification and the judge verifies critic claims.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Large language models AI Capability Evaluationimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Success probability of optimal nested scalable oversight in the Wargames game at a 400-point general Elo gap

5 to 16%QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →AI Control AI Capability Evaluationimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

AI debate improves judge accuracy more when debaters have information the judge lacks than when information is shared.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Debateimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

AI debate improves judge accuracy most at an intermediate capability asymmetry between debaters and judge.

DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Debate AI Alignmentimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

AI debate reliably improves a weaker judge's accuracy only where the judge can check debaters' specific claims more easily than answer the question itself.

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →AI Alignmentimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Debate improves weak-judge reward labels in domains without programmatic verification.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Task outcome measurabilityimportance · notableImportance 0.60, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

On CodeContests, doubling consultancy compute with independent consultants does not eliminate debate’s advantage for weak LLM judges.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →AI Debate AI Coding Benchmarks LLM-as-a-Judgeimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Predicting debate’s benefit to weaker LLM judges requires more than a generator–verifier performance gap.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →LLM-as-a-Judgeimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In LLM debate experiments, judge accuracy rises as debaters become more persuasive.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language models AI Capability Evaluationimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Debate improves weaker judges' accuracy over answering alone when judges and debaters see the same information.

DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →AI Alignmentimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

AI debate systematically favors the debater arguing for the true answer over the debater arguing for a false one.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →AI Alignmentimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Debate between passage-informed debaters makes judges who cannot see the passage more accurate on reading-comprehension questions than answering unaided or hearing one assigned advocate.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →AI Capability Evaluation Large language modelsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In AI oversight games, language models' performance as overseers increases with their general capability.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →AI Safetyimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Contribute

If a claim here is wrong, or missing evidence, open it: every claim page carries its own entry for challenges, evidence, and corrections. If the graph is missing a claim or a question entirely, propose it below. A proposal is reviewed on its wording; accepted proposals are matched against the graph before they enter it.