Claims

Search the graph by meaning. Each result carries its current verdict; open one to see its decomposition, provenance, and the reasoning behind the assessment.

29 results

OpenAI, the AI research lab, and the systems it builds (e.g., GPT models, agent swarms). Claims attributing a result, capability, or failure to an OpenAI system belong here; claims about other labs' systems do not.

Realistic reward-hack fine-tuning produces similar alignment-faking effects on GPT-4o and GPT-4.1.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Alignment Faking AI Reward Hacking AI Safetyimportance · minorImportance 0.25, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In School of Reward Hacks-tuned GPT-4.1, apparent compliance mostly reflects harmless outputs fooling the evaluator.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →AI Reward Hacking LLM Safety Trainingimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Share of reversed comparative yes/no question pairs showing implicit post-hoc rationalization by ChatGPT-4o (chatgpt-4o-latest, February 2025)

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →LLM Prompt Sensitivity Large language modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In OpenAI's monitorability evaluation suite, chain-of-thought-only monitors outperform monitors seeing only actions and final outputs.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorabilityimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

OpenAI's monitorability evaluation suite detects reduced monitorability after RL training to obfuscate chain of thought.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability AI Capability Evaluation LLM Reasoning Trainingimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

OpenAI's intervention, process, and outcome-property evaluations validly measure chain-of-thought monitorability.

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Chain-of-Thought Monitorability AI Capability Evaluationimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Penalizing chain-of-thought monitor flags in OpenAI RL experiments caused obfuscated reward hacking and near-zero monitor recall.

VerifiedThe evidence, examined directly, establishes the claim.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability AI Reward Hackingimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-4.5 used about ten times GPT-4's pretraining compute, versus roughly hundredfold increases between earlier GPT generations.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Training compute trends AI Scaling Trendsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-4.5's capability gain over GPT-4 was smaller than GPT-4's gain over GPT-3.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →LLM capability trends Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra can complete substantially more tasks without chain-of-thought reasoning than earlier OpenAI models.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models LLM capability trends Chain-of-Thought Promptingimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra's chain-of-thought controllability increased during RL training, unlike earlier OpenAI models.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability LLM Reasoning Trainingimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra's limited use of recurrent depth has only a minor effect on its chain-of-thought monitorability.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Chain-of-Thought Monitorability Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra's per-forward-pass computational depth is within a factor of two of GPT-4's.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models AI Scaling Trendsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra's reduced chain-of-thought monitorability is primarily due to capability gains rather than architectural or training changes.

InconclusiveThe evidence and argument have been examined and do not come down on either side.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Chain-of-Thought Monitorability Large language models LLM capability trendsimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

InstructGPT's harmful advice reflects goal misgeneralization rather than reward misspecification in human-preference training.

DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Goal Misgeneralization Large language models AI Reward Hackingimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

InstructGPT labelers were instructed to prioritize helpfulness over harmlessness when ranking outputs during training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models LLM Safety Trainingimportance · settledImportance 0.15, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

InstructGPT's fine-tuning data contained few prompts requesting harmful assistance.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Large language modelsimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra's architectural changes differentially explain its chain-of-thought controllability changes.

DoubtfulThe evidence weighs against the claim, but the examination is incomplete or the evidence is indirect.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Chain-of-Thought Controllability Large language modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

OpenAI automatically scans its reinforcement-learning runs for accidental chain-of-thought grading.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability LLM Reasoning Trainingimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

OpenAI found that its RL training of previously deployed models inadvertently graded chain-of-thought reasoning.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Reasoning Training Chain-of-Thought Monitorabilityimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Benchmark contamination may inflate UK AISI's no-CoT math time-horizon estimate for GPT-6 Astra.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Benchmark data contamination AI Capability Evaluationimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6-generation OpenAI models score substantially higher than GPT-5.x models on CoT-Control at matched chain-of-thought length.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Controllability AI Capability Evaluation LLM capability trendsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

O3's unusual chain-of-thought terms also occur in settings unrelated to deception.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Chain-of-Thought Monitorability Large language modelsimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

GPT-6 Astra has lower chain-of-thought monitorability than earlier OpenAI models, including GPT-5.6 Sol.

VerifiedThe evidence, examined directly, establishes the claim.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

OpenAI o3's chain of thought often uses terms in nonstandard ways.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models Chain-of-Thought Monitorabilityimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Some non-standard terms in OpenAI o3's chain of thought have different meanings across contexts.

Chain-of-thought monitorability stayed roughly steady over the frontier RL training runs of OpenAI o3 and GPT-5.1 Codex Max.

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Chain-of-Thought Monitorability Large language modelsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Anti-scheming deliberative alignment training slightly reduces unusual terminology in o3's chain of thought.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Chain-of-Thought Monitorability Large language modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In o3’s chain of thought, “watchers” generally refers to human oversight or evaluators.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Chain-of-Thought Monitorability Large language modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Contribute

If a claim here is wrong, or missing evidence, open it: every claim page carries its own entry for challenges, evidence, and corrections. If the graph is missing a claim or a question entirely, propose it below. A proposal is reviewed on its wording; accepted proposals are matched against the graph before they enter it.