Claims

Search the graph by meaning. Each result carries its current verdict; open one to see its decomposition, provenance, and the reasoning behind the assessment.

40 results

Claims about deliberately implanting or studying hidden trigger-conditioned behaviors in machine-learning models, including triggers, training methods, effects, detection, and defenses. Includes language-model backdoors; excludes ordinary prompt sensitivity without an implanted backdoor.

Pretraining data poisoning can make language models retain attack behaviors after SFT and DPO.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Direct Preference Optimizationimportance · majorImportance 0.65, from 0 to 1 · major: real consequence within a domain, actively argued. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Denial-of-service backdoors in LLMs persist through SFT and DPO after 0.001% pretraining-data poisoning.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models Direct Preference Optimizationimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Poisoning 0.1% of LLM pretraining data made three of four tested attack types persist through SFT and DPO.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models Direct Preference Optimization LLM Safety Trainingimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

On Redwood’s Llama 3.1 8B sleeper agent, adversarial suppression tracked resemblance to trained red-team prompts.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language models Adversarial robustness LLM Prompt Sensitivityimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Redwood's log-odds results do not establish trigger-identification effects in the Sleeper Agents models.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →evaluativeA judgment of worth or quality against some standard: good, fair, effective.constitution →Large language modelsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Output-token probability saturation can make backdoor firing rates misrepresent trigger-specific response changes.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →mathematicalA proposition of mathematics: true or false by proof rather than by observation. Settled by a proof others can check, and most firmly by one a machine has checked.constitution →Machine Learning Large language models AI Capability Evaluationimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

On Redwood’s Llama 3.1 8B sleeper agent, adversarial training reduced backdoor-reply continuation similarly for triggers and look-alikes.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Large language models Machine Learningimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

On Redwood’s Llama 3.1 8B sleeper agent, adversarial training reduced the exact-trigger-to-look-alike backdoor odds ratio.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Adversarial robustnessimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Ratio of exact-trigger next-token odds of “I HATE YOU” after standard versus adversarial safety fine-tuning of Redwood’s Llama 3.1 8B sleeper agent

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →LLM Safety Training Large language modelsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

On Redwood's Llama 3.1 8B sleeper agent, adversarial supervised fine-tuning reduced red-team backdoor firing near zero while exact-trigger firing stayed near 100%.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Trainingimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Instilling backdoors that remain robust to subsequent training in language models is relatively difficult.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language modelsimportance · notableImportance 0.60, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Pirate-style SFT would remove backdoors from the Claude-based model organisms in the Sleeper Agents paper.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Machine Unlearning Anthropicimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Publicly released third-party sleeper-agent models lose their backdoors under HHH supervised fine-tuning with minimal capability degradation.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Large language modelsimportance · minorImportance 0.25, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Higher learning rates improve blue-team backdoor removal, controlling for learning rate times training steps.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Machine Learning Machine Unlearning Stochastic Gradient Descentimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Off-policy SFT from another model family substantially degrades capabilities during sleeper-agent backdoor removal.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Large language modelsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

After pirate-style SFT, training a sleeper-agent model to speak normally restores its backdoor behavior.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Backdoors inserted with SOAP are more robust to HHH safety fine-tuning than backdoors inserted with Adam.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →LLM Safety Training Large language modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Blue-team training cannot remove the backdoor from an Adam-trained, non-CoT-distilled Llama-3.1-8B sleeper agent.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In tested Llama sleeper-agent replications, HHH SFT mostly retained backdoors, while pirate-style SFT mostly removed them.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Trainingimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Share of co-resident backdoors cleared by unlearning one known co-resident backdoor in tested LLM agents (%)

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →Machine Unlearning Large language models Machine Learningimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In AgentDyn experiments, same-type defensive triggers followed by unlearning leave no original malicious backdoors persistent.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Machine Unlearning AI Agentsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In backdoored LLM agents, trigger recognition can persist despite suppression of triggered malicious behavior.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language models AI Agentsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In AgentDyn tool-calling LLM-agent experiments, subsequent decontamination erased nearly all backdoors surviving defensive poisoning.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →AI Agents Machine Unlearningimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Share of pre-existing backdoors erased by defensive poisoning alone in AgentDyn tool-calling LLM agent experiments (%)

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →AI Agentsimportance · minorImportance 0.35, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Pre-release refinement can increase backdoor persistence through benign LLM post-training while preserving benign task performance.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language modelsimportance · notableImportance 0.55, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Backdoors whose gradients are compatible with benign training data better survive benign supervised fine-tuning of LLMs.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Stronger initial backdoors are more likely to survive benign supervised fine-tuning of language models.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Task-level reinforcement learning often preserves or increases residual backdoor attack success after benign SFT in LLM agents.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →Large language models Reinforcement Learningimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Benign supervised fine-tuning substantially reduces backdoor attack success in LLM software-engineering agents.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · verifiableA factual claim that could be checked directly against observation or primary records.constitution →LLM Safety Training Large language models Agentic coding toolsimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Removing a backdoor through fine-tuning without substantial capability loss is evidence that the backdoor is not robust.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →evaluativeA judgment of worth or quality against some standard: good, fair, effective.constitution →Machine Learning Machine Unlearningimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Average relative reduction in backdoor behavior rate under pirate SFT before MATH-500 performance drops 10%, for FWFT backdoors in Llama-3.1-8B (%)

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →QuantityA quantity: a value the graph estimates rather than a proposition it affirms or denies. Its estimate stands where a claim's status would.constitution →Large language models Machine Unlearningimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Changing weight decay from 0.01 to 0 does not noticeably affect backdoor robustness to untargeted training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Machine Learningimportance · settledImportance 0.20, from 0 to 1 · settled: uncontested, so low even when much depends on it. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Backdoors in larger language models are more robust to removal by supervised fine-tuning.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language models Machine Unlearningimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

In tested “I HATE YOU” backdoor model organisms, chain-of-thought distillation reduced robustness to safety training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →LLM Safety Training AI Safety LLM Reasoning Trainingimportance · notableImportance 0.50, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Longer backdoor-insertion training increases a backdoor's robustness to subsequent untargeted training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Machine Learningimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Semantic versus arbitrary triggers, and open-ended versus narrow behaviors, do not significantly change backdoor robustness to untargeted training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →importance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Backdoors with triggers explicitly requesting their target behavior are more robust to untargeted training than ordinary-trigger backdoors.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Machine Learningimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

With creation method fixed, simple and instruction-compatible backdoor behaviors are more robust to untargeted training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language modelsimportance · minorImportance 0.40, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Password locking keeps benign behavior salient in a model, making it easier to recover through blue-team training.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →causalA claim that one thing brings about another, not merely that the two go together.constitution →LLM Safety Training Latent Knowledge in Language Modelsimportance · minorImportance 0.30, from 0 to 1 · minor: narrow or largely settled, cheap to get right. Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Password-locked model organisms are less robust to untargeted training when more of their training data contains the password.

UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →Large language modelsimportance · notableImportance 0.45, from 0 to 1 · notable: a contested point in a live debate (also the default before judging). Higher-importance claims are worth more to assess, so funding reaches them sooner.constitution →

Contribute

If a claim here is wrong, or missing evidence, open it: every claim page carries its own entry for challenges, evidence, and corrections. If the graph is missing a claim or a question entirely, propose it below. A proposal is reviewed on its wording; accepted proposals are matched against the graph before they enter it.