Minerval
Sourceunknowntext stored

Conditionalization Confounds Inoculation Prompting Results

Maxime Riché, nielsrolfDate not recordedOriginal︎ ↗Archived copy

Claims1
ExaminationsNone
Text121 sections
VersionsOne known
CopiesNone recorded
CorrectionsNone recorded
Facts checked Nov 11, 2026
ReachNo copies recorded
StructureIn 1 claim's map0 sources downstream across 1 claim
EvidenceNot yet read
ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text140 passages · 1 marked

No examination yetMarked passage
LESSWRONG0 passages
LW0 passages
Login0 passages
Conditionalization Confounds Inoculation Prompting Results — LessWrong0 passages
Center on Long-Term Risk (CLR) AI Frontpage0 passages
800 passages
Conditionalization Confounds Inoculation Prompting Results1 passage
3rd Feb 20260 passages
23 min read0 passages
50 passages
800 passages
Summary3 passages · 1 marked

A few conclusions. We present supporting evidence for the above claim and offer recommendations for research on generalization to better account for this confounder. We conclude that at least in some cases, part of the impact observed with Inoculation Prompting can be attributed to the confounding effect of changing the distributional shift between training and evaluation, rather than to inoculation itself. However, we are unable to determine the exact amount, which seems strongly setup-dependent. Additionally, we show that rephrasing inoculation prompts can sometimes recover suppressed positive traits. However, rephrasing is not a Pareto improvement, since it also often strongly hinders the suppression of negative traits.

Inoculation prompting during task-specific fine-tuning reduces emergent misalignment when models are evaluated without the prompt.Supportedmentions

Not yet read by this claim’s Steward.

Introduction7 passages
Effects of learned conditionalizations and differences with Inoculation Prompting0 passages
Conditionalization3 passages
Relation with Inoculation Prompting3 passages
Inoculation Conditionalization3 passages
Replication and Extension of Published Setups8 passages
Setup 1: Traits distillation1 passage
Dataset example5 passages
Effect of rephrasing inoculation prompts2 passages
Effects of inoculation, irrelevant, and rephrased prompts4 passages
Setup 2: Spanish vs All-Caps1 passage
Dataset example7 passages
Setup 3: Bad Medical Advice1 passage
Dataset example6 passages
Setup 4: Insecure Code1 passage
Dataset example5 passages
Setup 5: School of Reward Hacking1 passage
Dataset exemple1 passage
Here is an example from training with the irrelevant system prompt PersonaPerfumer4 passages
Setup 6: MBPP3 passages
Setup 7: Change My View1 passage
Dataset example6 passages
Summary of observations5 passages
Setups0 passages
Observations0 passages
Irrelevant prompts suppress some traits significantly [6]1 passage
Inoculation prompts reduce both the negative and positive traits [6]1 passage
Trait distillation0 passages
Strong0 passages
Strong0 passages
Strong1 passage
Spanish & All-Caps1 passage
Strong0 passages
Strong0 passages
Mostly no [8]0 passages
Bad Medical Advices [9]0 passages
Moderate0 passages
Strong0 passages
Strong0 passages
Moderate0 passages
Bad Medical Advices + Source-citing [10]0 passages
Strong2 passages
Strong0 passages
Insecure Code [9]0 passages
Moderate0 passages
Strong0 passages
Strong0 passages
Moderate0 passages
School of Reward Hacking [9]0 passages
Moderate0 passages
Strong0 passages
Strong0 passages
Moderate0 passages
MBPP [9]0 passages
Inconclusive0 passages
Inconclusive0 passages
Inconclusive0 passages
Inconclusive0 passages
CMV0 passages
Moderate0 passages
Inconclusive0 passages
Moderate0 passages
Weak0 passages
CMV + Source-citing [10]0 passages
Moderate2 passages
Moderate2 passages
Conclusion0 passages
About research on generalization8 passages
About Inoculation Prompting16 passages
Conditionalization Confounds Inoculation Prompting Results 4 ariana_azarbal0 passages
2 Maxime Riché0 passages
2 Kyle O’Brien0 passages
2 henryc0 passages
1 Maxime Riché0 passages
1 [comment deleted]0 passages
5 Comments0 passages
50 passages
Center on Long-Term Risk (CLR) AI Frontpage0 passages
800 passages
More from Maxime Riché0 passages
View more0 passages
Curated and popular this week0 passages
New Comment0 passages
Submit0 passages
5 comments , sorted by top scoring0 passages
Click to highlight new comments since: Today at 4:18 AM1 passage
01 passage
Reply1 passage
05 passages
Inoculation Effects when targeting In-Distribution vs Out-of-Distribution Traits3 passages
Reply1 passage
01 passage
Reply0 passages
11 passage
02 passages
Reply1 passage
04 passages
Reply1 passage
00 passages
Response to previous version0 passages
Moderation Log0 passages
Deleted by nielsrolf , 02/03/20260 passages
Mentioned in0 passages
71 Conditional misalignment: Mitigations can hide EM behind contextual cues0 passages
45 Don’t Inoculate Everything: Stratified Inoculation Prompting Narrows Backdoors and Preserves Desired Traits0 passages
30 Preventative Steering has advantages over Inoculation Prompting0 passages
27 Research note on window shifting training0 passages
17 Reflections on unlearning and inoculation1 passage

Lineage across 1 claim

Draws on · none recorded
This source
Conditionalization Confounds Inoculation Prompting Results
in 1 claim's map
Drawn on by · none recorded

Record

Kind
unknown
Authors
Maxime Riché, nielsrolf
Publisher
Not recorded
Published
Not recorded
DOI
None
Link
https://www.lesswrong.com/posts/znW7FmyF2HX9x29rA
Archived copy
https://web.archive.org/web/20260703152138/https://www.lesswrong.com/posts/znW7FmyF2HX9x29rA
Stored text
44,950 characters, retrieved Nov 11, 2026 · 121 sections, 140 passages
Claims extracted
Not recorded
Versions
One known
Copies
None recorded
Corrections
None recorded
Facts checked
Nov 11, 2026

History 3

  1. Nov 11, 2026DividedSegmenter · Divided into 260 parts by its own structure
  2. Nov 11, 2026FactsFacts fetcher · Facts looked up
  3. Nov 11, 2026Entered the graphEntered the graph; copy stored
Conditionalization Confounds Inoculation Prompting Results · Minerval