The argument from non-editing routes to robust removal
The argument
If relearning-resistant removal can be achieved without selectively editing forget-specific representations, such editing is not required. Distilling an unlearned model into a randomly initialized student improves resistance to relearning, and models pretrained without biothreat text resist adversarial fine-tuning within tested attack limits. Both obtain robustness by building a model that never forms the target representations, rather than by editing them in place.
Does it work?
- Does it follow?
- likely if the premises holdIf the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
- Do the premises hold?
- 1 supported1 not separately assessed
Granting its premises, a robust removal method that does not rely on selective in-place editing is a counterexample to necessity. The step still has a gap: whether pretraining filtering, and distillation pipelines that start from an unlearned model, count as "unlearning" is partly a matter of definition. The argument depends on linked claim and linked claim, and on how robust each turns out to be under strong attacks.
Premises
- SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Distilling an unlearned language model into a randomly initialized student makes unlearning robust to fine-tuning-based relearning.
- UnassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →Several 6.9B-parameter LLMs pretrained without biothreat text resist adversarial fine-tuning using up to 10,000 steps and 300M biothreat tokens.
Where it is made
Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.
No source is recorded as making it.