argumentA line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from relearning-resistant methods

againstThis argument, if it holds, weighs against the claim.constitution →Current LLM unlearning methods leave unlearned pretraining knowledge largely recoverable by fine-tuning on other facts from the forget set.

The argument

The recovery experiments tested methods that edit an already trained model. A method that rebuilds the model without the knowledge would leave nothing latent to recover, and one is available: distilling an unlearned model into a randomly initialized student makes unlearning robust to relearning. Such methods count among current unlearning methods, so it is not true of current methods as a class that they leave unlearned knowledge largely recoverable.

Does it work?

Does it follow?: slight even if the premises holdEven if the premises are true, they make the conclusion only a little more likely.constitution →
Do the premises hold?: 1 supported

Granting its premises, the argument defeats a strict universal reading but moves the claim only a little as a generalization about methods in use: one costly exception, with compute approaching retraining, does not change what the commonly applied methods do. It rests on the reported robustness of distillation-based unlearning, which comes mainly from one group and has not been independently tested with held-out-fact recovery on pretrained knowledge. Its weight would grow if such methods became standard practice or if their robustness were confirmed at scale.

Premises

SupportedThe evidence favors the claim, but the examination is incomplete or the evidence is indirect.constitution →Distilling an unlearned language model into a randomly initialized student makes unlearning robust to fine-tuning-based relearning.

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

No source is recorded as making it.