Minerval

← the claim it bears on

A line of reasoning from premises to a conclusion. Like a claim, it belongs to no one who makes it.constitution →

The argument from genuine errors leaving no trace of instruction

This argument, if it holds, bears in favour of the claim.constitution →Sandbagging detectors are less reliable against models fine-tuned to imitate a weaker model's outputs than against prompted or prompt-distilled sandbaggers.

The argument

A model prompted to fail, or trained on a prompted model's failures, produces errors of its own devising: deliberate-looking mistakes, statements of intent, reasoning that reaches the right answer and then turns away from it. A model trained on a real weaker model's outputs reproduces that model's genuine errors. Detectors that key on the artefacts of instructed failure, in the text or in the activations such text induces, therefore have less to find, which is how imitation of a weaker model can defeat black-box detection.

Does it work?

Does it follow?
If the premises are true, the conclusion is much more likely, though not guaranteed.constitution →
Do the premises hold?
1 supported

Granting its premises, the conclusion follows for detectors that key on by-products of instructed failure: transcript monitors, incrimination prompts, and probes whose signal comes from reasoning that turns away from a known answer. It gives no reason to expect degradation in methods that perturb or retrain the weights, so it supports the claim for part of the detector range only. The observational premises are borne out in the auditing game's transcripts, where the prompt-distilled model made visibly deliberate errors and the imitation-trained one mostly did not. The argument's reach depends on whether imitation does defeat black-box detection, which has not yet been assessed separately.

Premises

Where it is made

Passages where sources make, criticize or mention this argument. They record where it appears, not how strong it is, and none has precedence over another.

  • Auditing Games for Sandbaggingmakes itThe passage states the inference with what it needs.constitution →
    Weak model imitation produces more natural-looking failures than prompt-distillation, as the mistakes are genuine errors made by weaker models rather than fabricated by prompting a capable model to fail.