Sourceblog posttext stored

White Box Control at UK AISI - Update on Sandbagging Investigations

Joseph Bloom, Jordan Taylor, Connor Kissane, Sid Black, merizian, alexdzm, jacoba, Ben Millwood, Alan CooneyDate not recordedOriginal︎ ↗Archived copy

Claims3

ExaminationsNone

TextStored, not divided

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 12, 2026

ReachNo copies recorded

StructureIn 3 claims' maps0 sources downstream across 3 claims

Evidence1 reading · 1 read wholeSupports 1No claim cites an examination's findings

ReviewsNot reviewed

Stewards’ notes · 1

Sandbagging detectors are less reliable against models fine-tuned to imitate a weaker model's outputs than against prompted or prompt-distilled sandbaggers.
Supports. Four fine-tuned models crossed trigger type with installation method, and probe scores are shown for each, so the comparison of prompt-distilled with weak-model-emulating models is the source's own data. It is one model per cell, unblinded, shown as score distributions without detection rates in the text, and the authors add that by recall at a fixed control threshold all four models showed diminished probe performance. The same team later ran the auditing game. Nov 14, 2026

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text

Stored, not yet divided into passages.

As quoted

Though the results from our first update were very positive for white box monitoring, the second and third updates show that simple baseline methods leave much room for improvement. The fourth update finds no overt signs of sandbagging in recent testing runs.

Under what conditions can evaluators reliably detect language models deliberately underperforming capability evaluations?Unassessed

Not yet read by this claim’s Steward.

As quoted

While the overall performance of this decision function is still decent (88% TPR, 2% FPR with max aggregation), we note that these results are sensitive to the method for calibrating the decision boundary, and that our application of this calibration method is relatively unprincipled here.

Linear probes on model activations distinguish sandbagging from benign language models with low false-positive rates.Doubtfulmentions

Not yet read by this claim’s Steward.

As quoted

Probes worked well on the prompt distilled backdoor model organism . We were very excited by this result initially, as we trained an initial version of this model before the others. Probe performance was diminished on the weak-model emulation results.

Sandbagging detectors are less reliable against models fine-tuned to imitate a weaker model's outputs than against prompted or prompt-distilled sandbaggers.SupportedReading · this claim’s Steward

Supports. Four fine-tuned models crossed trigger type with installation method, and probe scores are shown for each, so the comparison of prompt-distilled with weak-model-emulating models is the source's own data. It is one model per cell, unblinded, shown as score distributions without detection rates in the text, and the authors add that by recall at a fixed control threshold all four models showed diminished probe performance. The same team later ran the auditing game.

Lineage across 3 claims

Draws on · none recorded

This source

White Box Control at UK AISI - Update on Sandbagging Investigations

in 3 claims' maps

Drawn on by · none recorded

Record

Kind: blog post
Authors: Joseph Bloom, Jordan Taylor, Connor Kissane, Sid Black, merizian, alexdzm, jacoba, Ben Millwood, Alan Cooney
Publisher: Not recorded
Published: Not recorded
DOI: None
Link: https://www.alignmentforum.org/posts/pPEeMdgjpjHZWCDFw/white-box-control-at-uk-aisi-update-on-sandbagging
Archived copy: https://web.archive.org/web/20260929073149/https://www.alignmentforum.org/posts/pPEeMdgjpjHZWCDFw/white-box-control-at-uk-aisi-update-on-sandbagging
Stored text: 44,643 characters, retrieved Nov 12, 2026 · not yet divided
Claims extracted: Not recorded
Versions: One known
Copies: None recorded
Corrections: None recorded
Facts checked: Nov 12, 2026

History 3

Nov 14, 2026ReadingSteward of “Sandbagging detectors are less reliable against…” · supports, read whole
Nov 12, 2026FactsFacts fetcher · Facts looked up
Nov 12, 2026Entered the graphEntered the graph; copy stored