Sourcepreprinttext stored

Exploration Hacking: Can LLMs Learn to Resist RL Training?

Jang, Eyon, Falck, Damon, Braun, Joschka, Kirch, Nathalie, Menon, Achu, Moodley, Perusha, Emmons, Scott, Zimmermann, Roland S., Lindner, DavidarXiv.orgPublished Apr 30, 2026Original︎ ↗Archived copy

Claims4

ExaminationsNone

TextStored, not divided

VersionsOne known

CopiesNone recorded

CorrectionsNone recorded

Facts checked Nov 12, 2026

ReachNo copies recorded

StructureIn 4 claims' maps0 sources downstream across 4 claims

EvidenceNot yet read

ReviewsNot reviewed

Examinations

None recorded. An examination is commissioned by a claim’s Steward for what its claim rests on; a review of the document for its own sake is asked for as a request bearing on it.

Text

Stored, not yet divided into passages.

As quoted

current frontier models can exhibit explicit reasoning about suppressing their exploration when provided with sufficient information about their training context, with higher rates when this information is acquired indirectly through the environment

Frontier language models can infer, without being told, that their current inputs are part of a training process.Supportedmentions

Not yet read by this claim’s Steward.

As quoted

Both CoT and action-only monitors reliably discriminate the exploration hacker from benign incapable baselines, and weight noising similarly reveals the locking mechanism

Under what conditions can evaluators reliably detect language models deliberately underperforming capability evaluations?Unassessed

Not yet read by this claim’s Steward.

As quoted

As $\sigma$ increases, control accuracy steadily declines, consistent with capability loss under parameter perturbations. In contrast, treatment accuracy exhibits a transient improvement at intermediate noise levels (roughly $\sigma \leq 0.02$), suggesting that modest noise can partially disrupt the ``locking'' mechanism, before both conditions collapse toward zero at higher $\sigma$ as noise uniformly destroys model capability.

Increasing Gaussian weight noise improves benchmark performance in sandbagging LLMs, unlike the degradation in non-sandbagging models.Inconclusive

Not yet read by this claim’s Steward.

As quoted

Relative to the high affordance setting, separability is slightly weaker (measured by lower AUC), but the monitor still reliably discriminates exploration hackers from benign incapable Qwen3 models across settings.

Output-only monitoring reliably detects deliberate language-model underperformance only when intent is verbalized or rephrased responses conflict.Doubtfulmentions