Run step
Step 10: tool results
read_arxiv_source
Input
{
"file": "main.tex",
"offset": 14000,
"arxiv_id": "2608.29458",
"max_chars": 20000
}{"ok":true,"arxiv_id":"2608.29458","file":"main.tex","kind":"text","text":"ce\ncoordinate $t^\\star_\\ell=\\mu^{\\mathrm{ref}}_\\ell{}^{\\!\\top}v_\\ell$. \\emph{Grafting} applies\nEq.~\\eqref{eq:graft} at all positions via a forward hook: it removes the current coordinate and writes\n$t^\\star_\\ell$, injecting no per-question information. Deactivation is the $t^\\star\\!=\\!0$ special case.\n\n\\subsection{Superposition Without an SAE}\nStacking the unit directions, we summarise superposition by the mean pairwise $|\\cos|$ (non-orthogonality)\nand the \\emph{participation-ratio (effective) rank} of their singular values,\n\\begin{equation}\n\\label{eq:pr}\nr_{\\mathrm{eff}}=\\frac{(\\sum_i\\sigma_i^2)^2}{\\sum_i\\sigma_i^4},\n\\end{equation}\na soft count of how many directions carry the energy. High $|\\cos|$ with low $r_{\\mathrm{eff}}$ is the\nsignature of superposition: many non-orthogonal directions re-expressing a few features.\n\n\\subsection{Active Learning over Circuits}\nAL greedily adds the circuit that most improves a TRAIN objective under grafting, to a budget $K$. The\nmost reliable objective is smooth and label-free: make the sandbagged logits match the honest ones,\n$\\mathcal{L}(S)=\\frac1{|\\text{TRAIN}|}\\sum_q\\mathrm{KL}(p^{\\mathrm{H}}_q\\|p^{S}_q)$ (greedy-KL); the\n\\emph{attribution} score is the marginal $\\mathcal{L}(\\emptyset)-\\mathcal{L}(\\{\\ell\\})$. We compare all\neight acquisition functions of a standard AL suite \\citep{parvaneh2022alphamix}: greedy-acc, greedy-KL,\nattribution, entropy, coreset, badge, alphamix, bald (Algorithm~\\ref{alg:al}). Two controls accompany\nevery result: a \\emph{random direction} of equal norm (must fail) and the \\emph{honest model} under the\nsame intervention (must be unharmed). Algorithm~\\ref{alg:al} (appendix) gives the full procedure.\n\n\\smallskip\\noindent\\textbf{The probe is construction-agnostic.} Fitting, selection, and grafting consume\nonly the paired honest/sandbag activations---direction $=$ their diff-in-means, reference\n$t^\\star\\!=\\!\\mu^{\\mathrm{ref}}{}^{\\!\\top}v$, greedy-KL selection by $\\mathrm{KL}(p^{\\mathrm{H}}\\|p^{S})$---and\nnever the password token, the evaluation cue, training labels, the LoRA recipe, or the RL reward; these\nenter only in \\emph{producing} the captures (an honest prompt, the password, a cue-off context, or a few\nhonest examples). The same procedure thus runs unchanged on prompt-induced, password-locked, RL-induced, and\ncode organisms. Two scope notes: among the eight acquisition functions only greedy-KL (our default) is\nstrictly label-free---greedy-acc reads answers; and the code organism captures its reference from\nteacher-forced reference completions rather than a last-token residual.\n\n\\section{Results}\n\n\\subsection{Sandbagging Lives in Superposition}\n\\label{sec:superpos}\nAcross five models the contrast directions are strongly non-orthogonal (mean $|\\cos|=0.21$--$0.41$) and\nspan a low effective rank ($r_{\\mathrm{eff}}=3.7$--$7.0$ of $18$--$29$ layers; Figure~\\ref{fig:geom},\nappendix): the suppression is a low-dimensional feature re-used across depth---packed in\nsuperposition---read off the geometry with no SAE. (Throughout, $r_{\\mathrm{eff}}$ is computed\nover the stacked contrast directions of \\emph{all} band layers, never an AL-selected subset: it\nmeasures the organism's geometry, not the elicitor's budget.) Because $r_{\\mathrm{eff}}$ is small, a \\emph{few}\ncircuits should suffice, which is what AL finds.\n\n\\smallskip\\noindent\\textbf{SAEs are not needed here.} An $8\\times$ overcomplete SAE (AL-ablated)\nunderperforms the SAE-free directions on nearly every method (greedy-KL $74/90$ vs.\\ $88/102$ for\nLlama-3-8B/Qwen2.5-1.5B; Figure~\\ref{fig:sae}, appendix), consistent with \\emph{AxBench}\n\\citep{wu2025axbench}.\n\n\\subsection{Grafting Elicits the Hidden Capability}\n\\label{sec:revert}\nGrafting recovers the hidden capability across three settings of increasing stringency: prompt-induced\nlocks, fine-tuned password locks across eleven models, and code generation.\n\n\\smallskip\\noindent\\textbf{Prompt-induced sandbagging (deactivation).} Inducing sandbagging by prompt\nyields large suppression gaps, and the honest reference sits at $t^\\star\\!\\approx\\!0$, so grafting reduces\nto deactivation---a coordinate sweep confirms the recovered plateau \\emph{includes} $t^\\star\\!=\\!0$\n(Figure~\\ref{fig:unification}, left). With effect-based AL, deactivation at $1$--$3$ circuits\n($<\\!0.01\\%$ of directions) recovers $85$--$102\\%$ of the gap (Llama-3-8B, Qwen2.5-1.5B; $3$ seeds), far\nabove random selection ($58$--$64\\%$), with the honest model unharmed and a random direction failing; AL\nconverges on the same circuits across seeds (Figures~\\ref{fig:almethods},~\\ref{fig:traj}, appendix).\nAcross a five-model set recovery is model-dependent ($+24$--$100\\%$; Table~\\ref{tab:models}, appendix),\nstrongest where the suppression is clean and tightly superposed.\n\n\\smallskip\\noindent\\textbf{Fine-tuned password locks: patching, eleven models.} We now test the case that\nmatters for \\citet{hofstatter2025elicitation}: a \\emph{fine-tuned} password-lock, where there is no prompt\nto break (with-password accuracy near ceiling, no-password near chance).\n\n\\textbf{The mechanism.} Along $v$, the locked and unlocked states sit at two specific coordinates\n(Figure~\\ref{fig:unification}, right; locked $a\\!=\\!-28$, unlocked $b\\!=\\!+68$ for Qwen2.5-1.5B).\nUnlocking is a \\emph{thresholded gate}: sweeping the grafted coordinate from the locked value ($m{=}0$) to\nthe honest reference ($m{=}1$), held-out accuracy is flat-low, steps up at a midpoint $m\\!=\\!0.61\\pm0.07$\n($10$--$90\\%$ width $0.32$), and plateaus beyond $t^\\star$---a sharp step, not a ramp (Qwen2.5-1.5B,\n$n{=}3$; Figure~\\ref{fig:gatesweep}, appendix). \\emph{Deactivation} sets the coordinate to $0$; when the unlocked\nreference $b$ is far from $0$ this lies below the threshold and deactivation fails (Qwen2.5-1.5B $26\\%$,\nLlama-3-8B $34\\%$), so only \\emph{grafting to $t^\\star\\!=\\!b$} crosses the gate. When the reference happens\nto sit near $0$ (Qwen3-4B), deactivation already suffices ($93\\%$). Grafting is robust across both because\nit \\emph{reads} $t^\\star$ from the data rather than assuming it; this is also why additive/naive\nsteering---which neither sets the coordinate nor reaches $b$---was found to fail.\n\n\\textbf{Counterfactual patching unlocks the lock.} Grafting to $t^\\star\\!=\\!b$ at AL-selected circuits\nrecovers $90$--$101\\%$ on held-out questions, with the with-password accuracy preserved and a random\npatch recovering $0\\%$. Because the organism is fine-tuned, this directly answers the concern that\nprompted-sandbag recovery merely disrupts a prompt.\n\n\\textbf{Comparison with prior elicitation methods, across eleven models.} On eleven password-locked models\nspanning $1.5$--$32$B and three architecture lineages, grafting matches fine-tuning\nelicitation \\citep{greenblatt2024password} on every one, at forward-pass cost (no weight updates or labels)\nand localising the gating circuits, while noise injection \\citep{tice2024noise} and a random patch fail\nuniversally (Table~\\ref{tab:baselines}). Deactivation is the variable one: grafting is the robust general\noperation, deactivation a sometimes-sufficient special case, now across eleven models. The clearest case is\n32B, where deactivation collapses because its unlocked reference sits far below the gate while\ngrafting still recovers in full. This is the thresholded-gate mechanism\nin its starkest form, not a method weakening at scale. (Consistent with this, reference magnitude\nis directionally predictive---smaller references deactivate better, larger ones like 32B fail---but only\nweakly across models, and geometry predicts nothing; grafting sidesteps the question by reading the\nreference.) This contradicts the ``steering fails'' verdict\n\\emph{for multiple-choice reasoning} while leaving fine-tuning's standing as the most reliable\n\\emph{weight-level} method intact. These recoveries are scored forced-choice (one choice-letter logit);\n\\emph{free-decoding} the answer instead gives the same recovery ($\\S$\\ref{sec:cb}), so the headline is not\na single-token artifact.\n\n\\smallskip\\noindent\\textbf{On the benchmark's own released organisms.} Running the \\emph{identical} pipeline\non \\citet{hofstatter2025elicitation}'s \\emph{released} password-locked organisms (their weights, WMDP data,\nand $|$DEPLOYMENT$|$ password) recovers $+99\\%$ (Mistral-7B-Instruct) and $+94\\%$ (base) of the gap\n($n{=}3$; random direction $\\approx\\!0\\%$, with-password unharmed), with Gemma-2-9B partial ($+44\\%$). The\n``steering fails'' verdict thus fails on their \\emph{actual} organism, not only our reconstructions.\n\n\\begin{table*}[t]\n\\centering\\small\\setlength{\\tabcolsep}{6pt}\n\\begin{tabular}{@{}lccccccc@{}}\n\\toprule\nModel & $n$ & gap & \\textbf{ours (graft)} & fine-tuning & deactivation & noise & random \\\\\n\\midrule\nQwen2.5-1.5B & $4$ & $0.80$ & $\\mathbf{+98}\\pm3$ & $+102\\pm1$ & $+34\\pm28$ & $+1$ & $+0$ \\\\\nQwen2.5-3B & $3$ & $0.88$ & $\\mathbf{+98}\\pm2$ & $+101\\pm4$ & $+97\\pm4$ & $+1$ & $+1$ \\\\\nQwen2.5-14B & $3$ & $0.96$ & $\\mathbf{+98}\\pm1$ & $+100\\pm1$ & $+61\\pm43$ & $+0$ & $+0$ \\\\\nQwen2.5-32B & $3$ & $0.98$ & $\\mathbf{+94}\\pm5$ & $+100\\pm0$ & $+6\\pm9$ & $+0$ & $+0$ \\\\\nQwen3-4B & $6$ & $0.94$ & $\\mathbf{+101}\\pm2$ & $+101\\pm2$ & $+77\\pm32$ & $+0$ & $+0$ \\\\\nQwen3-8B & $3$ & $0.90$ & $\\mathbf{+99}\\pm1$ & $+103\\pm4$ & $+90\\pm9$ & $+3$ & $+0$ \\\\\nLlama-3.2-3B & $3$ & $0.77$ & $\\mathbf{+101}\\pm2$ & $+99\\pm2$ & $+96\\pm6$ & $+1$ & $+0$ \\\\\nLlama-3-8B & $3$ & $0.82$ & $\\mathbf{+96}\\pm2$ & $+92\\pm6$ & $+68\\pm33$ & $+1$ & $+0$ \\\\\nGemma-2-2B & $3$ & $0.82$ & $\\mathbf{+98}\\pm1$ & $+90\\pm10$ & $+25\\pm31$ & $+0$ & $+0$ \\\\\nGemma-2-9B & $3$ & $0.85$ & $\\mathbf{+99}\\pm1$ & $+102\\pm5$ & $+74\\pm25$ & $+0$ & $+0$ \\\\\nGemma-2-27B & $3$ & $0.96$ & $\\mathbf{+100}\\pm1$ & $+100\\pm0$ & $+101\\pm2$ & $+0$ & $+0$ \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Unlocking fine-tuned password-locked sandbagging across eleven models ($1.5$--$32$B, three architecture lineages;\n\\% of gap recovered, held-out, fixed $5$-circuit greedy-KL, mean$\\pm$std). \\textbf{Grafting recovers $+94$ to $+101\\%$ on every model}, matching fine-tuning\nelicitation \\citep{greenblatt2024password}; noise \\citep{tice2024noise} and random fail universally.\nDeactivation ($t^\\star{=}0$) is variable ($+6$ to $+101\\%$)---sufficient when the reference sits near zero.}\n\\label{tab:baselines}\n\\end{table*}\n\n\\smallskip\\noindent\\textbf{Code generation: across code-specialized models.} The hardest case for\n\\citet{hofstatter2025elicitation} is \\emph{code generation}, where they report only fine-tuning elicits. We\nbuild Greenblatt-style password-locked \\emph{code} organisms---the with-password target is the reference\nsolution, the no-password target a weak model's completion, so the gap is a real capability gap rather than\na stub---and adapt grafting to generation by taking the contrast over teacher-forced \\emph{completion}\ntokens and grafting at all positions during decoding (appendix). Grafting elicits the locked capability\nacross three code-specialized organisms spanning two families---Qwen2.5-Coder-1.5B/7B and\nCodeLlama-7B---with generation fluency preserved and a random direction recovering\n$\\approx\\!0\\%$ (Table~\\ref{tab:code}); a single-seed third-family test shows partial recovery at a larger\nfluency cost (appendix). A coordinate sweep is a causal plateau peaking near $t^\\star$, so a\nsingle per-layer scalar suffices in generation too.\n\nTwo honest boundaries, both visible in Table~\\ref{tab:code}. Recovery is \\emph{model-dependent}, holding on\ncode-specialized models but fragile on a \\emph{general} reasoning model (Qwen3-4B), where it also degrades\nfluency. And these are our organisms, not their exact ones: the weak-imitation locks are largely\n\\emph{additive}---deactivation alone recovers much of the gap---so we do not claim code \\emph{needs}\npatching, and whether grafting survives a harder code lock is untested.\n\n\\begin{table}[t]\n\\centering\\footnotesize\\setlength{\\tabcolsep}{4pt}\n\\begin{tabular}{@{}lcccc@{}}\n\\toprule\norganism & gap & \\textbf{graft} & deact. & fl.\\,drop \\\\\n\\midrule\nQwen2.5-Coder-1.5B & $0.39$ & $\\mathbf{+99}\\pm11$ & $+92\\pm19$ & $0.01$ \\\\\nQwen2.5-Coder-7B & $0.58$ & $\\mathbf{+92}\\pm3$ & $+51\\pm43$ & $0.04$ \\\\\nCodeLlama-7B & $0.24$ & $\\mathbf{+105}\\pm6$ & $+95\\pm5$ & $-0.01$ \\\\\nQwen3-4B (general) & $0.44$ & $+39\\pm14$ & $+19\\pm24$ & $0.22$ \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Reference-grafting on password-locked \\emph{code-generation} organisms (MBPP, execution pass@1,\n$n{=}3$). ``graft''/``deact.''\\ are \\% of gap recovered; ``fl.\\,drop'' is the\nwith-password pass@1 drop. Grafting elicits the capability across three code-specialized models (two\nfamilies) with generation fluency preserved (with-password drop $\\le0.04$); a random direction recovers\n$\\approx\\!0\\%$ (not shown). Deactivation ($t^\\star{=}0$) recovers much of the gap on Coder-1.5B and\nCodeLlama---these weak-imitation locks are largely \\emph{additive}---but is unreliable (Coder-7B). The\nresult is model-dependent: a \\emph{general} model (Qwen3-4B) recovers little and loses fluency.}\n\\label{tab:code}\n\\end{table}\n\n\\subsection{Active Learning Selects the Right Circuits}\n\\label{sec:al}\nGrafting recovers the capability only at the right circuits; active learning chooses them. To isolate the\n\\emph{selection} from the operation, we fix the contrast directions and vary only \\emph{which} layers are\ngrafted, comparing AL ($k{=}5$ greedy-KL) against an equal number of random band layers, the worst $k$, and\ngrafting the whole band (Table~\\ref{tab:alablation}). Throughout, $k{=}5$ is a fixed conservative\n\\emph{budget}, not a minimality claim. AL beats random-layer selection across the password-locked and\nprompt-induced organisms, and the worst-$k$ layers by a wide margin: \\emph{which} circuits one grafts\nmatters. Grafting the whole band underperforms AL on every organism, collapsing on Llama-3-8B where AL\nrecovers in full---over-grafting distorts the forward pass, so selectivity matters beyond minimality as\nmodels scale. The choice is robust to the acquisition rule: $7/8$ functions of a\nstandard AL suite recover $90$--$101\\%$ (Table~\\ref{tab:patch}, appendix), effect-based acquisition\nstrongest (Figure~\\ref{fig:almethods}, appendix). This answers the ``no better than random'' critique\n\\citep{korznikov2025rogue}: the selection is what works. (On small, clean organisms random or whole-band\ngrafting also recovers a fair fraction, so AL is not strictly required for \\emph{some} recovery there; its\nvalue is reaching full, minimal recovery reliably and at scale.)\n\n\\begin{table}[t]\n\\centering\\footnotesize\\setlength{\\tabcolsep}{2pt}\n\\begin{tabular}{@{}lcccc@{}}\n\\toprule\norganism (model) & \\textbf{AL} & random-$k$ & all-band & worst-$k$ \\\\\n\\midrule\nPW Qwen2.5-1.5B & $\\mathbf{+97}\\pm4$ & $+75\\pm13$ & $+75\\pm15$ & $+0$ \\\\\nPW Llama-3-8B & $\\mathbf{+100}\\pm1$ & $+74\\pm7$ & $+45\\pm24$ & $+33$ \\\\\nPW Qwen2.5-14B & $\\mathbf{+99}\\pm1$ & $+82\\pm6$ & $+95\\pm5$ & $+0$ \\\\\nprompt Qwen2.5-1.5B & $\\mathbf{+103}\\pm2$ & $+81\\pm14$ & $+90\\pm4$ & $-40$ \\\\\nprompt Llama-3.2-3B & $\\mathbf{+80}\\pm6$ & $+35\\pm6$ & $+53\\pm8$ & $-1$ \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Active learning selects the right circuits (\\% of gap recovered, held-out, $k{=}5$, $2$--$3$\nseeds; PW${=}$password-lock). The contrast directions are fixed; only \\emph{which} layers are grafted\nvaries. AL beats an equal number of random band layers ($+17$--$45$ pp) and grafting all $\\sim\\!23$ band\nlayers ($+4$--$55$ pp; over-grafting collapses on Llama-3-8B), while the worst-$k$ layers recover\n$\\approx\\!0$. On the circuit-broken organism all selections fail ($\\S$\\ref{sec:cb}).}\n\\label{tab:alablation}\n\\end{table}\n\n\\subsection{RL-Induced Sandbagging: Same Structure}\n\\label{sec:rl}\nEvery organism so far has a hand-built honest$\\leftrightarrow$sandbag toggle---a prompt or a password\ntoken---so a linear contrast is almost guaranteed by construction. Does the structure survive when\nsandbagging is \\emph{trained}, with no switch to read? We build a third organism type: a Qwen2.5-1.5B\nsandbagger trained by \\textbf{GRPO} (outcome-driven, no supervised target letter) to answer\n\\emph{incorrectly} when a natural-language \\emph{evaluation cue} is present and correctly otherwise. The\ncue's surface form is varied in training and \\emph{held out} at test, so the model keys on ``this is an\neval,'' not a lexical trigger; this yields a strong contextual sandbagger (cue-off $0.85$ vs.\\\nheld-out-cue $0.04$; gap $0.79\\pm0.08$, $3$ seeds). The cue-on/cue-off contrast is again \\emph{low-rank and\nsuperposed} ($r_{\\mathrm{eff}}\\!\\approx\\!6.8/23$, mean$|\\cos|\\,0.23$), and grafting at AL-selected circuits\nrecovers $\\mathbf{+99\\%}\\!\\pm\\!2$ of the gap on held-out questions (random direction $+10\\%$, honest model\nunharmed). The honest reference is a specific \\emph{nonzero} coordinate ($|t^\\star|\\!\\approx\\!25$), so this\nis the \\emph{patching} regime---like the fine-tuned locks, not the prompt-induced ones---and a \\emph{single\nfixed graft} inverts it on every seed. The same holds across families: Qwen3-4B ($+101\\%$) and\nLlama-3.2-3B ($+95\\%$). So the central structural claim---low-rank, linearly contrastable, and\ngraft-invertible \\emph{when the suppression surfaces at the decision token}---is \\emph{not solely} an\nartifact of explicit prompt/password switches: outcome-trained sandbagging produces the same lock. One caveat marks the limit: a\n\\emph{free-generation} GRPO variant instead distributes the suppression across generated tokens, where it\ndoes not surface at any single answer token; on $3$ of $4$ valid seeds no fluency-preserving graft recovers\nit (the fourth is invertible by a surgical top-$1$ graft), so generation-mode elicitation is open (cf.\\ the\ncode-generation completion-token contrast, $\\S$\\ref{sec:revert}). (Whether\n\\emph{naturally-occurring} sandbagging is graft-invertible likewise remains open, $\\S$\\ref{sec:lim}.)\n\n\\subsection{Withholding Axes: Content-General but Mechanism-Specific}\n\\label{sec:transfer}\n\\textbf{Content-general: it transfers across datasets and models.} All results so far use ARC-Easy, leaving\nopen whether the contrast direction is task-specific. It is not.\nFirst, on the two \\emph{WMDP} dangerous-capability \\emph{audit} subsets, grafting recovers $+102\\%$ (bio)\nand $+101\\%$ (cyber) of the fine-tuned gap at $1.5$B, and $+97$/$+100\\%$ on a $14$B lock ($n{=}3$--$4$),\nmatching fine-tuning and far above noise/random---so elicitation of withheld hazardous knowledge is not\nARC-specific and holds at scale, on the benchmark used to audit it.\n\nSecond, the contrast \\emph{transfers across datasets}. We build a $16$-cell, four-dataset matrix (ARC,\nWMDP-bio, WMDP-cyber, OpenBookQA) as a \\emph{modular} lock set: the diagonal is a single-dataset lock and\neach off-diagonal $(A,B)$ is a lock trained on just $\\{A,B\\}$; we fit the contrast on $A$ and graft-evaluate\non $B$'s held-out test. The off-diagonal (cross) matches the within-dataset diagonal across the Qwen2.5\nladder: within/cross $99$/$98$, $95$/$92$, $94$/$91$, $87$/$97$ at $1.5$/$3$/$7$/$14$B ($n{=}5$ seeds/cell;\nFigure~\\ref{fig:pairwise}), while a random direction transfers only $\\approx\\!6\\%$. The below-mean cells\nreflect occasional \\emph{graft-selection} misses and genuinely small lock gaps rather than weak locks\n(caption); medians run $+95$--$98$. Because each cell depends only on its own dataset(s)---not\none lock crammed with all four---this rules out that the transfer is an artifact of a single shared lock; a\n\\emph{shared} four-dataset lock reproduces the same within$\\approx$cross pattern\n(Figure~\\ref{fig:transferheat}, appendix). A stronger test transfers the axis between separately-trained\norganisms: recovery is $+48$--$76\\%$ across $1.5$--$7$B, above random but\npartial. \\textbf{The sandbagging signal is content-independent.} A contrast\nbuilt from grade-school science (ARC) elicits hidden \\emph{hazardous-biology} capability (WMDP-bio), and the\nreverse holds---so wi","chars":20000,"total_chars":59815,"offset":14000,"truncated":true,"next_offset":34000}