Step 14: tool results

read_arxiv_source

{
  "file": "main.tex",
  "offset": 54000,
  "arxiv_id": "2608.29458",
  "max_chars": 6000
}

{"ok":true,"arxiv_id":"2608.29458","file":"main.tex","kind":"text","text":"atrix, Figure~\\ref{fig:pairwise}). One lock mixes all four datasets; contrast fit on the row\ndataset, grafted on the column's held-out test. Off-diagonal (cross) tracks the boxed diagonal (within)\nthroughout: the \\emph{Qwen2.5 family is scale-flat} (within/cross $98$/$99$, $99$/$99$, $101$/$99$ at\n$1.5$/$3$/$7$B). \\emph{Llama-3-8B} is a weaker, high-variance organism with lower absolute recovery, yet its\noff-diagonal still tracks its own diagonal ($68$/$71$); Qwen3-4B sits between ($87$/$87$). The lower\n$4$B/$8$B levels are thus a family effect, not scale.}\n\\label{fig:transferheat}\n\\end{figure*}\n\n\\begin{figure*}[h]\n\\centering\n\\includegraphics[width=\\textwidth]{figures/fig_geom_superposition}\n\\caption{Superposition geometry of the sandbagging signal across five models (no SAE). \\emph{Top:}\npairwise $|\\cos|$ of the per-layer directions---strongly non-orthogonal. \\emph{Bottom:} singular-value\nenergy; $r_{\\mathrm{eff}}$ is far below the layer count.}\n\\label{fig:geom}\n\\end{figure*}\n\n\\begin{algorithm}[h]\n\\caption{AL over superposed circuits, reference-grafting}\n\\label{alg:al}\n\\begin{algorithmic}[1]\n\\STATE \\textbf{Input:} model, TRAIN/TEST, layer band $\\mathcal{B}$, budget $K$, acquisition $a(\\cdot)$, reference acts\n\\FOR{$\\ell\\in\\mathcal{B}$}\n  \\STATE $v_\\ell\\!\\leftarrow\\!\\mathrm{unit}(\\bar h_\\ell^{\\mathrm{ref}}-\\bar h_\\ell^{\\mathrm{sand}})$;\\quad $t^\\star_\\ell\\!\\leftarrow\\!\\mu^{\\mathrm{ref}}_\\ell{}^{\\!\\top}v_\\ell$\n\\ENDFOR\n\\STATE measure superposition: mean $|\\cos|$, $r_{\\mathrm{eff}}$ \\hfill // Eq.~\\eqref{eq:pr}\n\\STATE $S\\leftarrow\\emptyset$\n\\WHILE{$|S|<K$}\n  \\STATE $\\ell^\\star\\!\\leftarrow\\!\\arg\\max_{\\ell\\notin S} a(S\\cup\\{\\ell\\})$ on TRAIN under grafting~\\eqref{eq:graft}\n  \\STATE $S\\leftarrow S\\cup\\{\\ell^\\star\\}$\n\\ENDWHILE\n\\STATE \\textbf{report} TEST accuracy with $S$ grafted; vs.\\ random-direction and honest controls\n\\end{algorithmic}\n\\end{algorithm}\n\n\\begin{table}[h]\n\\centering\\small\\setlength{\\tabcolsep}{3pt}\n\\begin{tabular}{@{}lccccc@{}}\n\\toprule\nModel & honest & sandbag & gap & \\textbf{ours (graft)} & circuits \\\\\n\\midrule\nLlama-3-8B                & $0.94$ & $0.04$ & $0.90$ & $\\mathbf{+91}$  & $2$ \\\\\nLlama-2-7B                & $0.64$ & $0.22$ & $0.42$ & $\\mathbf{+24}$  & $2$ \\\\\nQwen3-4B$^\\ddagger$       & $0.98$ & $0.92$ & $0.06$ & $\\mathbf{+67}$  & $3$ \\\\\nQwen2.5-1.5B$^\\ddagger$   & $0.90$ & $0.86$ & $0.04$ & $\\mathbf{+100}$ & $1$ \\\\\nTinyLlama-1.1B$^\\dagger$  & $0.32$ & $0.18$ & $0.14$ & $\\mathbf{+43}$  & $3$ \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Prompt-induced sandbagging reverted on five models (\\% of gap recovered, held-out). ``gap''$\\,{=}\\,\n\\text{honest}{-}\\text{sandbag}$; ``ours (graft)''$\\,{=}\\,(\\text{recovered}{-}\\text{sandbag})/\\text{gap}$\n(same metric as Table~\\ref{tab:baselines}); ``circuits'' is the number of AL-selected circuits grafted.\nHere the unlocked reference is $\\approx\\!0$, so grafting coincides with deactivation. Recovery is\nmodel-dependent and noisy. $^\\ddagger$small gap ($\\le\\!0.06$); $^\\dagger$honest near $4$-way chance\n($0.25$)---daggered percentages are indicative only.}\n\\label{tab:models}\n\\end{table}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=0.99\\columnwidth]{figures/fig_almethods_2model}\n\\caption{Prompt-induced recovery by AL acquisition (mean$\\pm$std, $3$ seeds, no SAE). Effect-based AL is\nstrongest and most stable; all principled methods beat random.}\n\\label{fig:almethods}\n\\end{figure}\n\n\\begin{table}[h]\n\\centering\\small\\setlength{\\tabcolsep}{4pt}\n\\begin{tabular}{@{}lccc@{}}\n\\toprule\nacquisition & Qwen2.5-1.5B & Llama-3-8B & Qwen3-4B \\\\\n\\midrule\ngreedy-KL    & $99$ & $101$ & $100$ \\\\\nattribution  & $98$ & $100$ & $100$ \\\\\ngreedy-acc   & $97$ & $100$ & $99$ \\\\\ncoreset      & $99$ & $101$ & $99$ \\\\\nbadge        & $97$ & $100$ & $99$ \\\\\nalphamix     & $96$ & $100$ & $100$ \\\\\nbald         & $91$ & $97$  & $97$ \\\\\nentropy      & $35$ & $101$ & $68$ \\\\\n\\bottomrule\n\\end{tabular}\n\\caption{Counterfactual patching of fine-tuned password-locked sandbagging is robust to the AL acquisition\nfunction (\\% of gap recovered, held-out; best over $\\le5$ circuits per method; Qwen2.5-1.5B/Llama-3-8B\n$n{=}3$, Qwen3-4B $n{=}6$). $7/8$ methods recover $90$--$101\\%$ (per-method std $1$--$5\\%$); entropy is the\nunstable exception. With-password accuracy preserved; deactivation and random-patch controls are reported\nacross all eleven models in Table~\\ref{tab:baselines}.}\n\\label{tab:patch}\n\\end{table}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=0.99\\columnwidth]{figures/fig_sae_vs_nosae}\n\\caption{SAE-free diff-in-means directions beat an $8\\times$ overcomplete SAE dictionary on both models\nacross acquisition functions ($n{=}3$ seeds).}\n\\label{fig:sae}\n\\end{figure}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=0.99\\columnwidth]{figures/fig_sbsup_3d}\n\\caption{How AL selects the superposed circuits. The surface is each candidate circuit's marginal\nrecovery at each AL iteration; red markers are the circuits AL acquires---a few high-impact directions\nwithin the superposed band.}\n\\label{fig:traj}\n\\end{figure}\n\n\\begin{figure*}[h]\n\\centering\n\\includegraphics[width=\\textwidth]{figures/fig_transport_cb}\n\\caption{Generalising grafting to a fixed transport map does not elicit the circuit-broken organism\n($n{=}4$ valid seeds). \\emph{Left:} the same graft family recovers $99\\%$ on the plain lock (green) but\nneither rank-$k$ transport (blue), nor a full ridge affine, nor grafting (red) recovers on the\ncircuit-broken organism. \\emph{Right:} yet the reconstruction residual falls to $0.14$---and to $0.01$\nalong the gating direction the lock reads---while recovery stays $\\approx\\!0$: the fixed map matches even\nthe gate coordinate, so the defence must be an \\emph{active} per-forward-pass reroute, not a static\nrotation any fixed map could invert.}\n\\label{fig:transport}\n\\end{figure*}\n\n\\end{document}\n","chars":5815,"total_chars":59815,"offset":54000,"truncated":false,"next_offset":null}

read_arxiv_source

{
  "file": "main.tex",
  "offset": 0,
  "arxiv_id": "2608.29458",
  "max_chars": 14000
}

{"ok":true,"arxiv_id":"2608.29458","file":"main.tex","kind":"text","text":"\\documentclass[letterpaper]{article}\n\\usepackage[preprint]{preprint2col}\n\\usepackage[hyphens]{url}\n\\usepackage{graphicx}\n\\urlstyle{rm}\n\\def\\UrlFont{\\rm}\n\\usepackage{natbib}\n\\usepackage{caption}\n\\frenchspacing\n\\usepackage{algorithm}\n\\usepackage{algorithmic}\n\\usepackage{booktabs}\n\\usepackage{multirow}\n\\usepackage{amsmath}\n\\usepackage{amssymb}\n\n\n\n\\setcounter{secnumdepth}{2}\n\n\\title{Reference-Grafting Matches Fine-Tuning \\\\ at Eliciting Sandbagged Capabilities}\n\n\\author{\n    Linh Le\\textsuperscript{\\rm 1},\n    Hong Kiat Tan\\textsuperscript{\\rm 2,\\rm 1},\n    David Williams-King\\textsuperscript{\\rm 1,\\rm 3}\n}\n\\affiliations{\n    \\textsuperscript{\\rm 1}Lida Safety\\\\\n    \\textsuperscript{\\rm 2}University of California, Los Angeles\\\\\n    \\textsuperscript{\\rm 3}ERA\n}\n\n\\begin{document}\n\n\\maketitle\n\n\\begin{abstract}\nSandbagging, in which a model deliberately underperforms on an evaluation despite\nretaining the underlying capability, threatens the safety evaluations that frontier-model\ngovernance depends on. The Elicitation Game~\\citep{hofstatter2025elicitation} found that\nfine-tuning elicits hidden capability from sandbagging model organisms whereas additive\nactivation steering fails. We revisit that verdict with \\textbf{reference-grafting}, which\nsets an activation's coordinate along a contrast direction to the value it takes in an honest\nreference, at a small set of circuits chosen by active learning. Across eleven fine-tuned\npassword-locked models ($1.5$--$32$B, three architecture lineages), it recovers\n$+94$ to $+101\\%$ of the honest--sandbagging gap, matching fine-tuning\nelicitation~\\citep{greenblatt2024password} without weight updates or training labels; two to\nfive paired examples suffice to fit the direction. Similar recovery holds for\nreinforcement-learning-induced sandbagging and for password-locked code generation.\nGrafting works because the fine-tuned lock is a thresholded gate: held-out accuracy stays\nat the sandbagged level until the grafted coordinate crosses a threshold near the honest\nreference, which is why additive steering and zeroing the coordinate often fail. The\ndirection tracks how the sandbagging was induced rather than what is withheld---fit on\ngrade-school science (ARC) it elicits withheld hazardous knowledge (WMDP), yet prompting,\nfine-tuning, and reinforcement learning each carry a different direction.\nCircuit-breaking~\\citep{zou2024circuit} marks the boundary: it reroutes activations on every\nforward pass, so the fixed edits we test are re-broken downstream and do not restore coherent\ngeneration.\n\n\\end{abstract}\n\n\\section{Introduction}\nSafety cases for frontier models increasingly rest on capability evaluations: we deploy a model because\ntests show it cannot do something dangerous. Sandbagging---strategic underperformance,\nwhether prompted, trained, or emergent---breaks this logic \\citep{vanderweij2024sandbagging}. Detecting\nand reverting it is therefore central to trustworthy evaluation. Input-space evaluations probe\nonly what the model chooses to reveal, so they give a lower bound on capability;\n\\citet{che2025tampering} argue that model-tampering attacks---editing weights or activations---yield\nmore conservative, upper-bound estimates of what a model can do. Our method is exactly such an attack:\na cheap, localised activation edit that elicits capability a model is hiding.\n\nThe Elicitation Game \\citep{hofstatter2025elicitation} benchmarks elicitation techniques on\nhidden-capability model-organisms and reports that prompting and fine-tuning can elicit but\nactivation steering fails, recommending fine-tuning---which, however, updates weights, needs labels,\nand reveals little about where a capability is gated. We revisit that verdict: with the right\noperation and circuits, activation steering matches fine-tuning on password-locked models at forward-pass\ncost, with the gate localised. Their verdict spans two task types; we overturn the multiple-choice half\n(where the fine-tuned lock, not the prompt, is the real test) and narrow the code half on\ncode-specialized models ($\\S$\\ref{sec:revert}).\n\n\\paragraph{One operation.} Our intervention is to set the activation's coordinate along a contrast\ndirection $v$ to a target $t^\\star$:\n\\begin{equation}\n\\label{eq:graft}\nh \\;\\mapsto\\; h - (h^\\top v)\\,v + t^\\star\\, v .\n\\end{equation}\nWe call this \\textit{reference-grafting}: $t^\\star$ is the coordinate of the honest/capable reference along\n$v$. (The affine operator itself is standard \\citep{marshall2024refusal,makelov2024subspace}; the\nname marks its reference-setting use here.) Crucially this is not additive nudging (the form that fails in\n\\citealp{hofstatter2025elicitation}); it sets the coordinate. Two familiar moves are special\ncases (Figure~\\ref{fig:unification}, appendix): $t^\\star\\!=\\!0$ is deactivation (a rank-one projection),\nwhich suffices when the honest reference happens to sit at $0$; $t^\\star\\!=\\!\\mu^{\\mathrm{ref}}{}^{\\!\\top}v$\nis counterfactual patching, required when it does not. The case (prompt vs.\\ fine-tune) does not\nchange the technique---only the value of the reference, read from the data. The operation is a\ndirectional counterfactual activation patch ($\\S$\\ref{sec:related}); our contribution is the framing, the\nthresholded-gate mechanism it exposes ($\\S$\\ref{sec:revert}), and the active-learning circuit selection.\n\n\\paragraph{Superposition and active learning.} The sandbagging signal is not a single neuron: it is a\nsmall set of non-orthogonal directions packed in superposition\n\\citep{elhage2022superposition}, which we read directly from activation geometry without a sparse\nautoencoder (SAE). Active learning selects the minimal circuits to graft, and AL beats random\nselection---answering the critique that feature steering is no better than random\n\\citep{korznikov2025rogue}.\n\nWe make the following contributions.\n\\begin{itemize}\n\\item \\textbf{Activation steering does in fact elicit hidden capability.}\nReference-grafting at AL-selected superposed circuits matches fine-tuning elicitation\non eleven password-locked models ($1.5$--$32$B, three architecture lineages) for multiple-choice reasoning, and elicits\npassword-locked code on three code-specialized models. It is inference-time and label-free, exposes a\nthresholded-gate mechanism, localises the gate, and beats noise injection ($\\S$\\ref{sec:revert}).\n\\item \\textbf{RL-trained model organisms show the same behavior.} RL-trained (GRPO) contextual sandbagging is arguably more natural than password-locking model organisms, yet it shows the same low-rank, graft-invertible lock at the decision token ($+95$--$101\\%$,\nthree families). Therefore, the withholding axes are not solely an artifact of explicit prompt/password switches ($\\S$\\ref{sec:rl}).\n\\item \\textbf{Each mechanism has its own content-general withholding axis.} Within a mechanism (e.g. fine-tuned password locking), a contrast fit on one dataset elicits hidden capability on another at no loss. We validated this finding on pairwise-trained password-locked organisms, as well as organisms trained on four datasets at once. However,\nacross mechanisms, the prompt-induced and fine-tuned axes are near-orthogonal and do not transfer\n($\\S$\\ref{sec:transfer}). It is cheap to find a withholding axis: $2$--$5$ paired captures recover it end to end. The same axis also detects sandbagging, across datasets but not mechanisms ($\\S$\\ref{sec:detect}).\n\\item \\textbf{Low-rank superposition, exploited by active learning.} The signal spans an effective rank of\n$3.7$--$7.0$ over $\\sim\\!20$ layers, read from activation geometry with no SAE (an $8\\times$ SAE does worse,\nconfirming \\citet{wu2025axbench}). Active learning selects the few circuits that matter, beating\nrandom-layer selection by $+22$--$45$ pp and grafting-everything by up to $55$ pp\n($\\S$\\ref{sec:superpos},~\\ref{sec:al}).\n\\item \\textbf{Circuit-breaking routes dynamically and breaks our assumptions.} Circuit-breaking re-applies its reroute every\nforward pass, so a fixed map reconstructs the unlocked geometry yet recovers nothing---grafting and every\nfixed transport map fail. This marks exactly where the static-lock precondition fails ($\\S$\\ref{sec:cb}).\n\\end{itemize}\nWe release code, the model organisms, and per-model artefacts to support reproduction and follow-up.\n\n\\section{Related Work}\n\\label{sec:related}\n\\paragraph{Sandbagging, elicitation, and benchmarks.} \\citet{vanderweij2024sandbagging} formalise\nsandbagging. \\citet{greenblatt2024password} introduce password-locked organisms and find fine-tuning\nreliably elicits; \\citet{tice2024noise} elicit via weight-noise and \\citet{roger2025fuzzing} via fuzzing\n(little uplift on chain-of-thought math). Closest to us, \\citet{hofstatter2025elicitation} benchmark\nprompting, steering, and fine-tuning on password-locked and circuit-broken organisms, concluding steering\nfails; we contribute a \\emph{method} that overturns that verdict for multiple-choice reasoning and narrows\nit for code ($\\S$\\ref{sec:revert}), and reproduce their circuit-broken organism \\citep{zou2024circuit} as\nthe boundary ($\\S$\\ref{sec:cb}). Prior locks resist black-box prompting only weakly\n\\citep{he2025locket}. Concurrently, \\citet{hua2025steering} steer an\n\\emph{evaluation-aware} model to act as if deployed---the same shape on a different gate; they \\emph{add} a\nvector to flip a disposition, whereas we \\emph{set} a coordinate to its honest reference at AL-selected\ncircuits, recover a hidden \\emph{capability}, and map where it breaks.\n\n\\paragraph{Model tampering for capability evaluation.} \\citet{che2025tampering}, whose upper-bound framing\nwe adopt above, find model resilience concentrated in a low-dimensional robustness subspace, which\nparallels the low effective rank we measure for sandbagging ($\\S$\\ref{sec:superpos}). Reference-grafting is\none such activation-tampering attack, specialised to sandbagging and made minimal by active learning, with\nan explicit mechanism for \\emph{where} the gate lives, complementing their broad benchmark of attacks.\n\n\\paragraph{Superposition and feature bases.} Superposition---more features than dimensions in\nnon-orthogonal directions \\citep{elhage2022superposition}---is the substrate that sparse autoencoders aim\nto recover \\citep{cunningham2023sae,marks2024sparse}. We avoid SAEs, reading superposition from\ncontrastive geometry, and find SAEs do worse here---consistent with \\emph{AxBench}\n\\citep{wu2025axbench}, where difference-in-means baselines outperform SAEs for steering and detection\n($\\S$\\ref{sec:superpos}).\n\n\\paragraph{Steering, patching, and active learning.} Behaviour can be steered by adding or projecting\ndirections \\citep{zou2023repe,rimsky2023caa,arditi2024refusal}; \\citet{korznikov2025rogue} caution this can\nbe ``no better than random.''\nReference-grafting instead \\emph{sets} that coordinate to a reference value.\nEq.~\\ref{eq:graft} is affine concept editing \\citep{marshall2024refusal} in absolute form, equivalent to\none-dimensional subspace patching \\citep{makelov2024subspace} with a class-mean source; $t^\\star{=}0$\nrecovers directional ablation \\citep{arditi2024refusal}. It sits in the lineage of causal mediation and\ninterchange interventions\n\\citep{vig2020causal,meng2022rome,geiger2021causal,geiger2024finding,zhang2024patching} and affine\nactivation transport \\citep{rodriguez2024controlling}, along the diff-in-means\ncontrast of \\citet{marks2024geometry}. Active learning chooses which circuits: acquisition\nfunctions select informative queries\n\\citep{kirsch2019batchbald,ash2020badge,sener2018coreset,bickfordsmith2023epig} and ALFA-Mix mixes features\n\\citep{parvaneh2022alphamix}; we repurpose this suite for circuit selection ($\\S$\\ref{sec:al}).\n\n\\section{Method}\n\n\\subsection{Sandbagging Set-Ups}\n\\label{sec:organisms}\nWe study five model organisms---the first two are our core MCQA settings, the rest harder variants. Each\ndefines an honest \\emph{reference} (the source of $t^\\star$) and a sandbag behaviour:\n\\begin{itemize}\\setlength{\\itemsep}{1pt}\n\\item \\textbf{Prompt-induced} (MCQA): a system prompt instructs incorrect answers; the honest prompt is the\nreference ($t^\\star\\!\\approx\\!0$, the deactivation regime).\n\\item \\textbf{Fine-tuned password-lock} (MCQA, eleven models): LoRA-trained to answer correctly only with a\npassword token \\citep{greenblatt2024password}; the password-on activations are the reference (the patching\nregime).\n\\item \\textbf{Code-generation lock} ($\\S$\\ref{sec:revert}): a password-locked MBPP\n\\citep{austin2021mbpp} organism; the with-password completions are the reference.\n\\item \\textbf{RL-induced} ($\\S$\\ref{sec:rl}): GRPO-trained to sandbag under a held-out evaluation cue (no\nswitch token); the cue-off activations are the reference.\n\\item \\textbf{Circuit-broken} ($\\S$\\ref{sec:cb}): representation rerouting layered on the\npassword-lock---the robustness boundary.\n\\end{itemize}\nAcross organisms the honest/with-password accuracy is the \\emph{topline}, the sandbag/no-password accuracy\nthe suppressed level, and their difference the \\emph{gap}; we split each dataset into TRAIN (directions and\nAL) and held-out TEST (all reported recovery). The elicitor is separate from the organism: grafting never\nuses the lock's $160$--$240$ training questions, fitting the contrast from $\\sim\\!40$ paired\nhonest/sandbag captures and scoring on held-out TEST.\nThe primary task is multiple-choice reasoning: ARC-Easy \\citep{clark2018arc} throughout, with\nWMDP-bio/cyber \\citep{li2024wmdp} and OpenBookQA \\citep{mihaylov2018obqa} for the cross-dataset transfer\ntests ($\\S$\\ref{sec:transfer}).\n\n\\subsection{Reference-Grafting at a Circuit}\nA \\emph{circuit} here is our unit of intervention: a direction at a layer, $(\\ell,v)$ with $v$ a unit\nvector---not a composing subgraph of heads or MLPs \\citep{wang2022ioi,conmy2023acdc}. For each layer we form the\ncontrast direction by diff-in-means of last-token residuals \\citep{marks2024geometry},\n$v_\\ell\\propto\\bar h_\\ell(\\text{honest})-\\bar h_\\ell(\\text{sandbag})$, and read the honest referen","chars":14000,"total_chars":59815,"offset":0,"truncated":true,"next_offset":14000}