Run step
Step 16: tool results
read_arxiv_source
Input
{
"file": "main.tex",
"offset": 78000,
"arxiv_id": "2501.17148",
"max_chars": 14000
}{"ok":true,"arxiv_id":"2501.17148","file":"main.tex","kind":"text","text":"ach method as fairly as possible.\n\n\\section{Results}\n\n\\subsection{\\concepttag{} Concept detection}\n\n% \\begin{figure*}\n% \\centering\n% \\includegraphics[width=\\linewidth]{figures/concept500_roc.pdf}\n% \\caption{\\concepttag{} ROC curves for each method on every concept in \\textsc{Concept500} when evaluating layer 10 of Gemma-2-2B.}\n% \\label{fig:roc-detailed}\n% \\end{figure*}\n\nFor concept detection, \\textsc{Concept500} consists of passages of text with ground-truth labels for each concept. Each method provides us with token-level concept scores obtained from the representation of that token at a particular layer. To compute a passage-level score, we take the mean of the token-level concept scores. See \\cref{sec:concept_detection_examples} for a visualization of token-level concept scores.\n\n\\begin{table}[!t]\n \\small\n \\centering\n\\begin{tabular}{lccccc}\n\\toprule\n\\multirow[t]{2}{*}{\\textbf{Method}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-2B}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-9B}} & \\multirow[t]{2}{*}{\\textbf{Avg.}} \\\\\n& \\multicolumn{1}{c}{L10} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L31} \\\\\n% identifier & 2b, l10, concept500 & 2b, l20, concept500 & 9b, l20, concept500 & 9b, l31, concept500 \\\\\n% method & & & & \\\\\n\\midrule\nDiffMean & \\underline{0.948} & 0.946 & 0.955 & 0.921 & \\textbf{0.942} \\\\\nProbe & 0.940 & 0.946 & 0.933 & \\textbf{0.942} & \\underline{0.940} \\\\\nReFT-r1 & \\textbf{0.952} & \\textbf{0.965} & \\textbf{0.966} & 0.869 & \\underline{0.938} \\\\\n\\rowcolor{gray!20} Prompt & 0.910 & 0.921 & 0.940 & 0.943 & 0.929 \\\\\nSAE-A & 0.924 & 0.911 & 0.924 & 0.907 & 0.917 \\\\\n\\rowcolor{gray!20} BoW & 0.909 & 0.931 & 0.904 & 0.912 & 0.914 \\\\\nSSV & 0.934 & 0.950 & 0.910 & 0.854 & 0.912 \\\\\nLAT & 0.742 & 0.809 & 0.572 & 0.725 & 0.712 \\\\\nSAE & 0.735 & 0.755 & 0.631 & 0.659 & 0.695 \\\\\nPCA & 0.714 & 0.712 & 0.559 & 0.622 & 0.652 \\\\\nIG & 0.440 & 0.375 & 0.508 & 0.383 & 0.426 \\\\\nIxG & 0.243 & 0.217 & 0.193 & 0.330 & 0.246 \\\\\n\\bottomrule\n\\end{tabular}\n \\caption{\\concepttag{} Mean AUROC for each method on concept detection. \\textbf{Bold} indicates highest AUROC in that column; \\underline{underline} indicates no significant difference vs.~the best performer. \\textcolor{gray}{Gray} indicates non-representation steering methods.}\n \\label{tab:roc-auc}\n\\end{table}\n\n% \\paragraph{ROC curves.} Using the passage-level scores and the ground truth labels on the evaluation dataset, we compute the ROC (rate of change) curve for each method on each concept across all model settings. In \\cref{tab:roc-auc}, we report the average area under the ROC curve (ROC AUC) for each method over all concepts. We plot the average ROC curve for each method in \\cref{fig:avg-roc}.\n\n\\begin{figure}[!t]\n \\centering\n \\includegraphics[width=1.0\\linewidth]{figures/concept500_balance.pdf}\n \\caption{\\concepttag{} Mean F1 scores vs.~dataset balance.}\n \\label{fig:avg-f1}\n\\end{figure}\n\n\\paragraph{AUROC.} In \\cref{tab:roc-auc}, we report the average area under the ROC curve (AUROC) for each method over all concepts. Overall, we find that DiffMean, Probe, and ReFT-r1 are the best performers with no statistically significant difference ($p < 0.05$) between any of them under a paired $t$-test. Prompt, SAE-A, and SSV are not far behind and significantly outperform the remaining methods. LAT also performs better than random. Vanilla SAEs\n% without supervised feature selection \nare thus significantly outperformed by five supervised methods, all of which are much cheaper to train using a limited amount of synthetic data. The remaining methods (PCA, IG, and IxG) perform poorly; PCA's better-than-random performance is nevertheless impressive given its unsupervised nature. Additional results are given in \\cref{sec:detail-result-analysis}.\n\n% We plot all of the ROC curves under one model and layer setting (layer 10 in Gemma-2-2B) in \\cref{fig:roc-detailed}. We note that for PCA (and, to a lesser extent, LAT), ROC curves can be grouped into two classes by shape; near-optimal curves which ascend to the top left, and signal-less curves which are diagonal. This suggests that only sometimes is the direction with the most variation also the best classification direction.\n\n% \\begin{table}[]\n% \\small\n% \\centering\n% \\begin{tabular}{lccccc}\n% \\toprule\n% \\multirow[t]{2}{*}{\\textbf{Method}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-2B}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-9B}} & \\multirow[t]{2}{*}{\\textbf{Avg.}} \\\\\n% & \\multicolumn{1}{c}{L10} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L31} \\\\\n% % identifier & 2b, l10, concept500 & 2b, l20, concept500 & 9b, l20, concept500 & 9b, l31, concept500 \\\\\n% % method & & & & \\\\\n% \\midrule\n% ReFT-r1 & \\textbf{0.935} & \\textbf{0.949} & \\textbf{0.942} & 0.870 & \\textbf{0.924} \\\\\n% DiffMean & 0.919 & 0.927 & 0.922 & \\underline{0.904} & 0.918 \\\\\n% Probe & 0.908 & 0.914 & 0.907 & \\textbf{0.905} & 0.909 \\\\\n% SSV & 0.900 & 0.922 & 0.871 & 0.809 & 0.876 \\\\\n% LAT & 0.738 & 0.797 & 0.623 & 0.742 & 0.725 \\\\\n% SAE & 0.746 & 0.761 & 0.652 & 0.677 & 0.709 \\\\\n% PCA & 0.713 & 0.700 & 0.605 & 0.667 & 0.671 \\\\\n% IG & 0.552 & 0.540 & 0.568 & 0.528 & 0.547 \\\\\n% IxG & 0.536 & 0.533 & 0.535 & 0.546 & 0.538 \\\\\n% \\bottomrule\n% \\end{tabular}\n% \\caption{\\concepttag{} Mean classification accuracy for each method.}\n% \\label{tab:classification}\n% \\end{table}\n\n% \\begin{figure}[]\n% \\centering\n% % Subfigure\n% \\includegraphics[width=0.9\\linewidth]{figures/concept500_precision_vs_recall.pdf}\n% \\caption{\\concepttag{} Mean precision vs.~recall.}\n% \\label{fig:avg-prec-rec}\n% \\end{figure}\n\n\\paragraph{F1 score under class imbalance.} In real-world text, positive instances of concepts are much rarer than negative instances.\n% (as reflected by e.g.~SAE concept sparsity).\nWe thus report F1 on both the balanced setting (50\\% positive instances) and an imbalanced setting with 3600 additional negative examples ($\\approx$1\\% positive).\nWe choose classification threshold by maximising F1, binarise the resulting predictions, and report statistics on this discrete classification. \\Cref{fig:avg-f1} shows that the relative ordering of methods does not change substantially between the two settings; despite their sparsity, SAEs perform poorly, but LAT and PCA also degrade substantially.\n% We report the mean classification accuracy for each method in \\cref{tab:classification}, noting similar ordering to mean ROC AUC. We also report mean precision vs.~recall in \\cref{fig:avg-prec-rec}, noting the high precision but low recall for SAEs. This is probably a consequence of the auto-interp pipeline, which uses only positive exemplars to produce concept labels.\n\n\\subsection{\\steeringtag{} Model steering}\\label{sec:result-model-steering}\n\n\\begin{table}[!t]\n \\centering\n \\small\n % Subtable\n\\adjustbox{max width=\\textwidth}{\n\\begin{tabular}{lccccc}\n\\toprule\n\\multirow[t]{2}{*}{\\textbf{Method}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-2B}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-9B}} & \\multirow[t]{2}{*}{\\textbf{Avg.}} \\\\\n& \\multicolumn{1}{c}{L10} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L31} \\\\\n% identifier & 2b, l10, concept500 & 2b, l20, concept500 & 9b, l20, concept500 & 9b, l31, concept500 \\\\\n% method & & & & \\\\\n\\midrule\n\\rowcolor{gray!20}Prompt & \\underline{0.698} & \\textbf{0.731} & \\textbf{1.075} & \\textbf{1.072} & \\textbf{0.894} \\\\\n\\rowcolor{gray!20}LoReFT & \\textbf{0.701} & \\underline{0.722} & 0.777 & 0.764 & 0.741 \\\\\n\\rowcolor{gray!20}SFT & 0.637 & 0.714 & --- & --- & \\textit{0.676} \\\\\n\\rowcolor{gray!20}LoRA & 0.637 & 0.641 & 0.602 & 0.580 & 0.615 \\\\\nReFT-r1 & 0.633 & 0.509 & 0.630 & 0.401 & 0.543 \\\\\nDiffMean & 0.297 & 0.178 & 0.322 & 0.158 & 0.239 \\\\\nSAE & 0.177 & 0.151 & 0.191 & 0.140 & 0.165 \\\\\nSAE-A & 0.166 & 0.132 & 0.186 & 0.143 & 0.157 \\\\\nLAT & 0.117 & 0.130 & 0.127 & 0.134 & 0.127 \\\\\nPCA & 0.107 & 0.083 & 0.128 & 0.104 & 0.105 \\\\\nProbe & 0.095 & 0.091 & 0.108 & 0.099 & 0.098 \\\\\nSSV & 0.072 & 0.001 & 0.024 & 0.008 & 0.026 \\\\\n\\bottomrule\n\\end{tabular}\n}\n \\caption{\\steeringtag{} Mean overall steering scores for each method, after steering factor selection. \\textcolor{gray}{Gray} indicates non-representation steering methods.}\n \\label{tab:steer-judge}\n\\end{table}\n\n\\begin{figure}[!t]\n \\centering\n % Subfigure\n \\includegraphics[width=1.0\\linewidth]{figures/concept500_concept_vs_instruct.pdf}\n \\caption{\\steeringtag{} Mean concept score vs.~instruct score as the steering factor for each method is varied.}\n \\label{fig:mean-concept}\n\\end{figure}\n\nFor model steering, we take concept labels from \\textsc{Concept500} and apply the (pre)trained steering methods to the base model and sample generations. We score the generations using an LM judge as described in \\cref{sec:model-steering}. We additionally benchmark prompting, full-finetuning (SFT), and two parameter-efficient finetuning methods (LoReFT and LoRA) as non-steering baselines.\n\nFor steering methods, we note that steering factor is an important hyperparameter. We select the optimal steering factor for each method independently for every concept based on which factor achieves the highest \\textit{overall} steering score, as given by the LLM judge. Our actual steering magnitude (i.e., $\\alpha$, as described in \\cref{sec:methods}) is the product of the steering factor and the maximal activations aggregated over the evaluation dataset for concept detection.\\footnote{For SAEs, we query \\href{https://www.neuronpedia.org/}{Neuronpedia} to obtain the maximal activation per concept.}\n\n\\paragraph{Overall scores.} We report the mean overall score for each method (i.e.~the harmonic mean of three subscores: fluency, instruction-following, and concept presence) in \\cref{tab:steer-judge}. Prompting, along with slightly worse finetuning baselines, outperforms all steering methods on average, except for ReFT-r1. ReFT-r1 is competitive with prompting in Gemma-2-2B but significantly behind on Gemma-2-9B; prompting scores improve by a large margin on the larger model. Additionally, DiffMean significantly outperforms SAEs, particularly in earlier layers. \n\nThe remaining supervised steering methods fail to beat SAEs, and no steering methods besides ReFT-r1 approach prompting or finetuning performance. Importantly, we note that SAE-A slightly underperforms the unsupervised SAE; better classification does not directly lead to better steering.\n\n\\paragraph{Winrate.} We compute winrates against SAEs by comparing overall scores on each concept under each setting. We treat ties as 0.5 wins and 0.5 losses. We report the results in \\cref{tab:winrate}. Again, ReFT-r1 (88.0\\%) and DiffMean (61.6\\%) achieve winrates of greater than 50\\% against SAEs, and relative rankings are similar to those for overall score. We note that DiffMean and ReFT-r1 show higher winrates on earlier layers in both models.\n\n\\begin{table}[]\n \\centering\n \\small\n % Subtable\n\\adjustbox{max width=\\textwidth}{\n\\begin{tabular}{lccccc}\n\\toprule\n\\multirow[t]{2}{*}{\\textbf{Method}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-2B}} & \\multicolumn{2}{c}{\\textbf{Gemma-2-9B}} & \\multirow[t]{2}{*}{\\textbf{Avg.}} \\\\\n& \\multicolumn{1}{c}{L10} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L20} & \\multicolumn{1}{c}{L31} \\\\\n% identifier & 2b, l10, concept500 & 2b, l20, concept500 & 9b, l20, concept500 & 9b, l31, concept500 \\\\\n% method & & & & \\\\\n\\midrule\n% \\rowcolor{gray!20} Prompt & 85.1\\% & \\textbf{88.6\\%} & \\textbf{96.6\\%} & \\textbf{98.6\\%} & \\textbf{92.2\\%} \\\\\n% ReFT-r1 & \\textbf{90.0\\%} & \\underline{87.6\\%} & 90.0\\% & 84.4\\% & 88.0\\% \\\\\n% \\rowcolor{gray!20} SFT & \\textbf{90.0\\%} & 80.0\\% & --- & --- & \\textit{85.0\\%} \\\\\n% \\rowcolor{gray!20} LoReFT & 81.4\\% & 84.6\\% & 85.5\\% & 87.2\\% & 84.7\\% \\\\\n% \\rowcolor{gray!20} LoRA & 75.4\\% & 79.6\\% & 73.8\\% & 75.4\\% & 76.0\\% \\\\\n% DiffMean & 69.3\\% & 56.7\\% & 69.4\\% & 51.1\\% & 61.6\\% \\\\\n% \\rowcolor{blue!20} SAE & 50.0\\% & 50.0\\% & 50.0\\% & 50.0\\% & 50.0\\% \\\\\n% SAE-A & 48.1\\% & 47.1\\% & --- & --- & \\textit{47.6\\%} \\\\\n% Probe & 37.6\\% & 40.8\\% & 36.3\\% & 41.6\\% & 39.1\\% \\\\\n% LAT & 34.6\\% & 40.8\\% & 33.0\\% & 44.0\\% & 38.1\\% \\\\\n% PCA & 32.3\\% & 32.2\\% & 29.7\\% & 38.9\\% & 33.3\\% \\\\\n% SSV & 21.3\\% & 15.5\\% & 18.0\\% & 19.8\\% & 18.6\\% \\\\\n\\rowcolor{gray!20}Prompt & \\textbf{90.0}\\% & \\textbf{91.5}\\% & \\textbf{97.6}\\% & \\textbf{99.1}\\% & \\textbf{94.5}\\% \\\\\n\\rowcolor{gray!20}LoReFT & \\underline{88.9\\%} & 88.2\\% & 88.6\\% & 90.3\\% & 89.0\\% \\\\\n\\rowcolor{gray!20}SFT & \\textbf{90.0}\\% & 87.5\\% & --- & --- & \\textit{88.8}\\% \\\\\n\\rowcolor{gray!20}LoRA & 85.0\\% & 83.4\\% & 79.9\\% & 81.5\\% & 82.5\\% \\\\\nReFT-r1 & 85.2\\% & 82.3\\% & 83.6\\% & 76.0\\% & 81.8\\% \\\\\nDiffMean & 63.2\\% & 55.2\\% & 64.3\\% & 52.2\\% & 58.7\\% \\\\\nSAE & 50.0\\% & 50.0\\% & 50.0\\% & 50.0\\% & 50.0\\% \\\\\nSAE-A & 49.3\\% & 46.6\\% & 48.5\\% & 50.7\\% & 48.8\\% \\\\\nLAT & 43.5\\% & 48.2\\% & 42.7\\% & 48.6\\% & 45.8\\% \\\\\nPCA & 42.1\\% & 42.9\\% & 42.2\\% & 45.4\\% & 43.1\\% \\\\\nProbe & 40.4\\% & 44.0\\% & 41.9\\% & 45.6\\% & 43.0\\% \\\\\nSSV & 38.8\\% & 32.0\\% & 32.5\\% & 34.0\\% & 34.3\\% \\\\\n\\bottomrule\n\\end{tabular}\n}\n \\caption{\\steeringtag{} Winrate against SAEs for each method, after steering factor selection.}\n \\label{tab:winrate}\n\\end{table}\n\n% \\section{Analysis}\n\n\\paragraph{Steering factor.} We compare the effect of changing the steering factor on instruct vs.~concept scores in \\cref{fig:mean-concept}. We notice that increasing the factor monotonically reduces instruct score in all methods, i.e.~larger steering vectors harm capabilities; this agrees with prior findings \\cite{durmus2024steering,chalnev2024improvingsteeringvectorstargeting}. However, the effect varies by layer for concept score: concept score increases then decreases in earlier layers, while it roughly monotonically increases with steering factor in later layers. In all cases, ReFT-r1 traces a Pareto-optimal path, achieving the highest concept score for any chosen instruct score.\n\n% \\paragraph{\\textsc{Concept16K}.} We scale up two supervi","chars":14000,"total_chars":134717,"offset":78000,"truncated":true,"next_offset":92000}