SAEs Are Good for Steering – If You Select the Right Features

(a) Gemma-2-2B. (b) Gemma-2-9B. Figure 5: Even in later layers of the model (16–25 for Gemma-2-2B and 24–41 for Gemma-2-9B), filtering features with low output scores increases mean genera- tion success. Magenta: filtering by output scores. Green: filtering random sets of features of the same size. shown in Section E. 4.3 Quantitative Results Figure 4 shows the mean generation success @ 20 of steered generations when filtering out features with output scores below varying thresholds (ma- genta) on Gemma-2-2B and Gemma-2-9B. As the threshold increases, performance improves steadily, indicating that features with higher output scores consistently lead to more successful steering. We observe an increase in the mean generation suc- cess score from around 0.5 − 0.6 for both models without any filtering, to 1.1–1.4 using a threshold of 0.9. A threshold of 0.01 is sufficient for filter- ing out about 60% of the features, increasing the mean generation success by around 0.4 points. We compare this against a random baseline (green): fil- tering randomly sampled subsets of features of the same size does not lead to any significant improve- ments (results are average of 10 random samples per subset size). Llama-3.1 and Pythia show similar trends; see Section B. The results in Section 3 suggest that features with high output scores occur predominantly in later layers. Figure 5 shows the generation suc- cess when filtering based on the output score, eval- uated only on features from later layers of the model: 16–25 for Gemma-2-2B and 24–41 for Gemma-2-9B. Taking only features from later lay- ers, Gemma-2-2B and Gemma-2-9B achieve gen- eration success scores of about 0.8. By considering only top-scoring features, the mean generation suc- cess increases to around 1.1–1.4 for both models. These results show that even within these later lay- ers, the output score is a useful tool for filtering out features that lead to poor steering results. Gemma-2-9B-it L20 L31 Our results SAE(Sout≥0.1) 0.546 0.470 SAE(Sout≥0.01) 0.338 0.454 SAE(Sout≥0.001) 0.373 0.415 SAE(Sout≥0.0001) 0.325 0.401 SAE(No Filter) 0.293 0.387 AxBench reported results (Wu et al. 2025) Prompt 1.075 1.072 LoReFT 0.777 0.764 LoRA 0.602 0.580 ReFT-r1 0.630 0.401 DiffMean 0.322 0.158 SAE 0.191 0.140 SAE-A 0.186 0.143 Table 1: Results on the Concept500 dataset from AxBench on instruction-tuned Gemma-2-9B. (Top) Results when steering with SAEs after filtering out fea- tures with Sout lower than different thresholds. (Bot- tom) Results reported by Wu et al. (2025). Bold indi- cates the best score, underline indicates the best score among representation-based methods. Grey indicates non-representation-based methods. After filtering based on output scores, SAEs achieve the best score among representation-based methods at L31, and reach 90.7% of the best method’s performance at L20. 4.4 Evaluation on AxBench AxBench was recently proposed as a dataset to evaluate steering methods (Wu et al., 2025). They compare steering with SAE features to other meth- ods (including supervised methods), and find SAE features relatively ineffective. However, we believe this is partially due to a non-principled selection of SAE features; we propose to remedy this using the output score. We evaluate our findings on instruction-tuned Gemma-2-9B using the Concept500 dataset of Wu et al. (2025), which includes 1000 ⟨concept, SAE feature⟩ pairs from layers 20 and 31 of the model. As in AxBench, we randomly sample 10 instruc- tions for each concept-feature pair from instruction datasets aligned with the concept’s genre. Five are used to select the optimal steering factor (as detailed in Section 4.1), and the remaining five are used exclusively for evaluation. We evaluate the steered texts using the metrics defined by Wu et al.: (1) the concept score measures if the con- cept was incorporated in the generated text; (2) the fluency score measures the coherency of the text; and (3) the instruct score measures the alignment of the generated text with the given instructions. For each feature, we compute the harmonic mean of the three metrics. See Section F for additional

(a) Gemma-2-2B. (b) Gemma-2-9B. Figure 6: Relationship between input and output scores for features in Gemma-2-2B and Gemma-2- 9B. Most features lie near the axes, indicating that most features are either input or output features—though a few are both. details on the steering setup, metrics, and baseline methods. We report the mean score on layers 20 and 31 of instruction-tuned Gemma-2-9B (Table 1). We find a nearly threefold improvement in SAE steering scores relative to Wu et al.. Note that our replica- tion of their experimental setting (without filtering) yields higher scores for SAEs; this can be a result of different sampled instructions per feature,2 or possibly due to evaluation instability introduced by the use of an external LLM. With output score filtering, SAE features top steering performance among the representation-based methods at L31; for L20, they get 90.7% of the performance of the best method. This is in contrast with the results of Wu et al., where SAEs significantly underperforms ReFT-r1, a weakly supervised method they propose as a competitive alternative to prompting. These re- sults demonstrates that with effective feature selec- tion SAEs are comparable with existing methods, including supervised or weakly-supervised meth- ods which require concept-specific datasets. 5 The Relationship Between Input and Output Scores We next examine how input and output scores in- teract. Figure 3 suggests that high input and output scores are rarely observed in the same layers, but do they sometimes co-occur in the same features? Figure 6 visualizes the relationship between the two scores: Indeed, most samples cluster near the axes, exhibiting either a high output score with a near-zero input score, or vice-versa. However, there do exist hybrid features: features with both 2Wu et al. (2025) sample 10 instructions per feature from pre-existing datasets, but do not release these instructions. We sample 10 instructions from the same datasets, which may be different compared to the sample of Wu et al.. high input and high output scores. Figure 7 illustrates generation results when steer- ing with hybrid features from Gemma-2-2B. Gener- ated tokens that also appear in the feature’s top-20 logit lens tokens are highlighted in magenta. These are often not the top-ranked tokens under the logit lens, but they tend to rank moderately high and collocate with the top tokens (highlighted in blue). For instance, for feature 6820 in layer 18, “con- tact” is the top logit lens token, but the output text repeatedly includes “lenses”, a token that appears further down the list but that is semantically and syntactically related. We verify this intuition by quantifying collo- cation patterns between generated tokens and top logit lens tokens using their pointwise mutual infor- mation (PMI)3. A PMI of zero indicates that two tokens co-occur no more frequently than chance, and negative and positive scores indicate lower- or higher-than-chance co-occurrence, respectively. Features with high output score (Sout ≥ 0.1) and low input score (Sin < 0.1) have a negative PMI value on average (Figure 8). This can be attributed to the high generation success of these features, i.e., that the top logit lens token is equal to the generated token. In most cases, two instances of the same token are not likely to consecutively co-occur, thus obtaining a low PMI score. Importantly, features that have a high input score (Sin ≥ 0.1) tend to generate tokens with signifi- cantly higher PMI relative to their top logit lens tokens, regardless of their output score values. This suggests that hybrid features may be less favorable for steering, despite their high output score. 6 Related Work 6.1 Stages of Processing in LMs The different stages of processing within NLP mod- els have long been studied (Belinkov et al., 2017; Zhang and Bowman, 2018; Liu et al., 2019; Brun- ner et al., 2020). In transformer-based LMs, a large body of work shows that different properties emerge in different layers. Early layers focus on syntactic tasks such as POS, while semantic infor- mation appears in later layers (Tenney et al., 2019; Elazar et al., 2021; Geva et al., 2021). More recent work has demonstrated that intermediate layers are responsible for retrieving factual knowledge and 3We use the pre-computed PMI over the webtext corpus in NLTK (Bird and Loper, 2004), and only include token pairs that have this pre-computed score (100–200 pairs per model).