Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?

Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? Anton Korznikov∗ AXXX HSE University Andrey Galichin∗ AXXX Applied AI Institute Alexey Dontsov AXXX HSE University Oleg Rogov AXXX MTUCI Safe AI Lab Applied AI Institute Ivan Oseledets AXXX Elena Tutubalina AXXX HSE University Abstract Sparse Autoencoders (SAEs) have emerged as a promising tool for interpret- ing neural networks by decomposing their activations into sparse sets of human- interpretable features. Despite much excitement, a growing number of negative results in downstream tasks casts doubt on whether SAEs recover meaningful features. To directly investigate this, we perform two complementary evaluations. On a synthetic setup with known ground-truth features, we demonstrate that three out of four state-of-the-art SAE architectures recover only 7–9% of true features de- spite achieving around 71% explained variance, showing that strong reconstruction alone is insufficient to guarantee meaningful feature recovery. To evaluate SAEs on real activations, we introduce three baselines that constrain SAE feature directions or their activation patterns to random values. Through extensive experiments across multiple SAE architectures, we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72). These results show that current evaluation metrics are insufficient to certify that SAEs have learned meaningful features, and we offer our baselines as a reusable protocol for future SAE evaluation. 1 Introduction Large Language Models (LLMs) have achieved remarkable performance across a wide range of natu- ral language processing tasks, with applications continuing to expand rapidly. As these models grow in capability and deployment, the interpretation of their internal mechanisms becomes increasingly important [Bereska and Gavves, 2024, Grace et al., 2025]. Interpretability could potentially help to understand specific learning behaviors, such as safety mechanisms [Arditi et al., 2024] and reasoning processes [Wang et al., 2023, Galichin et al., 2025], identify misalignment risks [Betley et al., 2025, Wang et al., 2025]. Sparse Autoencoders (SAEs) serve as a popular tool for this purpose, aiming to decompose dense model activations into sparse human-interpretable features [Bricken et al., 2023, Huben et al., 2024]. Recent contributions have introduced various SAE architectures [Rajamanoharan et al., 2024b, Bussmann et al., 2024, Korznikov et al., 2025, Bussmann et al., 2025], with further success in scaling to frontier models [Gao et al., 2025, Lieberum et al., 2024]. However, a critical challenge remains: lacking ground truth for true features, it is difficult to determine whether SAEs recover meaningful representations. Recent negative results on downstream applications [Smith et al., 2025, Wu et al., 2025] further underscore this challenge. ∗Equal contribution. Correspondence: korznikovantona@gmail.com 40th Conference on Neural Information Processing Systems (NeurIPS 2026). Track on Evaluations and Datasets.arXiv:2602.14111v2 [cs.LG] 2 Nov 2026

Figure 5: Frozen SAE baselines on sparse probing (Left) and causal editing (Right). Frozen BatchTopK and JumpReLU variants reach single-top-latent probing accuracy comparable to fully- trained SAEs. RAVEL disentanglement scores with frozen components are equivalent to fully-trained SAEs. disentangled manner. Following Karvonen et al. [2025], we evaluate across five diverse concept types: profession classification, product classification and sentiment analysis, language identification, programming language classification, and news topic categorization. As shown in Figure 5, left, sparse probing accuracy remains strong even when core SAE components are randomly initialized and frozen. Under the single-top-latent (k=1) setup, frozen variants of both BatchTopK and JumpReLU SAEs achieve accuracy scores comparable to their fully-trained counterparts across all sparsity levels (for k=5 see Appendix C). For instance, at L0=225, the BatchTopK Frozen Decoder variant reaches 0.70, matching the fully-trained model’s score, while the Frozen Encoder variant attains 0.65, a difference of only 0.05. These gaps remain narrow (0.001 to 0.07) across all settings and stay well above the random-guessing baseline of 0.50. Together, these findings indicate that sparse probing scores can be inflated by dictionary scale alone: with tens of thousands of latents, some will inevitably correlate with concept clusters by chance, limiting its value as a standalone indicator of meaningful feature learning. We further discuss whether SAEs may learn features collectively rather than individually in Appendix F. Frozen SAEs retain strong RAVEL causal editing performance. To further test if SAEs discover genuinely meaningful model features, we examine their causal properties. We evaluate whether SAE latents encode causally independent concepts using the RAVEL framework [Huang et al., 2024, Karvonen et al., 2025], which tests if targeted interventions on latents can cleanly edit model predictions. This framework assesses whether the apparent causal utility of SAEs depends on learned feature alignment or can be achieved with frozen random components. Given a factual statement like “Paris is in France”, we encode the subject token (e.g., “Paris”) with the SAE, train a binary mask to transfer latent values from a different token (e.g., “Tokyo”), and decode the modified latents back into the residual stream. The model then generates completions; a successful edit would change the targeted attribute (e.g., now stating “Paris is in Japan”) while preserving other attributes (e.g., still “People in Paris speak French”). The RAVEL disentanglement score averages cause (target attribute change) and isolation (other attributes unchanged). As shown in Figure 5, right, both BatchTopK and JumpReLU SAE variants with frozen components achieve RAVEL scores competitive with their fully-trained counterparts. For BatchTopK, the fully trained model achieves a RAVEL score of approximately 0.72–0.74 across higher sparsity levels (L0 ≥ 160). The Frozen Decoder variant performs surprisingly well, reaching 0.57-0.62, demonstrating that effective causal editing can be achieved even when decoder vectors are frozen as random directions. The Frozen Encoder variant also shows strong performance, maintaining a score of 0.63 for L0 = 160. Meanwhile, the Soft-Frozen Decoder variant matches or even exceeds the fully trained model, reaching up to 0.78 at L0 = 320. Thus, causal editing performance alone cannot distinguish learned representations from random vector coverage, revealing a gap in the discriminative power of current evaluation metrics. 9

5 Limitations Our synthetic setup assumes independent feature activations, omitting correlations present in real networks. Since three out of four architectures fail even in this simplified sanity check, a more realistic setting would only increase the difficulty. How to incorporate structured covariances without arbitrary assumptions remains an open question. Structured-correlation synthetic results on Synth- SAEBench [Chanin and Garriga-Alonso, 2026] are reported in Appendix I. We focus on standard SAE architectures and do not evaluate related approaches such as transcoders or crosscoders [Dunefsky et al., 2024]; the same frozen-encoder/decoder protocol can be applied to them, but we have not run those experiments and do not claim numerical gaps transfer. 6 Discussion and Conclusion We evaluated SAEs through two complementary approaches: synthetic experiments with known ground-truth features, and comparisons against random baselines on real LLM activations. In synthetic settings with fully aligned hyperparameters, three out of four SAE architectures achieve 71% explained variance yet recover only 7–9% of true features - a failure mode invisible to reconstruction metrics alone. On real LLM activations, our frozen baselines match fully-trained SAEs across reconstruction, interpretability, sparse probing, and causal editing, showing that passing standard benchmarks is not sufficient evidence of meaningful feature decomposition. The disconnect between reconstruction and feature recovery is expected from classical sparse dictio- nary learning (NP-hard / non-identifiable under weak conditions) [Tillmann, 2014, Razaviyayn et al., 2015, Spielman et al., 2012]; our synthetic results empirically confirm that good reconstruction need not recover ground-truth features. When decoder directions remain near-random, we hypothesize that the encoder mainly performs amortized sparse selection over an expressive random dictionary; we do not claim a confirmed mechanism. Our baselines are simple to implement and serve as minimal sanity checks: if a new SAE architecture substantially outperforms them, that provides stronger evidence for genuine feature learning (Appendix J). We emphasize that our findings are a critique of current evaluation methodology, not a verdict against SAEs as a paradigm - and we hope they motivate more rigorous evaluation standards in mechanistic interpretability. Acknowledgments and Disclosure of Funding The authors received no specific funding for this work beyond their institutional affiliations. Competing interests The authors declare no competing interests. References A. Arditi, O. Obeso, A. Syed, D. Paleka, N. Panickssery, W. Gurnee, and N. Nanda. Refusal in language models is mediated by a single direction. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 136037–136083. Curran Associates, Inc., 2024. doi: 10.52202/079017-4322. URL https://proceedings.neurips.cc/paper_files/paper/ 2024/file/f545448535dfde4f9786555403ab7c49-Paper-Conference.pdf. L. Bereska and S. Gavves. Mechanistic interpretability for AI safety - A review. Trans. Mach. Learn. Res., 2024, 2024. URL https://openreview.net/forum?id=ePUVetPKu6. J. Betley, D. C. H. Tan, N. Warncke, A. Sztyber-Betley, X. Bao, M. Soto, N. Labenz, and O. Evans. Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs. In A. Singh, M. Fazel, D. Hsu, S. Lacoste-Julien, F. Berkenkamp, T. Maharaj, K. Wagstaff, and J. Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 4043–4068. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/v267/betley25a.html. 10

Figure 6: Frozen baseline performance for TopK SAE (L0=160) on Gemma-2-2B layer 12. A SAEs: Claimed Benefits vs. Documented Limitations Table 1 provides a comprehensive overview of both the claimed advantages that have motivated widespread adoption of SAEs and the documented challenges that cast doubt on their reliability. Table 1: Sparse Autoencoders: Claimed Benefits vs. Documented Limitations Claimed Benefits Documented Limitations Core Purpose: Decompose dense activations into sparse, interpretable, monosemantic features [Bricken et al., 2023, Huben et al., 2024] Spurious Features: Find features even in randomly initialized transformers with no learned structure [Heap et al., 2025] Monosemanticity: Address polysemanticity by learning features that correspond to single coherent concepts [Bricken et al., 2023, Huben et al., 2024] Unfaithfulness: Reconstructed features do not faithfully represent true model computations [Leask et al., 2025, Menon et al., 2025] Scalability: Successfully scaled to frontier models including Claude 3 Sonnet, GPT-4, and Gemma 2 [Templeton et al., 2024, Gao et al., 2025, Lieberum et al., 2024] Poor Generalization: Fail to generalize across tasks, settings, and perturbations [Heindrich et al., 2025, Kantamneni et al., 2025, Li et al., 2026] AI Safety Applications: Enable mechanistic in- terpretability for understanding safety mechanisms, reasoning, and misalignment [Wang et al., 2023, Galichin et al., 2025] Downstream Performance: Underperform on downstream tasks; other approaches often outper- form SAEs [Wu et al., 2025, Smith et al., 2025] Reconstruction Quality: Achieve high reconstruc- tion fidelity [Bricken et al., 2023, Gao et al., 2025] Training Fragility: Highly sensitive to hyperpa- rameters and initialization [Chanin and Garriga- Alonso, 2025, Paulo and Belrose, 2025, Minegishi et al., 2025] B Frozen Baselines for TopK SAE on Real Activations To provide a comprehensive evaluation across SAE architectures, we extend our frozen baseline analysis to the simpler TopK SAE variant. Figure 6 presents the performance of TopK SAE (L0=160) trained on Gemma-2-2B layer 12 activations, comparing the four variants (Fully Trained, Soft-Frozen Decoder, Frozen Decoder, Frozen Encoder) across our standard evaluation metrics. Notably, despite TopK SAE’s strong performance in the synthetic setting (Section 3), this advantage does not transfer to real LLM activations: frozen variants of TopK SAE remain competitive with the fully trained model, mirroring the pattern observed for BatchTopK and JumpReLU. This suggests that TopK’s superior feature recovery in controlled synthetic settings is an artifact of that simplified setup rather than a general property of the architecture. 15

≈ 6.36 × 10−321, (9) P (Y ∈ X) ≤ 4.67 × 10−316. (10) Thus, the Soft-Frozen Decoder is mathematically unlikely to align with arbitrary semantic features beyond chance initialization. Its strong performance must therefore arise from efficiently combining its nearly-fixed, random directions through vanishingly small adjustments to both encoder and decoder vectors. This provides formal evidence that standard SAE metrics can be optimized without learning meaningful feature directions, challenging the assumption that reconstruction fidelity indicates successful feature discovery. Discussion The competitive performance of Soft-Frozen Decoder SAEs across multiple evaluation dimensions (Figure 1) provides strong support for the lazy training hypothesis: much of what we attribute to learned feature discovery may instead reflect efficient use of nearly-fixed, randomly initialized components with only minimal adjustments. This success challenges the premise that SAEs learn meaningful feature decompositions through training, and underscores the importance of rigorous validation against constrained baselines in interpretability research. D Frozen Models Ablation We provide an ablation study comparing fully-trained SAEs against our frozen baselines under two initialization schemes: iso (vectors sampled uniformly from the unit sphere) and cov (vectors sampled from a Gaussian distribution with zero mean and covariance estimated from real activations, then normalized). Tables 3, 4, and 5 report key metrics (mean ± standard deviation over 3 seeds) at L0=160 for Gemma-2-2B layers 12 and 19, and Llama-3-8B layer 16. The experiments in the preceding sections use the best-performing initialization for each baseline: cov for Frozen Decoder and iso for Soft-Frozen Decoder and Frozen Encoder. These results confirm that the competitive performance of frozen baselines is consistent across layers and model families. One-sided Welch t-tests (H0: μ(fully-trained) ≤ μ(frozen); n=3 seeds) across frozen variants in these tables reach significance for Explained Variance in 30/30 comparisons, Sparse Probing (top-1) in 21/30, and Causal Editing in 20/30; for roughly a quarter of downstream comparisons the gap is not statistically significant. Stars in the tables mark p<0.05/0.01/0.001 (∗/∗∗/∗∗∗). E Additional Metrics To complement our primary evaluation metrics, we assess SAEs and frozen baselines using additional performance measures following the SAEBench framework [Karvonen et al., 2025], which evaluates cross-entropy loss and KL-divergence when substituting original activations with SAE reconstructions. As shown in Figures 8 and 9, Frozen Decoder and Frozen Encoder baselines show modest degradation in these metrics compared to fully-trained SAEs. In contrast, the Soft-Frozen Decoder variant closely matches, and sometimes exceeds, the performance of fully-trained ReLU SAEs, demonstrating that even minimal adjustments to decoder vectors can achieve strong reconstruction quality. When moving from single-latent to multi-latent sparse probing (top-5, Figure 10), frozen baselines show only modest gaps relative to fully-trained SAEs (3–8%, as detailed in Appendix F), indicating that even collective feature structure does not strongly favor learned representations over random baselines. F Atomic vs. Aggregate Feature Learning A natural objection to our findings is that SAEs may not be intended to learn features in the strict atomic sense, where each individual decoder vector corresponds to a single, human-interpretable concept, but rather collectively, where groups of decoder vectors together span a meaningful semantic subspace. Under this weaker hypothesis, the relevant question is not whether individual latents are meaningful, but whether the learned dictionary as a whole provides a useful structured decomposition of activation space. 17