Step 6: tool results

read_page

{
  "url": "https://arxiv.org/abs/2605.31183",
  "max_chars": 2500
}

{"ok":true,"url":"https://arxiv.org/abs/2605.31183","text":"[2605.31183] Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search &middot; Advanced search\n-->\nComputer Science > Computation and Language\narXiv:2605.31183 (cs)\n[Submitted on 29 May 2026]\nTitle: Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines\nAuthors: Mikkel Godsk Jørgensen , Lars Kai Hansen\nView a PDF of the paper titled Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines, by Mikkel Godsk J{\\o}rgensen and 1 other authors\nView PDF\nHTML (experimental)\nAbstract: Sparse Autoencoders (SAEs) have been seen as a promising avenue for exploring the internals of Large Language Models (LLMs) and for steering model output generation. When AxBench - a model steering benchmark - was introduced in Wu et al. (2025), SAEs did not seem to live up to their original hype due to poor steering performance relative to a set of simple baselines. This work serves as a partial rebuttal for Sparse Autoencoders and suggests that the results of Wu et al. (2025) did not do them full justice. We find that Sparse Autoencoders can, in fact, perform close to on par with the reference LoRA performance on the AxBench benchmark, when features are selected and labelled with our supervised pipeline. We also find that our pipeline selects features that are surprisingly causal of their identified labels when using only its interpretability-based components. Lastly, we present evidence that high sparsity (low l0) may not be crucial for successful steering based on interpretability, which is in contrast to the earlier findings in Wang et al. (2025).\nSubjects:\nComputation and Language (cs.CL) ; Artificial Intelligence (cs.AI); Machine Learning (cs.LG)\nCite as:\narXiv:2605.31183 [cs.CL]\n(or\narXiv:2605.31183v1 [cs.CL] for this version)\nhttps://doi.org/10.48550/arXiv.2605.31183\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Mikkel Godsk Jørgensen [ view email ] [v1]\nFri, 29 May 2026 11:53:54 UTC (4,566 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines, by Mikkel Godsk J{\\o}rgensen and 1 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nAdditional Features\nAudio Summary\nCurrent browse context:\ncs.CL\n< prev\n|\nnext >\nnew\n|\nrecent\n| 2026-05\nChange to browse by:\ncs\ncs.AI\ncs.LG\nReferences & Citations\nSciX\nGoogle S","chars":2500,"total_chars":4677,"offset":0,"truncated":true,"next_offset":2500}

read_page

{
  "url": "https://arxiv.org/abs/2602.14111",
  "max_chars": 2500
}

{"ok":true,"url":"https://arxiv.org/abs/2602.14111","text":"[2602.14111] Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search · Advanced search\n-->\nComputer Science > Machine Learning\narXiv:2602.14111 (cs)\n[Submitted on 15 Feb 2026 ( v1 ), last revised 2 Nov 2026 (this version, v2)]\nTitle: Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?\nAuthors: Anton Korznikov , Andrey Galichin , Alexey Dontsov , Oleg Rogov , Ivan Oseledets , Elena Tutubalina\nView a PDF of the paper titled Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?, by Anton Korznikov and 5 other authors\nView PDF\nHTML (experimental)\nAbstract: Sparse Autoencoders (SAEs) have emerged as a promising tool for interpreting neural networks by decomposing their activations into sparse sets of human-interpretable features. Recent work has introduced multiple SAE variants and successfully scaled them to frontier models. Despite much excitement, a growing number of negative results in downstream tasks casts doubt on whether SAEs recover meaningful features. To directly investigate this, we perform two complementary evaluations. On a synthetic setup with known ground-truth features, we demonstrate that SAEs recover only $9\\%$ of true features despite achieving $71\\%$ explained variance, showing that they fail at their core task even when reconstruction is strong. To evaluate SAEs on real activations, we introduce three baselines that constrain SAE feature directions or their activation patterns to random values. Through extensive experiments across multiple SAE architectures, we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72). Together, these results suggest that SAEs in their current state do not reliably decompose models' internal mechanisms.\nSubjects:\nMachine Learning (cs.LG)\nCite as:\narXiv:2602.14111 [cs.LG]\n(or\narXiv:2602.14111v2 [cs.LG] for this version)\nhttps://doi.org/10.48550/arXiv.2602.14111\nFocus to learn more\narXiv-issued DOI via DataCite\nSubmission history\nFrom: Anton Korznikov [ view email ] [v1]\nSun, 15 Feb 2026 11:53:55 UTC (4,856 KB)\n[v2]\nMon, 2 Nov 2026 13:17:33 UTC (5,772 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?, by Anton Korznikov and 5 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nAddit","chars":2500,"total_chars":4816,"offset":0,"truncated":true,"next_offset":2500}

read_page

{
  "url": "https://arxiv.org/abs/2505.20063",
  "max_chars": 2500
}

{"ok":true,"url":"https://arxiv.org/abs/2505.20063","text":"[2505.20063] SAEs Are Good for Steering -- If You Select the Right Features\nSkip to main content\nSearch\nSubmit\nDonate\nLog in\nSearch arXiv\nPress Enter to search · Advanced search\n-->\nComputer Science > Machine Learning\narXiv:2505.20063 (cs)\n[Submitted on 26 May 2025 ( v1 ), last revised 22 Dec 2025 (this version, v2)]\nTitle: SAEs Are Good for Steering -- If You Select the Right Features\nAuthors: Dana Arad , Aaron Mueller , Yonatan Belinkov\nView a PDF of the paper titled SAEs Are Good for Steering -- If You Select the Right Features, by Dana Arad and 2 other authors\nView PDF\nHTML (experimental)\nAbstract: Sparse Autoencoders (SAEs) have been proposed as an unsupervised approach to learn a decomposition of a model's latent space. This enables useful applications such as steering - influencing the output of a model towards a desired concept - without requiring labeled data. Current methods identify SAE features to steer by analyzing the input tokens that activate them. However, recent work has highlighted that activations alone do not fully describe the effect of a feature on the model's output. In this work, we draw a distinction between two types of features: input features, which mainly capture patterns in the model's input, and output features, which have a human-understandable effect on the model's output. We propose input and output scores to characterize and locate these types of features, and show that high values for both scores rarely co-occur in the same features. These findings have practical implications: after filtering out features with low output scores, we obtain 2-3x improvements when steering with SAEs, making them competitive with supervised methods.\nSubjects:\nMachine Learning (cs.LG) ; Artificial Intelligence (cs.AI); Computation and Language (cs.CL)\nCite as:\narXiv:2505.20063 [cs.LG]\n(or\narXiv:2505.20063v2 [cs.LG] for this version)\nhttps://doi.org/10.48550/arXiv.2505.20063\nFocus to learn more\narXiv-issued DOI via DataCite\nRelated DOI :\nhttps://doi.org/10.18653/v1/2025.emnlp-main.519\nFocus to learn more\nDOI(s) linking to related resources\nSubmission history\nFrom: Dana Arad [ view email ] [v1]\nMon, 26 May 2025 14:47:59 UTC (4,746 KB)\n[v2]\nMon, 22 Dec 2025 15:49:59 UTC (5,507 KB)\nFull-text links:\nAccess Paper:\nView a PDF of the paper titled SAEs Are Good for Steering -- If You Select the Right Features, by Dana Arad and 2 other authors\nView PDF\nHTML (experimental)\nTeX Source\nview license\nAdditional Features\nAudio Summary\nCurrent browse ","chars":2500,"total_chars":4784,"offset":0,"truncated":true,"next_offset":2500}