Position: Use Sparse Autoencoders to Discover Unknowns

Position: Use Sparse Autoencoders to Discover Unknowns Kenny Peng * 1 Rajiv Movva * 2 Jon Kleinberg 1 Emma Pierson 2 Nikhil Garg 1 Abstract While sparse autoencoders (SAEs) have gener- ated significant excitement, a series of negative re- sults have added to skepticism about their useful- ness. Here, we establish a conceptual distinction that reconciles competing narratives surrounding SAEs. We argue that even if SAEs may be less ef- fective for acting on known concepts, SAEs are es- pecially powerful tools for discovering unknown concepts. This distinction separates existing neg- ative results from positive results, and suggests several classes of SAE applications. Specifically, we outline use cases for SAEs in (i) ML inter- pretability, explainability, fairness, auditing, and safety, and (ii) social and health sciences. 1. Introduction Sparse autoencoders (SAEs) have been a popular topic in interpretability research, showing impressive capabilities for identifying interpretable directions in the text representa- tions underlying language models (Cunningham et al., 2023; Templeton et al., 2024). For example, an Anthropic paper found a “Golden Gate Bridge” direction, which could be ma- nipulated to make a chatbot that would always incorporate the Golden Gate Bridge into responses (Anthropic, 2024). However, two recent papers show that SAEs fail to outper- form simple baselines in large-scale evaluations on concept detection (probing) and model steering (Kantamneni et al., 2025; Wu et al., 2025). These results have led to pessimism about the usefulness of SAEs. For example, in response to this research, the mechanistic interpretability team at Google DeepMind announced that they would deprioritize research into SAEs (Smith et al., 2025). Nonetheless, there continues to be optimism about new applications of SAEs, including in hypothesis generation (Movva et al., 2025b) and in the “biology” of LLMs (Lindsey et al., 2025). *Equal contribution 1Cornell University 2UC Berkeley. Corre- spondence to: Kenny Peng <kennypeng@cs.cornell.edu>, Rajiv Movva <rmovva@berkeley.edu>. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s). How can we square continued interest in SAEs with thor- ough evaluations demonstrating negative results? Are new attempts to use SAEs misguided? Or is there something missing in our understanding of the negative results? This position paper reconciles conflicting narratives surrounding SAEs by making a conceptual distinction. Our position is that SAEs—even if less effective for acting on known concepts—are powerful tools for discovering unknown concepts. Consider the tasks where negative results have been shown. Concept detection involves detecting a known, prespecified concept (“Does this text mention dogs?”). Model steering involves steering a model to exhibit a specified concept (“Make outputs less sycophantic.”). Another negative result involves concept unlearning (Farrell et al., 2024) (“Unlearn knowledge about concepts related to biosecurity.”). In these tasks, concepts are inputs—known beforehand. Now consider the tasks where positive results have been shown. Hypothesis generation involves finding concepts that predict a target variable (“What concepts predict en- gagement of news headlines?”). Biology of LLMs involves finding concepts that LLMs represent when generating text (“What concepts does an LLM represent when doing ad- dition?”). In both tasks, concepts are outputs—unknown beforehand. 1 So even given results showing that SAEs underperform baselines when acting on known concepts, SAEs remain promising and underexplored tools for discovering unknown concepts. By enumerating concepts in an unsupervised man- ner, SAEs allow for the discovery of concepts that fit desired criteria. We outline how SAEs—as a tool for generating unknown concepts—can be used to advance research in (i) ML interpretability, explainability, fairness, auditing, and safety, and (ii) discovery in the social and health sciences. 1When we say “unknown” concept, we mean that the researcher or practitioner cannot specify the particular concept that answers their question. This concept (once revealed to the researcher) may be well understood. For example, a researcher may be very familiar with the concept of “surprise,” but did not know beforehand that this concept predicted an outcome of interest (e.g., number of clicks on a headlines). Furthermore, the concept also may be stored in language model representations; therefore, the primary task is to uncover the right concepts—a task we argue SAEs are well-equipped to do. 1 arXiv:2506.23845v2 [cs.LG] 4 Jul 2026

Position: Use Sparse Autoencoders to Discover Unknowns For example, in ML fairness and auditing, researchers can use SAEs to discover previously unknown concepts that bias model outputs. In the health sciences, researchers can use SAEs to discover previously unknown concepts that predict health outcomes, or to discover spurious correlations in existing predictive models. (On the other hand, we do not intend to suggest that SAEs do not also have potential in the known concepts regime. There are increasingly positive results in this setting compared to baselines (Cywi´nski and Deja, 2025; Arad et al., 2025), especially when considering cost and efficiency (Nguyen et al., 2025). Our paper focuses on establishing how SAEs are particularly useful in the unknown concepts regime, and that negative results in the known concepts regime need not dissuade researchers from exploring the use of SAEs.) Paper structure. Section 2 serves as a primer on SAEs (which can be readily skipped by readers familiar with SAEs). Section 3 shows that negative SAE results per- tain to tasks that act on known concepts. Section 4 surveys recent positive results, showing that these papers use SAEs to discover unknown concepts. Section 5 then explores use cases for SAEs in different research areas. 2. An SAE Primer We offer a brief primer on the SAE architecture, their history, and why and how they are now being used to interpret language models. (Readers familiar with SAEs may skip to Section 3.) Early work on autoencoders. Autoencoders are unsu- pervised neural networks that learn to reconstruct high- dimensional inputs via a series of learned transformations. For a D-dimensional input x, an autoencoder computes: z = encoder(x), (1) ˆx = decoder(z), (2) where encoder(·), decoder(·) are arbitrary neural networks, z is the latent feature representation, and ˆx is the reconstruc- tion. The autoencoder is trained with a mean squared error reconstruction loss, L = ||ˆx − x||2 2. (3) One classic application of autoencoders is compression: by restricting the latent representation z to a dimension size M ≪ D, the autoencoder learns a compressed representa- tion in z which can be used to approximate x (Hinton and Salakhutdinov, 2006). In this setting, z functions similarly to an M -dimensional principal component analysis of x,2 2If the encoder and decoder are linear, PCA minimizes the reconstruction loss (Baldi and Hornik, 1989). in that we wish to explain as much variance as possible in the distribution of x using only M dimensions. Sparse autoencoders. Sparse autoencoders (SAEs) per- form the same reconstruction task, but leverage a different intuition. In an SAE, M can be larger than D, but each indi- vidual z is forced to be sparse—that is, only a small number of its dimensions can be nonzero. This design is motivated by the idea that while an entire dataset (e.g., all text on the Internet, or all images in ImageNet) may span many possible concepts, a single datapoint (a sentence or image) often contains very few. Empirically, enforcing this struc- ture often produces latents in z that capture atomic concepts. For example, early work on SAEs trains directly on input images x (e.g., from MNIST), and z contains interpretable features like edges (Coates and Ng, 2011; Makhzani and Frey, 2014). To improve clarity, we define features and concepts: • A feature is one of many numerical values used to represent an input. In a neural network, a feature is a single dimension of a layer’s output vector. In an SAE specifically, a feature is an activation computed by a single latent neuron. We use the terms feature, activation, neuron, and dimension interchangeably de- pending on context. • A concept is a qualitative characteristic that may or may not be present in a given input. Here, we opera- tionalize concepts via natural language descriptions. • An interpretable feature, then, is a feature whose values correspond to the presence or absence of a single concept. Mathematical formulation of SAEs. One formulation of the sparse autoencoder follows the usual autoencoder for- ward pass, but adds an L1 penalty on z to the loss function (Coates and Ng, 2011): L = ||ˆx − x||2 2 + λ||z||1. (4) A larger λ encourages more zero elements in z. Another approach removes the L1 penalty, but explicitly applies a TopK function to the encoder that zeroes out all but the K largest activations in z (Makhzani and Frey, 2014), where K ≪ M . The full forward pass is given by: z = ReLU(TopK(Wenc(x − bpre) + benc)), ˆx = Wdecz + bdec, where bpre ∈ RD , Wenc ∈ RM ×D , benc ∈ RM , Wdec ∈ RD×M , bdec ∈ RD . 2

Position: Use Sparse Autoencoders to Discover Unknowns Neg. Results: Acting on Known Concepts Pos. Results: Discovering Unknown Concepts Concept detection (Wu et al., 2025; Kantamneni et al., 2025) Hypothesis Generation (Movva et al., 2025b) Is the following name a basketball player? What concepts predict engagement on news headlines? Is the following entity in New York City? What concepts predict partisanship in Congressional speeches? Model steering (Wu et al., 2025) Biology of LLMs (Lindsey et al., 2025) Make the LLM output more sycophantic. What concepts does an LLM represent after writing the first line of a poem? Make the LLM output discuss the Golden Gate Bridge. What concepts does an LLM represent when performing addition? Table 2. Negative SAE results act on known concepts whereas positive SAE results focus on discovering unknown concepts. a text. 2. Model steering (Wu et al., 2025): Steering the outputs of a language model to contain a concept. These are important, widely-studied problems, and under- standing how SAEs perform on them is clarifying. Notice, however, that these tasks each involve first prespecifying a concept and then acting upon it. In other words, in these tasks, concepts are inputs. We now summarize these papers’ findings in greater detail. Concept detection. Kantamneni et al. (2025) curate 113 binary classification tasks on text data, which they use to evaluate concept detection accuracy. For example, one task is to determine whether a given name corresponds to a basketball player. Another task is to determine whether a tweet conveys happy sentiment. For each such task, they fit a classifier to predict the presence of the concept using Gemma-2-9B’s representations of the final token in each text as input. They compare this to a classifier trained on the la- tent representations from a Gemma-2-9B SAE. They further examine class imbalance, data scarcity, and label noise. In each setting, the classifier trained on top of the SAE does not outperform the classifier trained directly from the LM. Wu et al. (2025) follow a similar approach. Starting from a list of 500 concepts, for each concept, they generate syn- thetic texts that either do or do not contain the concept. In addition to logistic regression using Gemma-2-2B representa- tions, they train several other representation-based concept detection methods. They also include methods that do not use representations at all, such as prompting an LLM to identify whether the concept is present in the text, as well as bag-of-words. Four such baselines, including logistic regression and prompting, outperform the SAE. Model steering. Wu et al. (2025) also study model steer- ing. Given a user prompt and a concept, like “where should I visit today?” and “Golden Gate Bridge,” they evaluate whether the model can generate a response that is fluent, relates to the prompt, and includes the concept. An LLM judge scores each attribute. To steer with an SAE, they iden- tify the SAE feature that is most predictive of the concept’s presence, and they generate a response after increasing the value of this feature. Non-SAE methods include editing ac- tivations with a steering vector (Marks and Tegmark, 2024), finetuning on responses containing the concept, or simply prompting to include the concept in the response. Prompting and finetuning both outperform SAE-based steering. What explains these negative results? We suggest some reasons why SAEs underperform baselines on these tasks. For concept detection, recall that SAEs are trained to re- construct the LM token representations. A reconstruction encodes strictly less information about a token than the orig- inal LM representation. It follows that, compared to the original representation, there is less information available in the SAE representation to predict the presence of a con- cept. For model steering, prompting performs well because LLMs are finetuned to be adept at instruction-following, and including a concept in a response falls well within this paradigm. The empirical results from both papers underscore an intu- ition that there are many natural methods besides SAEs to act on known concepts. (Though, methodological innova- tions may make SAEs more competitive at these tasks as well (Arad et al., 2025).) Crucially, however, these baselines are less equipped to perform another simple task: to enumer- ate a list of unknown concepts that satisfy some objective. This, as we show in the next section, forms the basis for tasks on which SAEs have a comparative advantage. 5

Position: Use Sparse Autoencoders to Discover Unknowns 4. Positive results: Discovering Unknown Concepts We now describe two positive results using SAEs5 (Movva et al., 2025b; Lindsey et al., 2025), which focus on the following tasks: 1. Hypothesis generation (Movva et al., 2025b): Identify- ing open-ended natural language concepts that predict a target variable. 2. Explaining language model outputs (“Biology of LLMs”) (Lindsey et al., 2025): Describing the con- cepts a language model uses to perform various tasks (e.g., poem completion or addition). We claim that these tasks are examples of discovering un- known concepts. To explain this, we summarize their find- ings in greater detail. Hypothesis generation. Movva et al. (2025b) study tasks where a large dataset of texts is annotated with a target vari- able, and the goal is to understand what concepts in the text predict the target. For example, one such dataset consists of news headlines and numerical engagement levels. While a traditional analysis of such a dataset may be hypothesis- driven (e.g., Robertson et al. (2023) study how negativity affects engagement), here, the task is to extract concepts with no prior specification. Such an approach can surface unknown concepts, which can be used as hypotheses for further study. They (1) train an SAE on dense text embeddings; (2) select SAE features that predict the target; and (3) run autointer- pretation to explain the selected features, which become hypotheses (i.e., “headlines that contain {concept} receive more engagement”). They find that the resulting hypotheses outperform those generated without an SAE, either by skip- ping step 1 and selecting features from text embeddings, or by using a different pipeline altogether (like LLM prompt- ing, topic modeling, or n-grams). Compared to these base- lines, it produces more statistically significant hypotheses, and raters identify these hypotheses as more helpful. Mechanistic explanation of LM outputs. Lindsey et al. (2025) explain how language models generate text that com- pletes a task. For example, prompted to write a rhyming couplet, an LM generates “He saw a carrot and had to grab it (line 1) / His hunger was like a starving rabbit (line 2).” They ask: what is the internal mechanism through which the LM rhymes the end of line 2 with the end of line 1? They find that, immediately after generating line 1, which 5Note that Lindsey et al. (2025) use sparse coding methods that differ slightly from SAEs (see note at end of §2). ends in “it,” there is an active SAE neuron corresponding to “words rhyming with ‘it’,” as well as a neuron for “rabbit.” The SAE, therefore, suggests that the LM plans line 2 im- mediately after generating line 1, rather than improvising a rhyming final token only after generating the first part of line 2. They confirm this with further intervention experi- ments. In another case, they look at how a model computes “36+59” in natural language. They find active neurons for “units digit 5,” and “addition problems of ∼40 plus ∼50,” which combine to produce “95.” These specific routes of task completion are difficult to forecast, underscoring how this analysis requires discovering unknown concepts. What explains these positive results? In hypothesis gen- eration, the goal is to find concepts that predict a target variable; in explaining LM behaviors, it is to find concepts that are active when completing a task. In both cases, our hope is to discover concepts that satisfy a property of inter- est, out of intractably many possibilities. SAEs produce a set of concepts that is both tractable and expressive, after which selecting a subset of concepts that are relevant to the task is straightforward. An empirical strength of the SAE is its precise concepts. If the rabbit feature instead activated on all animals, it would be difficult to answer whether the model improvises or plans rhymes. Also note that after identifying concepts of interest, it is possible to computationally validate whether they satisfy the desired property. In particular, once the concept has been identified, it is possible to directly annotate the data for the concept. These annotations can then be used instead of the activations. At this point, the activations are no longer needed. It is easy to evaluate whether a hypothesized head- line concept indeed correlates with engagement, or whether a hypothesized LLM addition feature is active during addi- tion. Because of this falsifiability, even if an SAE feature is unreliable (e.g., not all headlines that contain a concept activate the corresponding feature), it is possible to catch these issues downstream. In contrast, unreliability directly harms concept detection and steering. Practical examples of concept discovery. This pattern— enumerating concepts and selecting those that satisfy a crite- rion of interest—has already been useful in several settings beyond the two prior examples. Tjuatja and Neubig (2025) and Jiang et al. (2025) use SAEs to generate features of LM outputs, and they select features that distinguish different model versions. The former work finds, for example, that OLMo2-13B is better than OLMo2-7B at correctly using ar- chaic spellings in historical texts (e.g., “wood” for “would”). Movva et al. (2025a) use SAEs to identify which features of LM responses predict human judgments in RLHF data, revealing unsafe and surprising preferences (e.g., high win- rates for toxic responses, and low win-rates for responses 6

Position: Use Sparse Autoencoders to Discover Unknowns emphasizing environmental sustainability). A similar play- book has been promising for biological foundation models (Simon and Zou, 2025; Brixi et al., 2025); for example, Adams et al. (2025) train SAEs on protein LMs and identify amino acid features that predict properties like thermostabil- ity, yielding hypotheses for how structure encodes function. While each of these research questions (“how do LM out- puts differ”, “which protein structures are most stable”) could by studied be pre-specifying concepts and testing them, SAEs provide the opportunity to discover unknown concepts. Therefore, it is situations where we lack existing theories, or where we wish to expand upon them (Mul- lainathan and Rambachan, 2025), where SAEs excel. 5. Call to Action: Future Use Cases for SAEs Having conceptualized where SAEs are useful (discovering unknown concepts), we outline research areas where such a capability can be useful. In particular, while initial ex- citement about SAEs was shared primarily by researchers in mechanistic interpretability (Sharkey et al., 2025), we believe that clarifying the comparative advantage of SAEs reveals a significantly broader set of uses. The use cases we outline focus on the ability of SAEs to discover unknown concepts. Broadly, these use cases fall under two categories: using SAEs to understand language models (in ML fairness, in- terpretability, explainability, auditing, and safety) and using SAEs to understand the world (e.g., in the social sciences and healthcare). We summarize these potential use cases of SAEs for different research problems in Table 3. ML fairness, interpretability, explainability, auditing, and safety. Each of these areas aim to understand and build models with desiderata beyond accuracy in mind. Here, we see significant opportunity for SAEs. For example, SAEs can be used to identify natural language concepts that can explain black box model behavior (Lakkaraju et al., 2019). Then, by identifying the concepts that are used, it is possible to build models that are inherently interpretable (Rudin, 2019), and that incorporate only features that we want (e.g., that are considered fair, avoid spurious correla- tions, etc). SAEs are particularly valuable for studying models with unstructured text inputs or outputs. For example, whereas existing work documents how specific demographic features affect LLM-based decision-making (e.g., in hiring (Gaebler et al., 2024)), it is possible to use SAEs to uncover a wider range of input features that may affect LLM-generated deci- sions. Similarly, while demographic information has been demonstrated to affect LLM outputs in specific ways, it is possible to use SAEs to uncover a wider range of output features that are affected by variations in inputs. Health and social sciences. A wide variety of disciplines (e.g., sociology, economics, healthcare) have sought to lever- age large text datasets. This has led to prominent work de- veloping and applying methods for “text as data” (Grimmer, 2010; Gentzkow et al., 2019). These methods often attempt to discover interpretable patterns in text data—for exam- ple, quantifying changes in the language used to discuss immigrants (Garg et al., 2018; Card et al., 2022), or identi- fying features of clinical notes that predict health outcomes (Harrigian et al., 2023; Hsu et al., 2025). Existing methods automate these tasks through simple text features such as keywords or n-grams, or through topic models. These meth- ods are limited by the expressivity of these features: topic models and keywords do not precisely capture the range of concepts present in text. In contrast, text embeddings can better capture the information present in text, but are unin- terpretable. SAEs essentially convert uninterpretable text embeddings into interpretable text embeddings, enabling their use for the same applications as previous keyword or topic model methods—i.e., discovering concepts that reveal patterns in text—but potentially with significantly higher quality. SAEs provide a way to revisit important problems studied using text as data, but with the capabilities of mod- ern language models. Stepping back, SAEs are a promising tool for bridging the “prediction-explanation gap.” There are many settings in which text data have been shown to enable much greater predictive accuracy than existing human-specified features. While developing methods to quantify or improve predictive accuracy may be of independent interest, a growing line of work has suggested the need to bridge the gap between prediction and explanation (Hofman et al., 2017; 2021). Traditionally, scientific disciplines have sought to explain phenomena, rather than only predict outcomes. For exam- ple, Fudenberg et al. (2022) and Ludwig and Mullainathan (2024) each show gaps between predictive accuracy of ML models that take in all available features and models that take in existing human-specified features. This gap sug- gests that existing theories are incomplete, leading to work that has sought to build automated approaches for closing this gap: discovering interpretable features that are predic- tive. SAEs are a promising tool for this task (Movva et al., 2025b). SAEs can help close the prediction-explanation gap by converting black box representations into interpretable representations. These interpretable representations both capture much of the predictive power of the black box rep- resentations, while also enabling us to make predictions in terms of natural language concepts. For example, one important motivation for closing this gap is spurious correlation: it is well established that strong pre- dictive performance can be misleading in ML-based science 7