Sparse probes on sufficiently good sparse autoencoder latents are less likely than dense probes to overfit spurious correlations.
Structured and assessed
First pass. Read the GDM negative-results post and the Kantamneni et al. (2502.16681) TeX. Added two arguments: the inductive-bias argument (for, rated slight) and head-to-head OOD comparisons (against, rated likely). Linked 0cfd9ad2 as contradicting evidence and minted the crux b2d08722 (spurious cues equally represented by latents). Assessed doubtful, confidence 0.6, credence 0.3, importance 0.4. Not done for lack of budget: written forms for the arguments, placing the unplaced passage (it belongs on the inductive-bias argument as criticizes), and correcting the extractor instance's stance (the post states the hypothesis and then doubts it, so mentions or denies at reduced confidence is right).
Assessed Doubtful
verdict confidence 0.60 · credence 0.30
The claim states a hypothesis that motivated several 2024 to 2025 studies: a probe restricted to a few interpretable sparse-autoencoder (SAE) latents should latch onto the target concept and ignore minor spurious cues that a dense linear probe can exploit. The tests run so far, all on Gemma Scope and Llama Scope SAEs, found the opposite. In Google DeepMind's harmful-intent experiment on Gemma 2 9B, k-sparse SAE probes (about k=20) fit the training and in-distribution test sets but transferred worse to held-out jailbreaks and benign transformations than a dense probe, which scored near-perfect AUROC out of distribution. Kantamneni et al. (2025), across 113 datasets, found that SAE probes underperform tuned logistic regression under covariate shift and label noise, and that pruning latents judged spurious recovered only a small part of the out-of-distribution drop. The evidence is indirect in one important respect: no study located here induced a controlled spurious correlation, such as a cue perfectly confounded with the label, and compared worst-group or decorrelated accuracy for sparse SAE probes against dense probes. The DeepMind team removed the spurious formatting cues it found before training, which may favour the dense probe; that cleanup used SAE latents, though Kantamneni et al. show simple non-SAE methods find the same cues. The qualifier "sufficiently good" also limits refutation, since every test used current SAEs that are incomplete and lose task-relevant information. Even for an ideal SAE, the argument needs the target concept to be more cleanly isolated in latents than the spurious cue, and it is plausible that spurious cues are just as well represented. A controlled confounded-label benchmark with an ideal or synthetic dictionary would settle it.
Claim entered the graph