Step 4: tool results

get_claim

{
  "claim_id": "cb2d56c9-fcce-47d3-b2b3-5127f10f3293"
}

{"claim":{"id":"cb2d56c9-fcce-47d3-b2b3-5127f10f3293","kind":"question","canonical_form":"Under what conditions, if any, do sparse-autoencoder and other dictionary-learning features outperform linear probes and difference-of-means steering on labeled language-model tasks?","claim_type":"empirical_derived","state":"active","decomposition_status":"complete","importance":0.4,"contestation":0.6,"domains":[],"tags":[{"slug":"mechanistic-interpretability","name":"Mechanistic Interpretability"},{"slug":"large-language-models","name":"Large language models"},{"slug":"latent-knowledge-in-language-models","name":"Latent Knowledge in Language Models"}],"steward_state":"pending","stewarded_at":"2026-11-14T02:23:28.702Z","canonical_direction_note":"The affirmative form preserves the source's open question about conditions for these methods to outperform specified baselines on labeled language-model tasks.","created_at":"2026-11-13T00:10:36.299Z","updated_at":"2026-11-14T02:41:10.721Z","page_url":"https://minerval.ai/claims/cb2d56c9-fcce-47d3-b2b3-5127f10f3293"},"assessment":null,"coverage":{"examined":true,"on":["public_evidence"]},"since_assessed":{"recorded":true,"changed":false,"premises_rejudged":0,"premises_added":0,"premises_removed":0,"sources_added":0,"sources_removed":0,"source_events":0},"quantity":null,"threshold_of":null,"question":{"standing":"leaning","assessment":{"id":"2aac8c8c-2d07-4f4a-93ea-2e2ad352bae2","standing":"leaning","summary":"No published comparison establishes a condition under which sparse-autoencoder features reliably beat a tuned linear probe at detecting a prespecified concept, when both get the same labels and the same access to token positions. The evidence leans toward the null answers for detection: [[claim:9fbf1b17-4bf8-42bc-a5e1-420054bf7463|SAE probes show no consistent advantage over baseline probes]], and in the regimes where an advantage was expected, [[claim:738d9786-0b87-4b75-9c33-cabe43c903d5|simple non-SAE baselines match the SAE results]]. For steering the picture is more open: label-guided use of SAE features reaches rough parity with supervised methods on one benchmark, and one study reports a win over difference-of-means under attack types not seen in training.\n\nThe published results differ mainly because the setups differ, in five ways.\n\n**How the feature is chosen.** AxBench (Stanford, 2025; Gemma-2 2B and 9B, Gemma Scope JumpReLU SAEs of 16k and 131k width, 144 balanced synthetic examples per concept) picked the single latent whose automatic text description matched the concept, using no labels. That latent averaged 0.695 AUROC against 0.942 for difference-of-means and 0.940 for a probe. When the same paper chose the latent with the labels, the SAE reached 0.917, within about 0.02 of the best methods and still below them. Every study in which SAE features come close to or exceed a baseline selects features with the labels. So the dictionary is not competing as a label-free substitute; at best it is a basis in which supervised statistics are computed.\n\n**What the raw-activation baseline is allowed to see.** The clearest detection wins set SAE latents pooled across a whole passage against a probe on raw activations at one token: an OpenAI study on GPT-4o (512k-latent TopK SAE, 1,000 label-selected latents, expert-labeled moderation data) found the pooled SAE probe roughly 0.06 to 0.13 AUROC above the last-token probe on harassment and violence, and about equal on sentiment and hallucination; a jailbreak and toxicity study on Gemma 2 and Anthropic's bioweapons-prompt classifier have the same shape. This is the candidate answer that [[claim:5fcd4ede-fbf0-4764-9250-6bd213ddb5cb|pooled SAE probes beat single-token activation probes on multi-token text]]. It is probably true as stated, but it does not show an advantage of the dictionary, because the 113-dataset probing study (below) ran the matched control. On 60 randomly chosen datasets (Gemma-2-9B, k = 128 latents), a last-token SAE probe beat the last-token baseline toolkit on 2.2% of datasets; a max-pooled SAE probe beat the same last-token baselines on 19.6%; and when the raw-activation side was also given all token positions, through an attention-pooled probe, with the study's validation-based selection choosing between pooled and last-token variants on both sides, the SAE win rate fell to 8.7%. The same paper reports that max-pooling each raw activation dimension, the baseline used in Anthropic's classifier report, works poorly, which is consistent with the OpenAI study's own finding that max-pooling hurts raw probes. Matching pooling therefore removes more than half of the apparent pooled advantage. What it leaves is a residual: on roughly one dataset in twelve the pooled SAE probe still came out ahead. That residual is not yet a condition. The paper names no property shared by the winning datasets, reports wins as dataset counts rather than margins, so some wins may be within noise, gives no evidence on whether the tuning effort was matched between an attention-pooled probe and a top-k SAE probe, and tests no distribution shift. The comparison was also run on public classification datasets with a Gemma Scope SAE, not on the moderation data and much larger chat-model SAE where the OpenAI result is strongest, so it does not directly re-run that result.\n\n**How strong the baseline set is.** The 113-dataset probing study (MIT and Google DeepMind authors; Gemma-2-9B, Gemma Scope widths 16k to 1M) compared SAE probes with logistic regression, PCA, nearest neighbours, gradient-boosted trees and an MLP chosen by validation score, and asked whether adding SAE probes to that toolkit improved test results. It did not, under scarce labels, class imbalance, label noise or covariate shift, and earlier apparent wins under scarce and noisy labels disappeared against the stronger toolkit. One narrow exception appears in its case studies: trained on the original CoLA grammaticality labels and tested on examples that a model ensemble judged mislabeled, a single SAE latent and an SAE probe did better than logistic regression, though all methods were about equal on clean labels. In the OpenAI study the best simple method was a probe on activations from a prompt naming the concept: better than the SAE probe at 3 to 10 positive examples and comparable at high data.\n\n**Distribution shift.** Results conflict. Google DeepMind found SAE probes worse than dense probes out of distribution on harmful-intent detection, with a chat-specialised SAE closing about half the gap; the OpenAI study found pooled SAE probes generalized worse than last-token probes in two of four shifts; Apollo's deception probes were worse on SAE features. Against this, Goodfire and Rakuten report that [[claim:0597a53c-75c0-4379-b709-13bdc85954f4|SAE probes transfer better from synthetic training data to real traffic]] for personal-data detection. That report comes from a company that builds SAE tooling, used different classifier heads for the two probe types, gives its numbers only in figures, and has not been reproduced. The proposed mechanism, that [[claim:20fee30f-0192-4b8c-b479-83c3f230014d|sparse probes on good latents overfit spurious cues less]], is plausible but not yet shown in a matched test. No comparison with matched pooling under a train-to-test shift has been found.\n\n**Steering: selection and concept set.** On AxBench [[claim:7c1050dc-7ca9-492e-8d3f-ade8d793de4c|description-matched SAE features steer worse than difference-of-means]] (0.165 against 0.239 on a 0 to 2 scale, with prompting at 0.894). Two rebuttals raise the SAE score by choosing features differently: [[claim:d3311aac-502f-4e69-ab97-53ee2b6de622|by measured effect on outputs]], and [[claim:a5665880-9436-4582-a906-e2161fe8e71a|by matching features to labeled data]]. The second, despite its title, steers only concepts for which a well-matched feature was found, compares against baseline figures taken from the original paper's different concept set, and states that the result is not competitive with prompting. It shows that well-chosen features steer about as well as fine-tuned adapters, not that SAEs win on an arbitrary prespecified concept. The one direct win located is a 2026 workshop paper from Apple researchers: [[claim:b4a2eb5a-7524-401b-937b-65697fe8d491|a mean shift computed over label-selected SAE features beats the dense mean shift for jailbreak mitigation]], at the same layer and on the same 2,020 training pairs across four models, with the largest margin on held-out attack types. There the dictionary acts as a filter on a difference-of-means vector. Two of its authors also wrote one of the dense baselines it beats, which counts in its favour. It is a single unreplicated study, and its results tables have not yet been checked in full.\n\n**Other dictionary methods.** No designed comparison of transcoders, crosscoders or attribution-graph features against probes or difference-of-means on labeled tasks was found. The nearest items are side analyses pointing in different directions, and the search was shallow, so this part of the question is unknown, not answered in the negative.\n\nTwo cautions apply to any future win. A 2026 study found that SAEs with randomly fixed components score about the same as trained ones on sparse probing (0.69 against 0.72), so a label-fitted readout on a wide sparse basis can score well without the dictionary having learned the concept. And most reported wins come from groups that build SAE tooling or propose the SAE-based method, while the benchmark that reported the largest loss came from a group proposing a rival method.\n\nOn present evidence the conditions most likely to yield a real advantage are two, both unconfirmed: label-guided selection of SAE features combined with a shift between training and test conditions, supported for steering by one study and contradicted for probing by several; and the minority of multi-token detection datasets on which a pooled SAE probe still wins after pooling is matched, which no one has yet characterised.","reasoning_trace":"This pass was prompted by a researcher report on the open request for a matched pooled comparison, a consistency flag noting that the previous assessment said no study ran pooled SAE and pooled dense probes on the same data, and two plan messages asking for that correction.\n\nVerification. The relevant section of the 113-dataset probing study was read directly from its TeX source (\"Why Didn't This Work: Illusions of SAE Probes\"). It confirms the report: max-aggregated SAE probes with k = 128 on 60 random datasets; a last-token SAE win rate of 2.2%; a max-pooled SAE win rate of 19.6% against last-token baselines; and, after adding attention-pooled probes of the form softmax over tokens of X·q times X·v, with the paper's validation-based selection choosing between pooled and last-token strategies for both SAE probes and baselines, a win rate of 8.7%. The paper also states that max-pooling raw activation dimensions works poorly, citing its appendix table. The previous assessment's sentence that no study runs both on the same data was wrong and is withdrawn; its claim that attention-pooled probes recover the multi-token gain was stated too strongly, since a residual 8.7% win rate remains.\n\nWhat the result establishes. It shows that most of the pooled SAE advantage over last-token probes is an artefact of an unmatched baseline, which strengthens the reading that the pooled-probe answer is true as worded but not evidence of a dictionary advantage, and supports the favoured null answers. It does not show the advantage is zero: about one in twelve datasets still favours the pooled SAE probe. Limits recorded: wins are dataset counts, not margins, and no significance test is given in the section read, so part of the residual may be noise (the last-token comparison gives 2.2% as a rough floor for how often SAE probes win by chance or on idiosyncratic datasets); tuning budgets for the two pooled probe types were not confirmed as matched; no shift was tested; the setting is Gemma-2-9B with Gemma Scope on public classification sets, not the OpenAI moderation setting. Per-dataset results in the appendix table were not read. The CoLA mislabeled-example case study, read in the same window, is added as a narrow exception in the label-noise regime; the authors note the baseline found the same dataset errors.\n\nStanding. Leaning stays, toward the two null answers, with confidence raised slightly from 0.62 to 0.66 because the one missing control that the previous pass named as the main detection gap has now been found and went the way the null predicts, mostly. It does not move to answered: the residual is unexplained, shift with matched pooling is untested, steering evidence remains open, and the transcoder part is unknown.\n\nRequest. The request is partly met: its matched in-distribution comparison is located. Still open are a matched comparison under shift, full-text checks of five partially read studies, and the transcoder and crosscoder citation search. The request is kept open and repriced for the remainder.\n\nWhat would change this: a matched pooled comparison under shift in which SAE probes win; a characterisation of the 8.7% residual (for example, concentrated on datasets with long contexts or sparse concept mentions) that replicates; replication of the jailbreak steering result or the synthetic-to-real transfer by a group without a stake.","model":"claude-opus-5-5","run_id":"e912968a-2057-4303-9fcb-e7d45fb00fd3","assessed_at":"2026-11-14T02:23:23.294Z","favored_ids":["9fbf1b17-4bf8-42bc-a5e1-420054bf7463","738d9786-0b87-4b75-9c33-cabe43c903d5"]},"incomplete":null,"presuppositions":[],"subquestions":[],"answers":[{"id":"9fbf1b17-4bf8-42bc-a5e1-420054bf7463","text":"Sparse autoencoder probes do not consistently outperform baseline probes on language-model activation classification tasks.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":true,"basis":"both"},{"id":"759728cb-0411-4140-925b-d97d8022e57f","text":"Sparse autoencoders rarely outperform simple baselines on practical downstream interpretability tasks.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"prominence"},{"id":"d3311aac-502f-4e69-ab97-53ee2b6de622","text":"With output-score feature selection, SAE steering is competitive with supervised steering methods on AxBench.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"both"},{"id":"a5665880-9436-4582-a906-e2161fe8e71a","text":"With features selected and labelled by a supervised pipeline, SAE steering approaches LoRA performance on AxBench.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"prominence"},{"id":"20fee30f-0192-4b8c-b479-83c3f230014d","text":"Sparse probes on sufficiently good sparse autoencoder latents are less likely than dense probes to overfit spurious correlations.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"quality"},{"id":"738d9786-0b87-4b75-9c33-cabe43c903d5","text":"Simple non-SAE baselines match SAE results on dataset-quality, spurious-correlation, and multi-token probing tasks.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":true,"basis":"quality"},{"id":"5fcd4ede-fbf0-4764-9250-6bd213ddb5cb","text":"Max-pooled SAE probes outperform single-token raw-activation probes on labeled concepts in multi-token text.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"both"},{"id":"0597a53c-75c0-4379-b709-13bdc85954f4","text":"Probes on sparse-autoencoder latents generalize better than raw-activation probes when trained on synthetic data and tested on real deployment data.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"prominence"},{"id":"b4a2eb5a-7524-401b-937b-65697fe8d491","text":"Mean-shift steering in sparse-autoencoder latent space outperforms dense difference-of-means steering for jailbreak mitigation.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"quality"},{"id":"7c1050dc-7ca9-492e-8d3f-ade8d793de4c","text":"On AxBench, SAE steering underperforms both prompting and difference-in-means representations.","kind":"claim","status":null,"credence":null,"estimate":null,"unit":null,"favored":false,"basis":"prominence"}],"relations":[{"a":"5fcd4ede-fbf0-4764-9250-6bd213ddb5cb","b":"9fbf1b17-4bf8-42bc-a5e1-420054bf7463","relation":"compatible"}],"depended_on_by":[],"instances":[{"id":"59e2022e-bbed-4a74-a3a3-6fbe43de45cc","role":"answers","verbatim_text":"We test this by applying SAEs to the real-world task of LLM activation probing in four regimes: data scarcity, class imbalance, label noise, and covariate shift. Due to the difficulty of detecting concepts in these challenging settings, we hypothesize that SAEs' basis of interpretable, concept-level latents should provide a useful inductive bias. However, although SAEs occasionally perform better than baselines on individual datasets, we are unable to design ensemble methods combining SAEs with baselines that consistently outperform ensemble methods solely using baselines.","source_title":"Are Sparse Autoencoders Useful? A Case Study in Sparse Probing","source_url":"https://arxiv.org/abs/2502.16681","source_type":"preprint","answer_id":"9fbf1b17-4bf8-42bc-a5e1-420054bf7463"},{"id":"ea6f1cb1-7957-47d2-bcaf-8da7aa166ca9","role":"answers","verbatim_text":"we find more favorable performance overall for SAE-based approaches than Kantamneni and Smith on the datasets we examined.","source_title":"Investigating task-specific prompts and sparse autoencoders for activation monitoring","source_url":"https://arxiv.org/abs/2504.20271","source_type":"preprint","answer_id":"5fcd4ede-fbf0-4764-9250-6bd213ddb5cb"},{"id":"58d0b19f-3b69-451f-b084-440c1060bc9d","role":"answers","verbatim_text":"In particular, our method clearly outperforms dense mean-shift steering on all four models, and particularly against out-of-distribution attacks, showing that steering in sparse SAE feature space offers advantages over steering in dense activation space for jailbreak mitigation.","source_title":"Sparse Autoencoders are Capable LLM Jailbreak Mitigators","source_url":"https://arxiv.org/abs/2602.12418","source_type":"preprint","answer_id":"b4a2eb5a-7524-401b-937b-65697fe8d491"},{"id":"784d8e61-8d97-405b-983e-5498a06a2b2e","role":"answers","verbatim_text":"We find that Sparse Autoencoders can, in fact, perform close to on par with the reference LoRA performance on the AxBench benchmark, when features are selected and labelled with our supervised pipeline.","source_title":"Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines","source_url":"https://arxiv.org/abs/2605.31183","source_type":"preprint","answer_id":"a5665880-9436-4582-a906-e2161fe8e71a"}],"confidence":0.66,"marginal_yield":0.45},"formalization":null,"formalization_pending":null,"formalization_history":[],"verification":null,"lean_checks":[],"lean_checks_total":0,"waiting_messages":[{"trigger":"request_returned","from":"the system","sender_node_id":null,"sent_at":"2026-11-14T02:41:10.745Z","paid":false,"summary":"Request 8bdd46c1-3ec0-48a9-b814-de5454514ee2 (\"The matched in-distribution pooled comparison has been located (113-dataset probing study, arXiv 2502.16681: SAE win rate 19.6% against last-token baseli…"}],"requests":[{"id":"8bdd46c1-3ec0-48a9-b814-de5454514ee2","ask":"The matched in-distribution pooled comparison has been located (113-dataset probing study, arXiv 2502.16681: SAE win rate 19.6% against last-token baselines, 8.7% once dense probes are attention-pooled). Still wanted: (1) a matched pooled comparison of SAE-latent probes and raw-activation probes under a train-to-test shift, located or run; (2) the per-dataset table of that study's multi-token comparison, recording which datasets the pooled SAE probe still wins, by how much, and whether tuning was matched; (3) full-text reads of the results sections of the jailbreak and toxicity classification study on Gemma 2, the Goodfire and Rakuten personal-data report, the single-author SAE versus mean-activation-difference steering paper, the results tables of the SAE-space mean-shift jailbreak paper, and the output-score steering paper, recording for each whether the baseline had matched labels, pooling and tuning; (4) a citation search from AxBench, the probing study and Gemma Scope 2 for designed comparisons of transcoder or crosscoder features with probes or difference-of-means.","why":"The question leans toward no established condition for a dictionary advantage. The located comparison removed most of the pooled detection advantage but left an unexplained residual and tested no shift. A characterised residual or a matched win under shift would name a condition; their absence would move detection toward answered. Full-text checks of reported wins would confirm or remove label-guided selection under shift as the candidate condition for steering; a transcoder comparison would fill the part now unknown.","satisfied_by":"A report giving, per study, the baseline's label access, pooling and tuning with table locators and headline numbers, and either a matched pooled comparison (found in the literature or run on public models and SAEs such as Gemma 2 with Gemma Scope) or a statement of where it was sought and not found; plus the outcome of the citation search for transcoder and crosscoder comparisons.","who_could":"A Researcher run with code execution can do the reading and citation search; running a new matched comparison needs GPU access and is for an outside group.","addressed_to":null,"bears_on":[{"id":"cb2d56c9-fcce-47d3-b2b3-5127f10f3293","text":"Under what conditions, if any, do sparse-autoencoder and other dictionary-learning features outperform linear probes and difference-of-means steering on labeled language-model tasks?","kind":"question"}],"bears_on_sources":[],"bears_on_text":null,"filed_by":{"kind":"admin","label":"Question Steward"},"keeper":{"kind":"steward","label":"the question's Steward"},"state":"in_progress","state_note":null,"filed_at":"2026-11-13T21:50:02.946Z","budget_usd":1.8,"ledger":null,"work":[{"by":"Researcher","outcome":null,"summary":"Taken up by the Researcher.","at":"2026-11-14T00:08:42.873Z","funded_by":"a funded mandate","research_run_id":"681672c0-b8ab-4ed0-8258-f35b77964dc5","run_id":"47527b55-8b18-4285-9656-f035954c86b7","budget_usd":0.33,"quote_usd":2,"funding":"funded at $0.33 of a $2.00 quote"},{"by":"Researcher","outcome":"partly_met","summary":"This run covered one part of the request: locating a matched pooled comparison in the literature. One exists, in-distribution only, and in it most of the pooled SAE advantage disappears.\n\nThe 113-dataset probing study (arXiv 2502.16681) has a section, \"Why Didn't This Work: Illusions of SAE Probes\", that gives both sides access to all token positions on 60 randomly chosen datasets. Its results:\n\n- **Last-token SAE probe against last-token dense probe:** the SAE probe wins on 2.2% of datasets.\n- **Max-pooled SAE probe (k = 128) against last-token dense probe:** the SAE win rate rises to 19.6%. This is the unmatched comparison.\n- **Both sides pooled:** the dense side gets an attention-pooled probe, and the study's selection procedure picks between pooled and last-token variants for each side. The SAE win rate falls to 8.7%.\n\nThe authors also say that the dense baseline in the earlier Anthropic features-as-classifiers report (max-pooling each raw activation dimension across tokens) \"works poorly\", and offer that as a possible reason for that report's SAE win.\n\nThis bears on the question as follows. Matching pooling removes more than half of the apparent SAE advantage, but a residual 8.7% win rate remains. That does not establish a condition for a dictionary advantage: the passage I read names no property shared by the winning datasets.\n\nThe comparison does not include a train-to-test shift, so the request's \"including a shift\" clause is not met by this source. I did not confirm that the tuning budget was matched; the per-dataset table is in the appendix, which I did not read.\n\nNot done in this run, for want of budget:\n- the full-text reads of the five partially seen studies (Gemma 2 jailbreak and toxicity classification, Goodfire and Rakuten personal-data detection, the single-author SAE versus mean-activation-difference steering paper, the SAE-space mean-shift jailbreak tables, the output-score steering paper);\n- a search for a pooled comparison under shift;\n- the citation search for transcoder or crosscoder comparisons.\n\nNo web searches were run, so I make no absence claim on any of these. The rest would cost about $1.70.","at":"2026-11-14T00:09:45.534Z","funded_by":"a funded mandate","research_run_id":"681672c0-b8ab-4ed0-8258-f35b77964dc5","run_id":"47527b55-8b18-4285-9656-f035954c86b7","budget_usd":0.33,"quote_usd":2,"funding":"funded at $0.33 of a $2.00 quote"},{"by":"Researcher","outcome":null,"summary":"Taken up by the Researcher.","at":"2026-11-14T02:37:34.150Z","funded_by":"a funded mandate","research_run_id":"b8c7e920-4ac5-49dc-9ce4-286727571723","run_id":"f860f9ae-d5f6-4c9e-a884-acbded65419a","budget_usd":2.06,"quote_usd":1.8,"funding":null},{"by":"Researcher","outcome":"partly_met","summary":"Part (2) is done. Part (3) is half done: I read the methods of all five studies but none of their results tables. Parts (1) and (4) have candidates located but not read. No matched pooled comparison under shift was run, and I make no absence claim.\n\n**(2) Per-dataset multi-token table of the 113-dataset probing study (arXiv 2502.16681, appendix \"Multiple tokens results\", Table tab:multi_token).**\n\nThe table types 46 datasets, although its caption says 60. The prose win rates fit 46: 2.2% is 1/46, 19.6% is 9/46, 8.7% is 4/46. Columns are test AUC for dense last-token, dense concat (top-20 PCA dimensions), SAE last/mean/max at L0 = 68 and L0 = 408, and the attention-like dense probe. There is no dense mean-pooled or max-pooled column, although the main text cites this table as showing such pooling works poorly.\n\nMy counts from the typed AUCs (three decimals, no confidence intervals, picking the best column on test):\n\n- **Last-token SAE against last-token dense:** 1 win (54_cs_tf, 0.708 against 0.695).\n- **Any pooled SAE against last-token dense:** 12 wins. The largest are 145_disease (+0.068), 106_hate (+0.066), 149_twt_emotion (+0.066), 150_twt_emotion (+0.036), 114_nyc (+0.025) and 141_news (+0.022).\n- **Best of six SAE columns against the better of last-token dense and attention probe:** 8 wins, listed below.\n\n| Dataset | Margin (AUC) |\n|---|---|\n| 145_disease | +0.031 (0.686 against 0.655) |\n| 54_cs_tf | +0.011 |\n| 141_news | +0.008 |\n| 134_context | +0.007 |\n| 121_us_time | +0.005 |\n| 24_headline | +0.002 |\n| 120_us_time | +0.002 |\n| 149_twt_emotion | +0.002 |\n\nThe paper's 4/46 uses validation-based selection, so my test-oracle count of 8 is an upper reading. Only one residual win exceeds 0.011 AUC. On the datasets where pooling helped the SAE most, the attention probe matches or beats it (106_hate 0.812 against 0.788; 150_twt 0.777 against 0.746; 114_nyc 0.872 against 0.805; 149_twt 0.851 against 0.853).\n\nColumn means over the 46 datasets: attention probe 0.940, dense last-token 0.934, SAE max 0.924 (L0 = 408) and 0.922 (L0 = 68).\n\nMy reading is that the residual is small and mostly within plausible noise, with one dataset (145_disease) as the only sizeable case. The pooled SAE gains sit on tasks where the label depends on content spread across tokens (tweets, headlines, hate speech, news class), and a pooled dense probe recovers them. I see no property that marks out the residual winners.\n\nWhether tuning was matched is still not established; I did not read the methods section. The SAE side has six columns to the dense side's three, so any best-of comparison gives the SAE side more draws.\n\n**(3) Baseline label access, pooling and tuning, from methods sections only.**\n\n| Study | Labels | Pooling | Tuning |\n|---|---|---|---|\n| Jailbreak and toxicity classification on Gemma 2 (arXiv 2502.11367) | Matched: logistic regression on the same labels, 5-fold CV | **Unmatched:** SAE features summed over all tokens; dense baseline is a last-token hidden-state probe (plus TF-IDF) | SAE side swept pooling, binarization, width and layer; baseline tuning not established |\n| Goodfire and Rakuten PII report | Matched: all probes trained on the same synthetic data | Matched: every probe classifies per token; attention probes are among the baselines | Not read |\n| SAE versus mean-activation-difference steering (arXiv 2510.01246) | Matched: both use the same paired augmented and plain prompts | Both built from the same paired prompts; SAE latents picked at the last token | Decaying intervention schedule is presented as an SAE-side addition; whether the baseline got it is not established |\n| SAE-space mean-shift jailbreak defence, CC-Delta (arXiv 2602.12418) | SAE side uses paired harmful and jailbreak-wrapped prompts; whether dense baselines used the same pairs is not read | SAE side mean-pools over token-matched harmful-request tokens; baseline pooling not read | Not read |\n| Output-score steering (arXiv 2505.20063) | SAE selection uses no co","at":"2026-11-14T02:41:10.493Z","funded_by":"a funded mandate","research_run_id":"b8c7e920-4ac5-49dc-9ce4-286727571723","run_id":"f860f9ae-d5f6-4c9e-a884-acbded65419a","budget_usd":2.06,"quote_usd":1.8,"funding":null}],"kept_by_this_node":true}],"requests_total":1,"subclaim_count":10,"instance_count":4,"source_map":null}

get_claim_instances

{
  "claim_id": "759728cb-0411-4140-925b-d97d8022e57f"
}

{
  "claim_id": "759728cb-0411-4140-925b-d97d8022e57f",
  "total": 2,
  "limit": 20,
  "offset": 0,
  "has_more": false,
  "instances": [
    {
      "id": "5a9d6f7b-6da9-4da8-b64a-b90abdb4a6c6",
      "source_id": "307b7e03-4ea6-4f39-8180-ca246f360517",
      "verbatim_text": "Downstream Performance: Underperform on downstream tasks; other approaches often outper- form SAEs [Wu et al., 2025, Smith et al., 2025]",
      "context": "Table 1, documented limitations of SAEs, citing AxBench and the GDM negative-results update",
      "stance": "affirms",
      "confidence": 0.75,
      "speaker": null,
      "publication": null,
      "source_date": null,
      "link": null,
      "source_title": "Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?",
      "source_type": "conference_paper",
      "source_url": "https://arxiv.org/pdf/2602.14111",
      "source_published_date": "2026-02-15",
      "source_authors": [
        "Korznikov, Anton",
        "Galichin, Andrey",
        "Dontsov, Alexey",
        "Rogov, Oleg",
        "Oseledets, Ivan",
        "Tutubalina, Elena"
      ],
      "source_publisher": "arXiv.org",
      "source_doi": null,
      "source_interest": null,
      "recorded_by": "extractor",
      "display_rank": null,
      "reading": null,
      "draws_on": [],
      "story": {
        "standing": "untraced",
        "basis": "none",
        "depth": 0,
        "diverges": false,
        "downstream_total": 0
      }
    },
    {
      "id": "48bdc782-781e-4c7c-908c-7acedb1e119b",
      "source_id": "447e476f-1376-48e2-9679-30bc4fa0e96c",
      "verbatim_text": "if current SAEs really are a big step forwards for interpretability, it should not be so hard to find compelling scenarios where they beat baselines.",
      "context": "Authors summarize negative results from their work and parallel work on probing, unlearning, and steering, and note positive results are in niche or contrived settings.",
      "stance": "affirms",
      "confidence": 0.8,
      "speaker": null,
      "publication": null,
      "source_date": null,
      "link": null,
      "source_title": "Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2)",
      "source_type": "blog_post",
      "source_url": "https://www.lesswrong.com/posts/4uXCAJNuPKtKBsi28/negative-results-for-saes-on-downstream-tasks",
      "source_published_date": null,
      "source_authors": [
        "lewis smith",
        "Senthooran Rajamanoharan",
        "Arthur Conmy",
        "CallumMcDougall",
        "Tom Lieberum",
        "János Kramár",
        "Rohin Shah",
        "Neel Nanda"
      ],
      "source_publisher": null,
      "source_doi": null,
      "source_interest": null,
      "recorded_by": "extractor",
      "display_rank": null,
      "reading": null,
      "draws_on": [],
      "story": {
        "standing": "untraced",
        "basis": "none",
        "depth": 0,
        "diverges": false,
        "downstream_total": 0
      }
    }
  ]
}