Step 7: assistant

thinking

Calls update_claim_assessment

{
  "status": "supported",
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "assessment": "The claim is that probes trained on sparse autoencoder (SAE) latents do not reliably beat ordinary probes trained on raw language-model activations when classifying activations. The evidence supports it. The largest head-to-head study, \"Are Sparse Autoencoders Useful? A Case Study in Sparse Probing\" (Kantamneni, Engels, Rajamanoharan, Tegmark, Nanda, 2025), compared SAE probes against tuned baselines (logistic regression, PCA, k-nearest neighbours, XGBoost, MLP) on about 113 binary classification datasets using Gemma 2 SAEs. It tested the regimes where SAEs were expected to help: scarce training data, class imbalance, label noise and covariate shift. It found no meaningful improvement in any regime. When SAE probes were added to the candidate set and the best probe was chosen on validation data, test performance did not improve. An apparent multi-token advantage for max-pooled SAE probes (a 19.6% win rate) fell to 8.7% once the baselines were also allowed attention pooling. In a separate set of negative results, Google DeepMind's interpretability team reported that SAE-based probes for detecting harmful user intent generalised out of distribution worse than linear probes on the residual stream. A 2026 sanity-check study (Korznikov et al.) adds indirect evidence. In that study, SAEs whose feature directions or activation patterns were fixed to random values nearly matched fully trained SAEs on sparse probing (0.69 vs 0.72), which suggests that the learned dictionary contributes little to probing performance.\n\nThe claim says \"do not consistently outperform\", not \"never outperform\". It is compatible with isolated wins, and the studies do report some on individual datasets. The most prominent pro-SAE rebuttals are about steering, not classification. Arad, Mueller and Belinkov (2025) found that SAE steering becomes competitive with supervised methods once features with low output scores are filtered out, and Jørgensen and Hansen (2026) argue that SAEs can approach LoRA performance on the AxBench steering benchmark. Neither paper shows SAE probes beating tuned activation probes, so neither bears directly on this claim. Some position papers argue that the negative probing result is expected, because a reconstruction holds strictly less information than the activation it reconstructs. What remains open is whether newer architectures, larger models or particular task types could reverse the result. The main caveat is independence: most of the controlled probing evidence comes from one group and one model family (Gemma 2 with Gemma Scope SAEs).",
  "confidence": 0.78,
  "claim_credence": 0.85,
  "marginal_yield": 0.35,
  "reasoning_trace": "Study by study:\n(1) arXiv 2502.16681, Kantamneni et al. (DeepMind-affiliated authors including Nanda and Rajamanoharan). Gemma 2 with Gemma Scope SAEs at several widths, roughly 113 datasets, hyperparameter-tuned baselines, regimes: data scarcity, class imbalance, label noise, covariate shift. \"Consistently\" was operationalised as win rate and as gain in test AUC from adding SAE probes to a validation-selected set of candidate probes. Result: no meaningful improvement in any regime. SAE probes won on some individual datasets, and the multi-token win rate fell from 19.6% to 8.7% against attention-pooled baselines. One OOD failure was traced to a language-specific latent. Setup details beyond the recorded passages come from background knowledge of the paper.\n(2) DeepMind \"Negative results for SAEs on downstream tasks\" (2025). SAE probes for harmful intent did worse than linear probes OOD. Shares authors with (1), so it is not an independent line.\n(3) arXiv 2602.14111 v2, Korznikov et al. (independent group). Abstract only: SAE baselines with randomised directions or activation patterns score 0.69 on sparse probing against 0.72 for trained SAEs. The comparison is with random SAEs, not with raw-activation probes, so it bears on the claim only indirectly.\n(4) arXiv 2505.20063, Arad, Mueller and Belinkov, and (5) arXiv 2605.31183, Jørgensen and Hansen. Both are about steering (AxBench and output-score feature selection), not classification probes. They are relevant to the parent question but not to this claim.\n(6) arXiv 2506.23845, position paper: SAE reconstructions carry strictly less information than the original activations.\nOnly abstracts were read for (3) to (5), because of budget. Credence 0.85."
}

Calls record_claim_instance

{
  "url": "https://arxiv.org/abs/2602.14111",
  "title": "Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?",
  "stance": "mentions",
  "context": "Compares trained SAEs with randomised-SAE baselines, not with raw-activation probes; indirect bearing on the claim.",
  "speaker": "Anton Korznikov et al.",
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "confidence": 0.7,
  "source_date": "2026-02-15",
  "source_kind": "preprint",
  "verbatim_text": "we show that our baselines match fully-trained SAEs in interpretability (0.87 vs 0.90), sparse probing (0.69 vs 0.72), and causal editing (0.73 vs 0.72)."
}

Calls log_stewardship_decision

{
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "reasoning": "Read the abstracts of 2505.20063, 2605.31183 and 2602.14111. The first two are steering studies, not probing studies, and do not bear directly on this claim. The third gives indirect support. Refined the assessment, recorded 2602.14111 as a mention, and lowered marginal_yield to 0.35.",
  "action_taken": "reassessed"
}