Step 3: assistant

thinking

Calls update_claim_assessment

{
  "status": "supported",
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "assessment": "The claim is that probes trained on sparse autoencoder (SAE) latents do not reliably beat ordinary probes trained on raw language-model activations when classifying activations. The evidence supports it. The largest head-to-head study, \"Are Sparse Autoencoders Useful? A Case Study in Sparse Probing\" (Kantamneni, Engels, Rajamanoharan, Tegmark, Nanda, 2025), compared SAE probes against tuned baselines (logistic regression, PCA, k-nearest neighbours, XGBoost, MLP) on about 113 binary classification datasets using Gemma 2 SAEs. It tested the regimes where SAEs were expected to help: scarce training data, class imbalance, label noise and covariate shift. It found no meaningful improvement in any regime. When SAE probes were added to the candidate set and the best probe was chosen on validation data, test performance did not improve. An apparent multi-token advantage for max-pooled SAE probes (a 19.6% win rate) fell to 8.7% once the baselines were also allowed attention pooling. In a separate set of negative results, Google DeepMind's interpretability team reported that SAE-based probes for detecting harmful user intent generalised out of distribution worse than linear probes on the residual stream.\n\nThe claim says \"do not consistently outperform\", not \"never outperform\". It is compatible with isolated wins, and the studies do report some: individual datasets where SAE probes come out ahead, some interpretability-driven debugging of spurious features, and dataset-quality findings. Some position papers argue that this pattern is expected, because a reconstruction holds strictly less information than the activation it reconstructs. The live disagreement is not over whether SAE probes win on average, which they do not in these setups. It is over whether newer SAE architectures, wider dictionaries, larger models or particular task types could reverse the result. Evidence for such a reversal is thin. The main caveat is that most of the controlled evidence comes from one research group and one model family (Gemma 2 with Gemma Scope SAEs), so the evidence is less independent than the number of experiments suggests.",
  "confidence": 0.75,
  "claim_credence": 0.85,
  "marginal_yield": 0.45,
  "reasoning_trace": "Primary evidence weighed: arXiv 2502.16681 (Kantamneni et al.), the passages recorded on this node: \"At all parameter values in each regime, SAEs show no meaningful improvement over baselines\"; \"Without using the quiver of arrows approach, we still find SAEs underperform baselines\"; the multi-token result (19.6% falls to 8.7% against attention-pooled baselines); the OOD failure traced to a language-specific latent. The setup draws on my background knowledge of the paper: Gemma 2 2B/9B, Gemma Scope SAEs at several widths, baselines tuned with hyperparameter search, roughly 113 datasets. The DeepMind post \"Negative results for SAEs on downstream tasks\" (Smith, Rajamanoharan, Nanda et al., March 2025) reports SAE probes underperforming linear probes OOD on harmful-intent detection. Position paper arXiv 2506.23845 offers the information-loss explanation. Limits of this pass: the budget was tight, and arXiv 2505.20063, 2605.31183 and 2602.14111, which the brief asked to compare study by study, were not opened. If any of them shows a robust, well-baselined SAE-probe advantage in some regime, the word \"consistently\" still holds, but the assessment should name that regime. Independence caveat: much of the evidence shares authors (Nanda, Rajamanoharan), models and SAEs. Credence 0.85 reflects the strong evidence against an average advantage and the weak wording of the claim, discounted for the unread studies and the narrow range of models."
}

Calls set_claim_importance

{
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "reasoning": "This is a central practical question for whether SAEs are useful in interpretability. It shapes research direction within the field, but its reach outside the field is narrow. Few dispute the average result, though some do dispute how far it generalises.",
  "importance": 0.5,
  "contestation": 0.45
}

Calls log_stewardship_decision

{
  "claim_id": "9fbf1b17-4bf8-42bc-a5e1-420054bf7463",
  "reasoning": "First pass under a tight budget, with $0.49 already consumed at the start. Assessed as supported (credence 0.85), relying mainly on 2502.16681 and the DeepMind negative-results post. Left the claim atomic. The study-by-study comparison of 2505.20063, 2605.31183 and 2602.14111 was not done, and marginal_yield is set to 0.45 so a later pass can do it. The seven unplaced argument passages were left for a later pass.",
  "action_taken": "assessed"
}

Calls answer_messages

{
  "answers": [
    {
      "note": "Assessed as supported, with no decomposition.",
      "message": "m1",
      "outcome": "changed"
    },
    {
      "note": "Recorded a first assessment based on 2502.16681 and the DeepMind post. The budget did not cover opening 2505.20063, 2605.31183 or 2602.14111 for the full study-by-study comparison. The gap is stated in the assessment, and marginal_yield is 0.45.",
      "message": "m2",
      "outcome": "changed"
    }
  ]
}