Step 21: assistant

text

Now the related links to the sibling answers and the assessment.

Calls add_related_claim

{
  "kind": "related",
  "claim_id": "7c1050dc-7ca9-492e-8d3f-ade8d793de4c",
  "reasoning": "Revisits the AxBench SAE steering result with a different feature-selection method (output scores); a reader of either needs the other to see that the later result changes the method rather than the measurement.",
  "other_claim_id": "d3311aac-502f-4e69-ab97-53ee2b6de622"
}

Calls add_related_claim

{
  "kind": "related",
  "claim_id": "7c1050dc-7ca9-492e-8d3f-ade8d793de4c",
  "reasoning": "A 2026 partial rebuttal of the AxBench SAE steering result that changes how features are selected and labelled; it bears on how the original result generalizes, not on whether it was measured correctly.",
  "other_claim_id": "a5665880-9436-4582-a906-e2161fe8e71a"
}

Calls update_claim_assessment

{
  "status": "verified",
  "claim_id": "7c1050dc-7ca9-492e-8d3f-ade8d793de4c",
  "assessment": "The claim restates the headline steering result of AxBench (Wu, Arora et al., Stanford, ICML 2025), and the benchmark paper's own tables bear it out. AxBench steers instruction-tuned Gemma-2-2B (residual stream, layers 10 and 20) and Gemma-2-9B (layers 20 and 31) toward 500 concepts drawn from the Neuronpedia descriptions of Gemma Scope SAE latents. A language-model judge rates each steered response 0 to 2 for concept presence, relevance to the instruction and fluency, and the score is the harmonic mean of the three. The steering strength is chosen per concept on half the instructions and evaluated on the other half.\n\nAveraged over the four sites, prompting scored 0.894, difference-in-means (DiffMean) 0.239, the SAE latent whose description named the concept 0.165, and the SAE latent chosen by AUROC on the 144 labelled training examples 0.157. The SAE fell below DiffMean at every one of the four sites, most clearly in earlier layers, and the paper reports the DiffMean advantage as significant, with a 58.7% per-concept win rate against SAEs. Prompting beat the SAE by a wide margin everywhere. The SAE did outperform several other supervised directions (probe, PCA, LAT, supervised steering vector), and every representation method except ReFT-r1 trailed prompting badly.\n\nLater work does not dispute this measurement. It changes the method being measured. Arad, Mueller and Belinkov (EMNLP 2025) report that [[claim:d3311aac-502f-4e69-ab97-53ee2b6de622|keeping only SAE features with high output scores]] improves SAE steering two to threefold and makes it competitive with supervised methods. A 2026 preprint by Jørgensen and Hansen reports that [[claim:a5665880-9436-4582-a906-e2161fe8e71a|features selected and labelled by a supervised pipeline]] approach LoRA on AxBench. Both results bear on whether SAEs must steer worse, not on whether they did under AxBench's protocol. The claim's general wording (\"SAE steering\") is best read as the benchmark's standard configuration: description-matched Gemma Scope latents added to the residual stream. For that configuration it is established. It should not be read as a finding that any SAE-based steering method loses on AxBench.",
  "confidence": 0.85,
  "claim_credence": 0.92,
  "marginal_yield": 0.2,
  "reasoning_trace": "Read directly from the TeX source of arXiv 2501.17148 (main.tex: abstract, introduction, setup, methods, results).\n\nSetup. 500 concepts (Concept500) sampled from the Neuronpedia/Gemma Scope concept list. Sites: Gemma-2-2B-it L10 and L20 (using Gemma Scope SAEs trained on the base model, since none existed for the instruction-tuned model), and Gemma-2-9B-it L20 and L31. Training data: 144 synthetic examples per concept generated by gpt-4o-mini, balanced positives and negatives. SAE method: the Gemma Scope JumpReLU latent corresponding to the concept label, with its decoder vector added to the residual stream (activation addition, as for DiffMean). The magnitude is the steering factor times the latent's maximum activation from Neuronpedia; clamping is ablated in an appendix. SAE-A picks the latent with the highest AUROC on the training labels, using max-pooled activations. DiffMean is the unit-normalised mean of positive minus negative token activations. Prompting uses an LLM to engineer a steering prompt prepended to the instruction. Steering metric: an LLM judge scores concept, instruct and fluency each 0, 1 or 2, and the score is their harmonic mean. Ten Alpaca-Eval instructions per concept, 14 steering factors, up to 128 tokens; the factor is selected on one half of the instructions and the result reported on the other half.\n\nResults (Table \"Mean overall steering scores\"): Prompt 0.698 / 0.731 / 1.075 / 1.072 (average 0.894); DiffMean 0.297 / 0.178 / 0.322 / 0.158 (0.239); SAE 0.177 / 0.151 / 0.191 / 0.140 (0.165); SAE-A 0.166 / 0.132 / 0.186 / 0.143 (0.157). The text says \"DiffMean significantly outperforms SAEs, particularly in earlier layers\". Win rate of DiffMean against SAE: 63.2 / 55.2 / 64.3 / 52.2%, average 58.7%. Prompting's win rate against SAE: 94.5%. The SAE beat Probe (0.098), PCA (0.105), LAT (0.127) and SSV (0.026). Concept detection, for context: SAE AUROC 0.695 and SAE-A 0.917, against DiffMean 0.942.\n\nMargins. Against prompting the gap is very large. Against DiffMean the absolute gap is modest (0.074 on a 0 to 2 scale), and at 9B L31 it is small (0.158 against 0.140, win rate 52.2%). So \"underperforms difference-in-means\" holds on average and at each site, but is weakest at late layers. Credence is held a little below 1 for this reason and for the dependence on one LLM judge and one protocol.\n\nLater papers, read at abstract level only. Arad et al. 2505.20063 change the selection (output-score filtering) and report two- to threefold gains and competitiveness with supervised methods. Jørgensen and Hansen 2605.31183 call themselves a \"partial rebuttal\" and use a supervised selection and labelling pipeline. The question node's assessment, from earlier research, records that this work steers only concepts with a well-matched feature, compares against the original paper's baseline figures on a different concept set, and concedes it is not competitive with prompting. Neither paper reports re-running the original configuration and getting different numbers. Chanin (2026) cites the result as found. No source located disputes the measurement itself. The full results tables of the two rebuttals were not read this pass, so the size of their improvements is taken from their abstracts.\n\nReading of the claim. The claim names no feature-selection method. Its natural reading is the AxBench result as reported. Taken as \"no SAE steering method can beat DiffMean on AxBench\", it would be weaker, given the rebuttals. The assessment states the scope explicitly."
}