Step 12: tool results

{
  "content": "On Gemma-2 activations, Kantamneni et al. found three apparent SAE wins: SAE latents flagged likely mislabeled GLUE CoLA examples, an SAE latent exposed a spurious end-of-text punctuation cue in an AI-vs-human dataset, and max-pooled multi-token SAE probes beat last-token baselines (19.6% win rate). Rerunning each with a dense method, logistic regression flagged the same CoLA sentences, the logistic-regression probe's top-activating Pile tokens were punctuation, and an attention-pooled dense probe cut the pooled SAE win rate to 8.7% while baselines won 34.8%. Since the dense methods reached the same conclusions on the same data and model, the SAE results are matched by simple baselines.",
  "argument_id": "d09fa10c-dec1-4d68-88d0-0fcfe0ed38d4"
}

{
  "content": "The same study reports that on the CoLA examples whose labels an LLM ensemble judged wrong, the single SAE latent classifier (accuracy 0.63) and the dense SAE probe (0.56) beat logistic regression (0.46) considerably; pooled SAE probes still beat the strongest baseline on 8.7% of datasets; and the baseline routes to the CoLA and punctuation findings were run after SAE latents had already suggested what to look for. A baseline that can confirm a hypothesis once pointed at it does not show it would have generated the hypothesis unaided, so \"match\" holds for the conclusions reached but not uniformly for the numbers or for discovery.",
  "argument_id": "8395f9d1-9885-42bf-9754-d2c1bb76e1f2"
}

{
  "reading": "full",
  "argument_id": "d09fa10c-dec1-4d68-88d0-0fcfe0ed38d4",
  "instance_id": "1175c448-86b3-406f-8812-847c680986d6"
}

{
  "reading": "full",
  "argument_id": "d09fa10c-dec1-4d68-88d0-0fcfe0ed38d4",
  "instance_id": "79baa35d-2745-4fc6-a02c-379ff0242b70"
}