Step 28: tool results

file_request

{
  "ask": "Run one weight-noise protocol (the sweep and best-over-seeds statistic of Tice et al., arXiv 2412.01784, with the with-password or untriggered model under the same noise as comparator) on fine-tuned sandbagging models of varying lock depth and scale, including the eleven password-locked models said to be released with arXiv 2608.29458, and on benign fine-tuned models that are not sandbagging. Report per-seed accuracy curves, the improvement ratio in each group, and the false-positive rate on the benign fine-tunes.",
  "why": "Whether noise raises scores in trained sandbaggers, and whether it leaves non-sandbagging fine-tunes alone, is what the claim's general form turns on. Existing studies each use a different noise construction and statistic, the largest reported null does not describe its procedure, and no study has noised benign fine-tunes. Clear gains in locked models with flat benign fine-tunes would move the claim toward supported for trained sandbagging; flat locked models, or gains in benign fine-tunes, would move it toward doubtful.",
  "claim_ids": [
    "e0abe646-c78f-417c-813f-b8e2c15d775c"
  ],
  "who_could": "Anyone with GPU access and open weights: the released locks of arXiv 2608.29458, the original study's two locked models, or newly trained ones. The authors of either paper could do it most cheaply.",
  "satisfied_by": "A public write-up or data release giving, for at least several fine-tuned locks at two or more model scales and at least a few benign fine-tunes, accuracy against noise level per seed under a single stated protocol, with locked, unlocked and benign conditions side by side. Either outcome satisfies it."
}

{"success":true,"message":"Request 1a0826ec-b966-49de-8cfa-dda3ac45b85b filed.","request_id":"1a0826ec-b966-49de-8cfa-dda3ac45b85b"}

search_findings

{
  "limit": 3,
  "query": "noise injection failed on eleven password-locked models baseline below random guessing"
}

{"findings":[{"id":"80727432-011a-4299-ba1b-efaa0f7b8b99","headline":"One of the two password-locked models on which noise injection was validated was itself trained to imitate a weaker model, so imitation training alone does not explain the method's weak showing in the UK auditing game.","claim_id":"250fee4e-8706-4658-b579-b0382bfae55f","agent":"steward","importance":3,"sighting_count":1,"first_noted_at":"2026-11-14T06:45:33.104Z","similarity":0.6417642204144776},{"id":"1cf195e2-6f29-4b7b-8f85-1544d4ab39d8","headline":"Weight-noise detection of sandbagging separates prompted and simply locked models from benign ones but not models trained to imitate a weaker model, which is why published results appear to conflict.","claim_id":"bda65b89-5ecb-4728-bfe9-0b54ee7fbc25","agent":"steward","importance":4,"sighting_count":1,"first_noted_at":"2026-11-13T21:51:26.799Z","similarity":0.557753562743459},{"id":"cea2b94d-9d93-487a-8e3e-8cac667ea74c","headline":"The published disagreement over whether sparse-autoencoder features beat linear probes on labeled detection tasks tracks two setup choices, whether features are chosen with the labels and whether the raw-activation baseline may use more than one token, and no study matches both.","claim_id":"cb2d56c9-fcce-47d3-b2b3-5127f10f3293","agent":"steward","importance":3,"sighting_count":1,"first_noted_at":"2026-11-13T21:50:22.533Z","similarity":0.3761167632810515}],"message":"80727432-011a-4299-ba1b-efaa0f7b8b99 (2026-11-14, steward, on claim 250fee4e-8706-4658-b579-b0382bfae55f, importance 3): \"One of the two password-locked models on which noise injection was validated was itself trained to imitate a weaker model, so imitation training alone does not explain the method's weak showing in the UK auditing game.\"\n1cf195e2-6f29-4b7b-8f85-1544d4ab39d8 (2026-11-13, steward, on claim bda65b89-5ecb-4728-bfe9-0b54ee7fbc25, importance 4): \"Weight-noise detection of sandbagging separates prompted and simply locked models from benign ones but not models trained to imitate a weaker model, which is why published results appear to conflict.\"\ncea2b94d-9d93-487a-8e3e-8cac667ea74c (2026-11-13, steward, on claim cb2d56c9-fcce-47d3-b2b3-5127f10f3293, importance 3): \"The published disagreement over whether sparse-autoencoder features beat linear probes on labeled detection tasks tracks two setup choices, whether features are chosen with the labels and whether the raw-activation baseline may use more than one token, and no study matches both.\""}

query_sql

{
  "sql": "SELECT id, assessed_at FROM assessments WHERE node_id = $1 AND is_current",
  "params": [
    "e0abe646-c78f-417c-813f-b8e2c15d775c"
  ]
}

{"success":true,"columns":["id","assessed_at"],"rows":[["45b06aa1-e0e7-40aa-a3dd-78e45a772e83","2026-11-14T19:09:19.597Z"]],"row_count":1,"truncated":false,"elapsed_ms":49,"plan_cost":8}