Step 2: tool results

get_claim

{
  "include": [
    "arguments",
    "subclaims",
    "related"
  ],
  "claim_id": "86063716-4ef1-48dc-abe6-02ef8b75f176"
}

{"claim":{"id":"86063716-4ef1-48dc-abe6-02ef8b75f176","kind":"claim","canonical_form":"In agent-based alignment auditing, black-box tools outperform white-box interpretability tools.","claim_type":"empirical_verifiable","state":"active","decomposition_status":"complete","importance":0.65,"contestation":0.55,"domains":[],"tags":[{"slug":"ai-alignment","name":"AI Alignment"},{"slug":"mechanistic-interpretability","name":"Mechanistic Interpretability"},{"slug":"ai-safety","name":"AI Safety"},{"slug":"ai-agents","name":"AI Agents"}],"steward_state":"running","stewarded_at":"2026-11-14T20:41:05.518Z","canonical_direction_note":"The affirmative form captures the source's comparison favoring black-box tools, rather than its qualification that white-box tools can still help.","created_at":"2026-11-13T01:24:48.986Z","updated_at":"2026-11-14T20:39:57.034Z","page_url":"https://minerval.ai/claims/86063716-4ef1-48dc-abe6-02ef8b75f176"},"assessment":null,"coverage":{"examined":false,"on":[],"note":"No judgment of this node is in force. That is not a finding either way."},"quantity":null,"threshold_of":null,"question":null,"formalization":null,"formalization_pending":null,"formalization_history":[],"verification":null,"lean_checks":[],"lean_checks_total":0,"waiting_messages":[{"trigger":"mandate_plan","from":"a mandate's plan","sender_node_id":null,"sent_at":"2026-11-14T20:39:57.038Z","paid":true,"summary":"First assessment. Read the agent-based auditing studies linked to this node and its siblings (arXiv 2603.04426 and 2602.22755 are likely among them, with the earlier auditing-agents work). For each: t…"}],"requests":[],"requests_total":0,"subclaim_count":0,"instance_count":1,"source_map":null,"subclaims":[],"subclaims_assessed":0,"arguments":[],"related":[],"unplaced_argument_passages":[{"instance_id":"ea9ebbbd-53ab-4387-86ec-28d1c3996856","source":{"id":"a8bae70a-6ace-4953-ada7-146a55bcf8d3","url":"https://arxiv.org/pdf/2602.22755","title":"AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors"},"verbatim_text":"White-box tools outperform the default in SDF models with SFT adversarial training: averaged over the five white-box tools, success rates rise from 40.2% to 46.9%, while on TD models they stay roughly flat (34.4% vs. 33.5%).","context":"Comparison of white-box tool performance by target model type.","stance":"advances","direction":"for"},{"instance_id":"5046a003-e13a-4cf9-bdec-275e48ba5a18","source":{"id":"a8bae70a-6ace-4953-ada7-146a55bcf8d3","url":"https://arxiv.org/pdf/2602.22755","title":"AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors"},"verbatim_text":"However, even in the best case, white-box tools are outperformed by scaffolded black-box affordances.","context":"Summary of white-box versus scaffolded black-box comparison.","stance":"advances","direction":"for"},{"instance_id":"82dca709-64f0-4e15-9745-8272448622a9","source":{"id":"a8bae70a-6ace-4953-ada7-146a55bcf8d3","url":"https://arxiv.org/pdf/2602.22755","title":"AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors"},"verbatim_text":"In contrast, on KTO-trained models, white-box tools are on average worse than the default by 3.5 pp on SDF and 3.8 pp on TD models, suggesting their effectiveness depends heavily on how the target model was trained. However, even in the best case, white-box tools are outperformed by scaffolded black-box affordances.","context":"Results comparing white-box tools (SAEs, logit lens, activation oracles, steering) with scaffolded black-box tools.","stance":"advances","direction":"for"},{"instance_id":"1d9cb9c1-f540-42cb-9b7c-9abf3755108f","source":{"id":"a8bae70a-6ace-4953-ada7-146a55bcf8d3","url":"https://arxiv.org/pdf/2602.22755","title":"AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors"},"verbatim_text":"The unscaffolded sampling tools\nperform similarly to the default agent (34.5–35.0% vs. 35.5%). In contrast, as seen in Figure 4,\nscaffolded text completion, user sampling, and prefilling reach 58.0%, 53.2%, and 51.6%, respectively,\nmaking them the three best tools overall.","context":"Overall success rates of tools against the default agent","stance":"advances","direction":"for"}],"unplaced_argument_passages_total":4}