Step 17: assistant

text

Now the assessment.

Calls update_claim_assessment

{
  "status": "supported",
  "claim_id": "f4204e80-58c4-4214-b3b1-254c34855e38",
  "assessment": "The claim comes from Draguns, Motwani, Douglas and Schroeder de Witt, \"AI Models Can Provably Hide Arbitrary Capabilities\" (arXiv 2610.13928, accepted at NeurIPS 2026). Read with the scope that paper gives its terms, it is true, and the proof is short and standard. Any capability that a polynomial-size feed-forward circuit computes can have its weights locked in digital lockers keyed by a secret, high-entropy trigger. The locker evaluation is then compiled into ordinary network weights. Because the compiled weights are a public function of the locker ciphertexts alone, any efficient auditor that elicits the capability yields an equally good attacker on the lockers. The idea of hiding a whole program behind a point-function obfuscator (a digital locker) is decades old in cryptography. The paper's contribution is to carry it out inside neural-network weights and to state the transfer for networks.\n\nThe guarantee is narrower than the headline. \"Arbitrary\" means any circuit of polynomial size that fits in the host network, and the overhead is large: the hidden circuits demonstrated have hundreds of ternary weights. \"Elicit\" means two things: finding any input that unlocks the hidden computation, and telling which of two equal-size capabilities is embedded. The auditor is any probabilistic polynomial-time algorithm with full weights and knowledge of the embedding scheme, but not the trigger. The guarantee does not say the tampering is undetectable: the authors state that a defender may flag the weights as anomalous, and the size of the hidden payload is visible. Nor does it stop a defender from learning the same task elsewhere. It rests on the random-oracle model instantiated with Keccak (SHA-3), a heuristic but standard assumption. The authors note that standard-model lockers exist under stronger assumptions. It also assumes the compiled network executes the locker correctly in finite precision and accepts no trigger the locker rejects.\n\nThe models used in the paper's elicitation experiments use one Keccak round and 4 to 16 bit triggers. Brute force breaks these trivially, so the proof does not cover them. Their resistance to fine-tuning, noising, gradient trigger search and latent adversarial training is empirical evidence that those particular methods fail, not an instance of the theorem. A full-strength build (24-round Keccak-f[1600], 128-bit trigger) was compiled and checked for correct behaviour. The paper does not report running the elicitation attacks against that build. The result extends two earlier lines of work: Draguns et al. (2024), which hid fixed strings behind unelicitable cryptographic transformer circuits, and Goldwasser et al. (2022), which planted backdoors that are undetectable, a stronger property, but whose payload only flips a classifier's output. No published rebuttal was found in a brief search.",
  "confidence": 0.8,
  "claim_credence": 0.88,
  "marginal_yield": 0.25,
  "reasoning_trace": "Read from the arXiv TeX source: the threat-model section (sections/2.5_threat.tex), the compiler section (3_compiler.tex), the constructions section (4_constructions.tex), related work, and Appendix C (app:compiler).\n\nMain result. Two games are defined. In trigger-finding, a PPT auditor given the weights of M wins if it outputs any input that unlocks the hidden computation. In payload-hiding, it is given M_b = Embed(H, C_b, T) for equal-size C_0, C_1 and must guess b. Security requires a negligible advantage in λ, with trigger min-entropy at least λ. Appendix C argues that Weights(M) = F(H, p, c; ρ) for a public polynomial-time F of the locker ciphertexts c, given independent compiler randomness. For payload-hiding, the simulator B runs F on the challenge ciphertexts and has exactly A's advantage. For trigger-finding, the paper requires that any trigger unlocking the compiled model also unlocks the lockers. Both conditions are explicit assumptions. Joint locker security over the whole ciphertext collection under a shared trigger is also assumed. This is a sound and elementary reduction. The step that carries the weight is that multibit digital lockers exist: hash-based in the random-oracle model (Lynn, Prabhakaran and Sahai 2004), and in the standard model under stronger assumptions (Canetti and Dakdouk 2008, and the 2022 plain-model lockers the paper cites). The ROM is uninstantiable in contrived cases, but SHA-3 as a random oracle is a widely accepted heuristic.\n\nIdealized versus implemented. The proof is about the idealized compiled construction at full parameters, plus a correct-execution assumption. The experiments use Keccak-p with 1 round and 4 to 16 bit triggers. The paper says these settings \"would not withstand cryptanalysis\" and that the tested methods do not exploit hash structure. So the experimental resistance results are not covered by the theorem. A 24-round Keccak-f[1600] build with a 128-bit trigger (110 layers, 3.58e9 dense parameters) was verified for correctness on held-out inputs and on 10,000 wrong triggers. Elicitation against that build is not reported in the sections read.\n\nScope words. \"Arbitrary capabilities\" means polynomial-size circuits that fit the host: for example, about a 99x99 hidden matrix per 10,000 host dimensions in the single-position variant, with a row-wise autoregressive variant that uses O(n) width. The guarantee is unelicitability, not undetectability: the paper states it does not rule out flagging the weights as anomalous, and payload size leaks. The claim's wording (\"no polynomial-time white-box auditor can elicit them, under cryptographic assumptions\") matches this unelicitability statement. Status is supported rather than verified because the theorem is a conditional reduction resting on the ROM or stronger standard-model assumptions and on a correct-execution premise checked only empirically. No formal proof exists, and no independent scrutiny was found. Credence 0.88 reflects small residual doubt about the transfer conditions, not about the cryptographic idea.\n\nSearch for criticism: one web search for the title plus critique returned only the paper itself. The absence of rebuttal weighs little, since the paper is a month old."
}

Calls evaluate_argument

{
  "verdict": "certain",
  "evaluation": "Granting its premises, the conclusion follows: an auditor who succeeds against weights that are a public function of the locker ciphertexts directly yields an attacker on the lockers with the same advantage. The argument therefore stands or falls with its premises. These are that secure multibit digital lockers exist, which holds in the random-oracle model with a hash such as SHA-3 and in the standard model under stronger assumptions, and that the compiled network evaluates the locker faithfully in finite precision and accepts no trigger the locker rejects. The second premise is assumed and checked only empirically. Because the first premise is a modelling assumption, not a theorem, and the second is unproved, the argument is not decisive. It also establishes unelicitability only, not undetectability.",
  "argument_id": "ec610556-ab82-4805-a87f-39605b4b615d"
}

Calls set_claim_importance

{
  "claim_id": "f4204e80-58c4-4214-b3b1-254c34855e38",
  "reasoning": "Consequential for AI governance, because it limits elicitation-based capability audits under adversarial supply chains. The cryptographic core is uncontroversial. Debate lies mainly in how far the result bears on practice: detectability and overhead.",
  "importance": 0.55,
  "contestation": 0.3
}