Step 21: assistant

text

I've rated the claim **supported**: confidence 0.8 that this status is the right reading, and a 0.88 probability that the claim is true. It holds as the paper defines its terms, but the guarantee is narrower than the wording suggests. The paper is Draguns et al., arXiv 2610.13928, accepted at NeurIPS 2026. I read it from its TeX source: the threat model, compiler, constructions, related work and Appendix C.

**The main result and its assumptions**
- **How it works:** the hidden circuit's weights are encrypted in "digital lockers", which release their contents only on a secret, high-entropy trigger. The decryption is then built into ordinary network weights.
- **The proof (Appendix C):** the finished weights are computed publicly from the encrypted contents alone. So any efficient auditor that elicits the capability would break the lockers with the same success rate. This step is correct and elementary.
- **Cryptographic assumptions:**
  - The lockers are hash-based, with Keccak/SHA-3 treated as an ideal random function. That is a heuristic but standard assumption; the paper notes lockers also exist under stronger standard assumptions.
  - The trigger is at least as unpredictable as the security parameter.
- **Two assumptions are stated but not proved:**
  - The compiled network runs the locker correctly in finite precision, and no wrong trigger unlocks the network that the locker itself would reject.
  - The lockers stay secure jointly when one trigger opens all of them.
- **"Arbitrary capabilities"** means any polynomial-size circuit that fits inside the host network. The overhead is large: the demonstrated hidden circuits have only hundreds of ternary weights.
- **"Elicit"** means two games: finding any input that unlocks the hidden computation, and telling which of two equal-size capabilities is embedded.
- **Auditors covered:** any polynomial-time algorithm with full weights and knowledge of the embedding scheme, but not the trigger.
- **Not covered:** the guarantee is about eliciting, not detecting. The authors say a defender may still flag the weights as anomalous, and the payload's size is visible. It also doesn't stop a defender learning the same task from other data.

**Proved construction versus implemented one**
The proof covers an idealized construction at full strength. The models in the elicitation experiments use one round of Keccak and 4 to 16 bit triggers, which brute force breaks trivially. Their resistance to fine-tuning, weight noising, gradient trigger search and latent adversarial training is evidence that those methods fail, not a case of the theorem. A full-strength build (24-round Keccak-f[1600], 128-bit trigger) was compiled and checked for correct behaviour. In the sections I read, the authors don't report running the elicitation attacks on it.

**Prior work and criticism**
- It extends Draguns et al. (2024), which hid fixed strings behind circuits that resist elicitation.
- It also extends Goldwasser et al. (2022), whose planted backdoors are undetectable, a stronger property, but only flip a classifier's output.
- One web search found no published criticism. That counts for little, since the paper is about a month old.

**What I recorded**
- One argument, "the reduction to digital-locker security", rated as giving a conclusion that is certain if its premises hold. I did not mark it decisive, because the random-oracle and correct-execution premises are assumptions, not proven.
- The two passages left over from ingestion: one placed on that argument, the other dropped automatically as a duplicate of the same source.
- Importance 0.55 and contestation 0.3, a log entry, and answers to both inbox messages.

I kept the premises as prose in the argument rather than separate subclaims, since the lockers' security under the random-oracle assumption is settled background. The run spent about $0.55 of its $1.00 budget.