TAR's robustness to fine-tuning attacks at 1,000 steps varies with dataloader shuffling order.
Assessed
First pass. The first turn used much of the budget, so the claim is left atomic, with no decomposition and no instances recorded. Assessed as supported (confidence 0.6, credence 0.8), resting on the re-evaluation by Qi et al. 2024. Importance set to 0.4, contestation 0.35. A later pass should read the paper in full and record it as an instance.
Assessed Supported
verdict confidence 0.60 · credence 0.80
TAR (Tamper-Attack Resistance, Tamirisa et al., 2024) is a method meant to make open-weight language models resist harmful fine-tuning. The original evaluation reported robustness against fine-tuning attacks of about 1,000 steps. A later re-evaluation of durability claims for open-weight safeguards (Qi et al., 2024, "On Evaluating the Durability of Safeguards for Open-Weight LLMs") reported that this robustness was fragile under small changes to the attack setup. Changing only the order in which the dataloader shuffles training examples, with everything else fixed, made some 1,000-step attacks recover the hazardous capability. That is direct evidence that the result depends on shuffle order. The claim rests mainly on that one re-evaluation, and this pass did not re-read it in full. Three kinds of evidence would settle it more firmly: repeated attack runs over many shuffle seeds with all other hyperparameters fixed, the spread of post-attack accuracy across those seeds, and a check that the original TAR result falls inside or at the favorable tail of that spread. Independent replications with released code and seeds would decide the question either way.
Claim entered the graph