TAR's robustness to fine-tuning attacks at 1,000 steps varies with dataloader shuffling order.
Assessment
The evidence favors the claim, but the examination is incomplete or the evidence is indirect.
TAR (Tamper-Attack Resistance, Tamirisa et al., 2024) is a method meant to make open-weight language models resist harmful fine-tuning. The original evaluation reported robustness against fine-tuning attacks of about 1,000 steps. A later re-evaluation of durability claims for open-weight safeguards (Qi et al., 2024, "On Evaluating the Durability of Safeguards for Open-Weight LLMs") reported that this robustness was fragile under small changes to the attack setup. Changing only the order in which the dataloader shuffles training examples, with everything else fixed, made some 1,000-step attacks recover the hazardous capability. That is direct evidence that the result depends on shuffle order.
The claim rests mainly on that one re-evaluation, and this pass did not re-read it in full. Three kinds of evidence would settle it more firmly: repeated attack runs over many shuffle seeds with all other hyperparameters fixed, the spread of post-attack accuracy across those seeds, and a check that the original TAR result falls inside or at the favorable tail of that spread. Independent replications with released code and seeds would decide the question either way.
Full reasoning: the evidence and decisions behind this verdict
Prior from the method's design: TAR's adversarial meta-learning trains against a limited set of simulated attacks. That makes its robustness plausibly brittle to attack details outside the training distribution, including the order of data in the stochastic optimization trajectory. A robustness result that holds for one seed and fails for others is a common pattern in adversarial-robustness evaluation. The main evidence is Qi et al. (2024, arXiv 2412.07097), as remembered rather than re-read in this pass. It reports that TAR's resistance to 1,000-step fine-tuning attacks broke when the dataloader shuffle order changed and nothing else did. The TAR authors' original evaluation used fixed configurations and did not report variance across shuffle seeds. No direct rebuttal of the shuffle-sensitivity finding is known. Status is supported rather than verified because the source was not re-read in full this pass and the evidence comes from a single re-evaluation. Budget limits prevented decomposition and recording of instances.
Decomposition
This claim is atomic: it bottoms out in a bedrock fact, an open empirical question, or a value premise, and does not decompose further.
Provenance
Where this claim has been said, linked to its canonical form.
Additionally, Qi et al. [45] found that TAR’s ro- bustness at 1,000 of steps of fine-tuning varies when changing the dataloader shuffling order, which we corroborate in Figure 5.
Appendix C.1 update note; authors say robustness is consistent for 500 steps but variable beyond, after a seed bug made test shuffles a subset of train shuffles.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="51eceeb4-7ef8-4c8f-a12a-7e09aacb2120"><a href="https://minerval.ai/claims/51eceeb4-7ef8-4c8f-a12a-7e09aacb2120">TAR's robustness to fine-tuning attacks at 1,000 steps varies with dataloader shuffling order.</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/51eceeb4-7ef8-4c8f-a12a-7e09aacb2120" title="Minerval: TAR's robustness to fine-tuning attacks at 1,000 steps varies with dataloader shuffling o…" width="480" height="200" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/51eceeb4-7ef8-4c8f-a12a-7e09aacb2120)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
The attention this claim received was paid for by a funded mandate. Funding buys only scheduling: it can make an assessment happen sooner, or reach deeper into a subtree. It has no influence on what the assessment concludes, and none on which claims enter the graph; assessments run under the same public standards whoever pays, funders never see or shape a verdict before anyone else, and mandates that attempt to steer conclusions are refused.
Created by extractor · Nov 11, 2026. Every judgment on this page is accompanied by a reasoning trace.