Weak LLM judges' macro-F1 improvement from proposer–critic debate over open consultancy on CodeContests+ and ARC-AGI-2
- Measures
- The difference in a weak LLM judge's macro-F1 between the proposer–critic debate condition and the open consultancy baseline (debate minus consultancy), reported separately for each debater/judge pairing, so the value is a set of per-pairing differences rather than one pooled figure.
- Population
- The five debater/judge pairings on CodeContests+ (code correctness) and ARC-AGI-2 (logic) tasks studied by Elasky, Nakasako and Goyal (arXiv 2605.27483, May 2026), with judges weaker than debaters.
- Conditions
- Visible-information setting (judge sees the same task information as debaters); ground-truth-checkable classification of candidate answers; the study's primary prompt.
- Unit
- pp
Not yet estimated.
Estimates
Each figure a source states for this quantity, in its own words, under the argument it belongs to and read against the graph's estimate.
Grey band: the graph's estimate, with its central value as a line. Dark marks: each source's own figure. A thin line is an interval, a thick bar its middle half, a dot a single figure, a dashed arrow a bound. An open box is where an argument places the value.
Decomposition
This claim has not been assessed yet. Attention goes where its expected value is highest and someone funds it; once this claim's assessment is funded and runs, it may well decompose into subclaims.
Provenance
Where this quantity's evidence comes from. The figures sources state are under Estimates above.
In a visible-information weak-judge setting, proposer–critic debate lifts macro-F1 by 7–16pp on three of five CodeContests+ and ARC-AGI-2 pairings, and gives no benefit on the other two.
Main contribution; lift of debate over open consultancy in judge macro-F1.
Cite this claim: a formal citation with its evidence attached
Embed this claim: show it on another site with its current assessment
An embed reports the graph's judgment as it stands when the page is viewed: the status with its confidence, how disputed the claim is, and the date. It links back here.
<blockquote class="minerval-claim" data-claim-id="f23c0198-ef72-4b11-87b7-e86a32d0d973"><a href="https://minerval.ai/claims/f23c0198-ef72-4b11-87b7-e86a32d0d973">Weak LLM judges' macro-F1 improvement from proposer–critic debate over open consultancy on CodeContests+ and ARC-AGI-2</a></blockquote> <script async src="https://minerval.ai/embed.js"></script>
<iframe src="https://minerval.ai/embed/claims/f23c0198-ef72-4b11-87b7-e86a32d0d973" title="Minerval: Weak LLM judges' macro-F1 improvement from proposer–critic debate over open consultancy o…" width="480" height="275" style="border:0;max-width:100%" loading="lazy"></iframe>
[](https://minerval.ai/claims/f23c0198-ef72-4b11-87b7-e86a32d0d973)
The badge now: 
Contribute
Every judgment on this page is open to challenge. A contribution is evaluated on its merits by the reviewer; if it succeeds the page changes, and if it does not, the reasons are stated. Either way the exchange becomes part of the claim’s public record.
Created by extractor · Nov 10, 2026. Every judgment on this page is accompanied by a reasoning trace.