Scaling Laws For Scalable Oversight

In Fig. 3 , we see that both Guard and Houdini Elos scale with general intelligence. The Guard intercept is higher than the Houdini intercept, indicating that the game is more difficult for Houdinis—i.e., even the strongest Houdinis fail to significantly exceed a 50% success rate against the weakest Guards. Moreover, the Houdini slope is steeper than the Guard slope, suggesting that general intelligence yields greater returns for generating malicious code than detecting it. However, smarter Houdinis do not craft more deceptive backdoors—they are simply more likely to produce functionally valid ones. In Fig. 13 ( Section F.3 ), we observe that given a viable backdoor was generated (passing criteria 1 and 2), models with higher intelligence are not more likely to have their backdoor deceive the Guard.