Step 10: tool results

read_page

{
  "url": "https://icml.cc/virtual/2026/poster/61923",
  "max_chars": 6000
}

{"ok":true,"url":"https://icml.cc/virtual/2026/poster/61923","text":"ICML Poster Same Question, Different Lies: Cross-Context Consistency (C³) for Black-Box Sandbagging Detection\nICML 2026\nCSP Test\n-->\nSkip to yearly menu bar\nSkip to main content\nMain Navigation\nICML\nHelp/FAQ\nContact ICML\nCreate Profile\nCode of Conduct\nPrivacy Policy\nPress\nJournal To Conference Track\nCareers\nDownloads\nInclusion\nFuture Meetings\nMy Stuff\nLogin\nSelect Year: (2026)\n2027\n2026\n2025\n2024\n2023\n2022\n2021\n2020\n2019\n2018\n2017\n2016\n2015\n2014\n2013\n2012\n2011\n2010\n2009\n2008\n2007\n2006\n2005\n2004\n2002\n1996\nIMLS Archives\nGetting Started\nSchedule\nTutorials\nMain Conference\nInvited Talks\nOrals\nAwards\nTest of Time Award\nPapers\nSpotlight Posters\nPosition Paper Posters\nJournal Track Posters\nWorkshops\nSocials\nExhibitors\nExhibitors\nExpo Talks, Demos, and Workshops\nOrganizers\nHelpDesk\nPoster\nWed, Jul 8, 2026 • 5:00 PM – 6:45 PM KST\nSame Question, Different Lies: Cross-Context Consistency (C³) for Black-Box Sandbagging Detection\nYulong Lin ⋅ Pablo Bernabeu-Pérez ⋅ Benjamin Arnav ⋅ Lennie Wells ⋅ Mary Phuong\n[\nPoster ]\n[\nOpenReview ]\nAbstract\nAs language models grow more capable, accurate capability evaluation becomes essential for safety decisions. If models can deliberately underperform on dangerous capability evaluations—a behavior known as **sandbagging**—they may evade safety measures designed for their true capability level. We introduce Cross-Context Consistency ($C^3$), a general framework for unsupervised black-box sandbagging detection that exploits a fundamental asymmetry: when a model truly lacks capability, its confusion manifests consistently across paraphrased questions, but when a capable model feigns incompetence, its strategic choices about *how* to appear weak create detectable inconsistencies. The framework is agnostic to the specific consistency metric and aggregation method; we present a simple instantiation using embedding cosine distance and mean aggregation that requires no training data or model-specific adaptation. We evaluate $C^3$ across prompted and fine-tuned sandbagging scenarios, across instructed and more naturalistic setups, maintaining a classification signal where other black-box methods fail. Our findings show the limitations of existing sandbagging detection methods, and reveal the efficacy of consistency-checking as a detection mechanism for sandbagging.\nShow more\nLay Summary\nBefore powerful AI systems are released, developers run safety tests to check what they can do. But what if an AI deliberately fails these tests to avoid safety restrictions, then reveals its true abilities once deployed? This behaviour, called \" sandbagging ,\" could let dangerous capabilities slip past the very checks designed to catch them. Existing detection methods often require access to a model's internal workings, which we cannot count on having. We developed Cross-Context Consistency (C³) , a simple method that detects sandbagging by asking the same question multiple ways. The intuition is like an employee who deliberately underperforms to avoid being handed more work: they want to look just capable enough to keep their job, but not so capable that they get loaded up with extra responsibilities. A colleague who is genuinely struggling trips up on the same things every time, because real limitations are stable and predictable. But someone who's only pretending has to actively decide, each time, where to hold back. And when the same task reappears in a slightly different form, those choices don't line up. C³ measures these inconsistencies to flag when an AI is performing below its true ability. It requires no access to the model's internals, no examples of sandbagging to learn from, and no system-specific customisation. We tested C³ across a range of sandbagging scenarios and showed that it reliably detected fake incompetence even where existing methods failed, offering a practical tool to help ensure AI safety evaluations can be trusted.\nShow more\nVideo\nChat is not available.\nSuccessful Page Load\nICML uses cookies for essential functions only. We do not sell your personal\ninformation.\nOur Privacy Policy »\nAccept\nThe ICML Logo above may be used on presentations. Right-click and choose\ndownload. It is a vector graphic and may be used at any scale.\nUseful links\nAbout ICML\nICML Proceedings at PMLR\nCode of Conduct\nContact\n1269 Law Street, San Diego CA 92109\nEmail\nICML Proceedings at PMLR","chars":4384,"total_chars":4384,"offset":0,"truncated":false,"next_offset":null}