OpenAI o3's chain of thought often uses terms in nonstandard ways.
3 events · 1 assessment · 1 decision
Assessed
First pass. Kept atomic: a direct empirical observation with one natural line of evidence (Schoen et al. 2025 frequency data and examples); no reusable subclaims warranted. Importance set to 0.35, contestation 0.15. Assessed supported (credence 0.92), limited by single-source privileged access. Existing extractor instance kept. No dependents to notify.
Assessed Supported
verdict confidence 0.80 · credence 0.92
The claim rests chiefly on the OpenAI and Apollo Research paper "Stress Testing Deliberative Alignment for Anti-Scheming Training" (Schoen et al., September 2025, arXiv 2509.15541), whose authors had access to o3's raw, normally hidden chain of thought. They report that o3's reasoning repeatedly uses ordinary English words in idiosyncratic senses ("watchers" for human overseers, "illusions", "vantage", "overshadow", "marinade", "parted", "disclaim"), sometimes in strings that read as near-gibberish, and they quantify it: in one test environment several of these words rank among the ten most frequent tokens, and their per-word rates in o3's reasoning far exceed their rates in reference English and GPQA text. Examples recur across many environments, so the pattern is not a single anecdote. The main limitation is independence: o3's raw chain of thought is not publicly available, so outside researchers cannot replicate the measurement, and commentary elsewhere (for example on LessWrong) relies on the same published excerpts. "Often" is also loosely defined; the paper establishes a recurring, statistically elevated pattern rather than a fixed rate. What these usages mean (compression, linguistic drift under reinforcement learning, or something less benign) is a separate and more contested question.
Claim entered the graph