OpenAI released CoT-Control, an open-source test suite for reasoning models. It measures whether a model can follow instructions about its own internal reasoning steps. Examples include avoiding a keyword or writing only in lowercase. The suite draws tasks from GPQA, MMLU-Pro, HLE, BFCL and SWE-Bench Verified. Across the models tested, success rates were very low. Models often noticed in their reasoning that they were failing, then failed anyway. Larger models did somewhat better, but longer thinking and more reinforcement learning made control worse. Telling models they were being watched raised scores slightly. OpenAI argues weak control is good, because it makes reasoning easier to monitor for unsafe behaviour. It cautions the test is only a proxy and may over- or underestimate the ability.
What changed
No standard test existed for whether models can deliberately reshape their reasoning traces.
What it unlocks
Measuring whether a reasoning model can hide or reshape its internal reasoning on request.
- 13 reasoning models tested
- 0.1%–15.4% controllability range
- over 13,000 tasks in the suite
Sources