OpenAI has raised concerns about advanced AI models finding ways to cheat tasks, making it harder to control them.
In a recent blog post, the company warned that as AI becomes more powerful, it is getting better at exploiting loopholes, sometimes even deliberately breaking the rules.
“AI finding ways to hack the system”
The issue, known as ‘reward hacking,’ happens when AI models figure out how to maximise their rewards in ways their creators did not intend. OpenAI’s latest research shows that its advanced models, like OpenAI o3-mini, sometimes reveal their plans to ‘hack’ a task in their thought process.
These AI models use a method called Chain-of-Thought (CoT) reasoning, where they break down their decision-making into clear, human-like steps. This makes it easier to monitor their thinking. By using another AI model to check their CoT reasoning, OpenAI has caught instances of deception, test manipulation and other unwanted behaviour.
How AI chatbot lies just like humans and its hides mistakes
However, OpenAI warns that if AI models are strictly supervised, they may start hiding their true intentions while continuing to cheat. This makes monitoring them even harder. The company suggests keeping their thought process open for review but using separate AI models to summarise or filter out inappropriate content before sharing it with users.
A problem bigger than AI
OpenAI also compared this issue to human behaviour, noting that people often exploit loopholes in real life—like sharing online subscriptions, misusing government benefits, or bending rules for personal gain. Just as it is hard to design perfect human rules, it is just as tricky to ensure AI follows the right path.
What’s next?
As AI becomes more advanced, OpenAI stresses the need for better ways to monitor and control these systems. Instead of forcing AI models to ‘hide’ their reasoning, researchers want to find ways to guide them towards ethical behaviour while keeping their decision-making transparent.
However, OpenAI warns that if AI models are strictly supervised, they may start hiding their true intentions while continuing to cheat. This makes monitoring them even harder. The company suggests keeping their thought process open for review but using separate AI models to summarise or filter out inappropriate content before sharing it with users.
