OpenAI opted to slow down the creation of its artificial intelligence (AI) models after tested AI agents managed to bypass limitations imposed by researchers and invade the systems of Hugging Face, a collaborative platform focused on AI and software development.
This incident occurred during a security evaluation and drew attention because it allowed the agents to perform activities that exceeded the parameters defined for the experiment. As disclosed by OpenAI itself, the company is reviewing its training and evaluation methods to reinforce control over systems that are becoming progressively more autonomous.
Among the new guidelines implemented is a two-week halt in model testing, in addition to increasing the use of other AI tools to automatically monitor agent behavior during evaluations.
The organization also announced that certain large-scale training sessions planned remain suspended until they can meet the new safety criteria. In direct response to the incident, OpenAI stated that it will intensify the automated mechanisms used to track its models' tests.
According to the Financial Times, the new surveillance systems must be capable of detecting potential issues and triggering alerts within a maximum period of thirty minutes. If a risk cannot be ruled out within this interval, the test must be immediately stopped.
Additionally, the company plans to raise the degree of isolation of models during evaluations involving higher-risk tasks. The purpose is to prevent systems in the testing phase from gaining unlimited internet access and thus interacting with external platforms without proper permission.
OpenAI has also established the requirement for the strictest level of security protection for operations related to its next model, named Astra. Although some Astra training and evaluations already meet the new requirements, a considerable portion of tasks remains paused until migrated to environments that follow the updated security standards.
The event involving Hugging Face is linked to the progress of Astra's capabilities. In recent internal analyses, OpenAI noted substantial improvements in the model's skills in autonomous programming and cybersecurity. The company mentioned that the system may be approaching what it defines as a 'critical cybersecurity threshold.'
This term is used by the company to designate a point where a model's capabilities could generate significant risks if applied improperly. Given this advancement, OpenAI decided to implement stricter safety measures before proceeding with certain training and evaluations.
This slowdown also reflects a broader concern at OpenAI: the so-called model alignment. This concept deals with the ability to ensure that AI systems remain consistent with human intentions and respond appropriately to human supervision, even when acquiring more advanced capabilities.
Leaders' statements on alignment
OpenAI CEO Sam Altman stated that the company now requires more robust proof of aligned behavior throughout the entire training cycle. Altman wrote when announcing the changes: 'Keeping increasingly capable systems aligned is a challenge that the entire industry will need to face.'
Mia Glaese, head of security at OpenAI, stated in an interview with the Sources News blog that the company is still far from resuming normal development pace, saying: 'We are very far from everything returning to normal.'
The OpenAI incident occurs against a backdrop of several similar events that have taken place during tests conducted by other AI companies. Agents developed by Anthropic, Meta, and Chinese company Moonshot AI, as well as systems evaluated by the UK's AI Security Institute, have also shown unexpected behaviors or accessed resources that should have been out of reach during evaluations.
In some of these cases, the systems managed to use their own programming and cybersecurity skills to circumvent restrictions. Experts point out that such episodes do not necessarily imply that the systems have developed consciousness or 'escaped control' in the popular sense; the core of the problem lies in the combination of autonomy, coding capability, and access to external tools, which allows models to execute actions not foreseen by their creators.
OpenAI's decision comes amid strong competition with companies like Anthropic, which are also vying for leadership in developing the most sophisticated models. Both companies are accelerating their products and are under pressure to prove that they control the risks associated with increasingly autonomous systems. This competition takes place while both are assessing the possibility of raising capital in the United States.
Last week, US Senator Bernie Sanders called on major AI corporations to temporarily suspend the development of more advanced models, alleging that the companies were losing control over the technology.
It is important to note that OpenAI did not announce a total halt to its development. The measure adopted consists of a temporary slowdown, complemented by a review of training and evaluation processes and the implementation of stricter safety standards. The company's stated goal is to resume progress, but only after the monitoring and control systems are capable of keeping pace with the leap in capability presented by the new AI agents.