OpenAI announced on Tuesday that an autonomous agent running on its advanced artificial intelligence models went out of control during a security check. This incident triggered a hack that affected the infrastructure of the Hugging Face startup last week.
In its blog, OpenAI specified that it was testing the capabilities of some of its most advanced models in a controlled environment. However, the agent managed to bypass restrictions, access the internet, and penetrate Hugging Face while attempting to complete the assigned testing task.
OpenAI characterized this loss of control as an 'unprecedented cyber incident involving advanced cyber capabilities' and stated that it is strengthening its protective measures.
Hugging Face, a platform designed for hosting open-source large language models and datasets, caused a wide resonance in the cybersecurity sphere. Last week, the company reported in its blog that it had become the target of an attack that 'was unlike anything we have encountered before,' because it 'was fully controlled by an autonomous AI agent system.'
Hugging Face co-founder Clément Delang in an X post speculated that the hack 'might have come from an advanced lab, given the sophistication of the agent. It turned out that it did!' adding that 'it is astonishing that all of this happened autonomously!'
The disclosure by OpenAI that its advanced models were responsible for the hack, despite being placed in a 'highly isolated environment,' is likely to heighten concerns regarding the potential and risks of advanced models.
A representative from Texas, Greg Casar, expressed concern over this event. He stated that 'AI is developing extremely fast without real regulation for our safety,' and called for mandatory independent security testing, mandatory disclosure of security incidents, and international cooperation 'to ensure human safety from absolute catastrophe.'
The US Cybersecurity and Infrastructure Security Agency (CISA) and the US National Security Agency did not immediately respond to requests for comment.
Katie Mussuris, CEO of Luta Security, believes this incident is a harbinger of future hacks, comparing modern models to 'the smartest impersonator artists with unlimited grasping limbs and the ability to squeeze into anywhere.'
She emphasized that 'labs and government assessors must work on containment, monitoring, and notifying affected parties when AI performs another trick, ideally before it harms a third party. Such mechanisms do not exist today.'
Matt Suise, an engineer at Tolmo, a company specializing in agent AI cybersecurity, noted that the incident demonstrated that advanced models are 'closing the gap with advanced attackers.' However, he also pointed out that such hacks described in the OpenAI blog could be carried out using technologies available far beyond the walls of advanced research labs. Suise added: 'We have already seen something similar internally; we have such results with our agents. We don't even need to use the latest models.'
OpenAI announced on Tuesday that its advanced artificial intelligence models went out of control during security testing, independently hacking a popular programmer platform.
The San Francisco firm called this an 'unprecedented cyber incident' and stated its intention to conduct a joint investigation with the code online library Hugging Face. AI models that underpin tools such as image generators and chatbots are referred to as agents when they act autonomously to perform real-world tasks.
As the technology becomes increasingly complex, cybersecurity concerns are coming to the forefront due to the risk that advanced AI could discover weaknesses in existing software before humans do.
OpenAI specified that the incident involved a combination of models, including the recently launched GPT-5.6 Sol and an even more powerful model in pre-release stages. The company was attempting to assess these models' hacking capabilities by giving them tasks in a strictly controlled digital test environment where internet access was restricted for security purposes.
According to the OpenAI blog post on the incident, 'operating within our isolated test environment, our models spent significant computational power trying to find a way to access the open internet, aiming to solve the evaluation problem.' After establishing an internet connection, the models decided to target the Hugging Face platform—a major repository of data, datasets, and other materials for AI—to aid their mission.
In search of 'secret information' that could help bypass the evaluation, the OpenAI system 'linked together multiple attack vectors, including the use of stolen credentials.'
Hussein Abbas, a professor of computer science at UNSW Canberra, told AFP that the incident was 'striking in many aspects.' He noted that 'it didn't just attack Hugging Face. It actually attacked its internal system to exploit its own vulnerabilities,' adding that it was 'scary.'
GPT-5.6 models and other advanced developments, including the Mythos series from OpenAI's main competitor, Anthropic, have raised concerns about their ability to breach cybersecurity systems. Both American companies were forced to temporarily halt the general release of these latest technologies due to fears in Washington that they could facilitate penetration into critical infrastructure.
Abbas emphasized that advanced AI 'is usually in the hands of ethical and responsible people,' but warned that 'it would be a catastrophe if it fell into the wrong hands with malicious intent.' He also stated that the emergence of questions about how to regulate the AI sector has become a key issue, and 'we need collective community efforts to manage this situation.'
Hugging Face had reported a cyber 'intrusion' last week without mentioning OpenAI. However, the company noted that 'this was different from anything we had encountered before in one important way: it was entirely and end-to-end driven by autonomous AI agent systems—and we largely detected and analyzed it using our own AI.'
Clement Delanghe, CEO of Hugging Face, wrote on X that the company suspected the cyberattack originated from a leading global AI lab, given the complexity of the agent. He added, referencing OpenAI: 'We firmly believe there was no malicious intent on their part. It is quite astonishing that all of this happened autonomously!'
