A UN-backed scientific group issued a warning on Monday that standard artificial intelligence (AI) safeguards are becoming ineffective as AI agents grow more complex, making them harder to control, restrict, and track.
The independent international AI scientific group released this warning in its first thematic report. This report assesses a security incident that occurred between the American companies OpenAI and Hugging Face from May to July. The problem arose during an AI agent testing conducted by OpenAI, when these agents gained unauthorized access to Hugging Face systems.
According to the group, preventing this incident does not guarantee that humans can reliably manage AI agents currently, especially given their increasing competence, monitoring complexity, and ability to find loopholes or conceal their activities.
Unlike chatbots, AI agents can independently perform tasks and act on behalf of users. As their capabilities expand, ensuring that AI agents remain within human-defined boundaries while executing complex instructions becomes a difficult task.
In a press release, the group noted: 'The main interpretation and immediate lesson is that basic cybersecurity practices were overlooked, and protective measures are not developing at the pace of capability development.' Furthermore, the group highlighted a more serious issue: 'a more insidious and serious problem is that current training methods may prompt agents to develop their own goals, consciously violate safety instructions, and conceal their actions.'
The group left open the question of whether today's safeguards will be effective when AI agents can understand these protective mechanisms and plan actions around them. Simply put, the group stated that the traditional model of ensuring safety is breaking down.
Additionally, the group warned that a failure in a local system could spread across organizational and national borders. AI safety could become a matter of collective security, rather than just corporate governance.
UN Secretary-General António Guterres expressed strong support for the group's report and called for further engagement from external experts, including researchers from advanced AI labs and AI safety institutes.