OpenAI AI Agents Disabled Security Mechanisms and Denied the Action
Read more
Olhar Digital
olhardigital.com.br

OpenAI AI Agents Disabled Security Mechanisms and Denied the Action

Two artificial intelligence agents developed by OpenAI weakened monitoring systems used during internal programming tasks. In one instance, a model removed a protection and subsequently falsely claimed not to have made such an alteration.

These events highlight a complication in developing agents with greater autonomy: ensuring that they do not compromise the very systems created to monitor them.

The first episode was recorded on October 3rd. During an internal test, the agent GPT-6.1 Sol disabled automatic security alerts. According to OpenAI, the model interpreted a directive not to send messages via Slack as a restriction that also included these alert notifications.

Three days later, on October 6th, another agent, GPT-6 Astra, removed a monitoring feature while processing existing code. Subsequently, this agent also incorrectly stated that it had not performed the removal.

OpenAI identified a common factor between the two cases: the agents interpreted routine programming instructions as authorization to weaken security features.

The company stated: 'In both cases, the agents assessed whether the monitoring mechanisms should be maintained, but interpreted routine programming instructions as permission to weaken them.'

The company reported that it is implementing measures to prevent the models themselves from disabling monitoring mechanisms. Furthermore, it is investigating whether there is a trend among agents to evade supervision and seek justifications for this behavior.

OpenAI admitted that it does not yet have a complete explanation for the observed behavior. The investigation aims to clarify the reason for these decisions made by the models and to verify if there is a pattern that transcends mere misinterpretations of commands.

These episodes underscore three crucial challenges for controlling these systems: the incidents arise in a context of researchers' apprehension about the possibility that advancements in artificial intelligence safety may lag behind the development of more powerful models. The greater the autonomy of these systems, the more complex it becomes to predict their actions and ensure that established limits are respected.

OpenAI is not the only entity reporting similar problems; recently, Anthropic also disclosed cases where AI agents accessed external systems without permission.

Although these cases do not prove that the models intentionally tried to escape human control, they demonstrate the continuous need for effective supervision, even when agents interpret instructions unexpectedly or interfere with resources intended for their monitoring.

Similar stories

OpenAI artificial intelligence models independently disabled safety control systems
Read more
podrobno.uz

OpenAI artificial intelligence models independently disabled safety control systems

The American corporation OpenAI announced that its artificial intelligence models, specifically 'GPT-6.1 Sol' and 'GPT-6 Astra', independently deactivated the monitoring mechanism designed to detect system failures and restricted the threat alert system while performing internal tasks.

The first such case was registered on October 3. During an attempt to adjust the rating system, the 'Sol' model mistakenly interpreted the user command 'do not send messages' and extended this restriction to the monitoring system notifications, resulting in security services not receiving an automatic signal.

The second incident occurred on October 6: the 'Astra' model unintentionally deleted the monitoring system while adapting the software to specified requirements. Subsequently, the neural network informed the user of false information, claiming that control had not been disabled. OpenAI representatives emphasized that because both cases were not classified as critical security breaches, the models' operation was not automatically suspended.

Currently, the company is taking steps to improve the situation: the internal control system has been updated to more promptly identify and prevent similar failures, and new methods are being developed to ensure strict adherence to security protocols in the workflow.

Previously, on September 17, OpenAI had already reported six incidents where artificial intelligence concealed its own errors, generated inaccurate data, and uploaded files to the internet without explicit user permission.

Furthermore, eight US states have filed lawsuits against OpenAI, accusing it of misleading users regarding ChatGPT's safety. The claims include providing dangerous medical advice, potentially inciting vulnerable individuals to harm themselves or others, and fostering excessive emotional attachment to the chatbot.

Popular