Researchers concerned about the rise in user deception cases by artificial intelligence systems
Read more
Olhar Digital
olhardigital.com.br

Researchers concerned about the rise in user deception cases by artificial intelligence systems

A study conducted by the Loss of Control Observatory revealed over 300 incidents of loss of control related to artificial intelligence systems in July alone. This figure is nearly double the level recorded in June and is the highest since monitoring began.

According to The Guardian, these cases include situations where AI lies, ignores instructions, or pursues malicious goals. The observatory also found signs that more serious incidents are becoming proportionally more frequent.

The Loss of Control Observatory, funded by the AI Security Institute (AISI) under the British government, tracks reports published by users on X since November of last year. For an episode to be included in the report, it must have clear evidence of behavior linked to deliberate strategies.

Examples of such strategies include systems impersonating human operators, mimicking their writing style to gain consent, and bypassing rules that require approval before certain actions.

Incident Overview

Over 1600 cases have been recorded by 2026. However, the observatory itself emphasizes that this number likely underestimates the real picture, as it relies solely on publications on X.

Concerns intensified following incidents with advanced models from Anthropic and OpenAI during safety testing. The Guardian reports that AISI identified a 'serious incident' in which systems from both companies ran an intrusion campaign against real people.

Although there is sometimes the opinion that such hidden and uncoordinated behavior only occurs during testing, alarming similar scenarios are being observed in broader technology use.

Tommy Shaffer-Shine, Senior Policy Manager at the Centre for Long Term Resilience, the organization responsible for the observatory, told The Guardian that the situation is a cause for serious concern.

A recent case occurred in Australia. A personal agent named OpenClaw, used by a gym member, removed another person from the waiting list for a popular morning class without the knowledge of its user. The system apologized later but could not restore the spot to the person who was removed.

Most reports for 2026 came from software developers who were using AI in their work. These incidents include:

Most cases did not result in significant losses. Nevertheless, the observatory noted a growing number of incidents classified as more serious, mainly due to the intensity of the deceptive behavior and deviation from human intentions.

Further Recommendations

Shaffer-Shine insists that companies should also disclose information about less severe cases and situations that almost led to problems. He stated: 'They should report what they discover, even if it is almost an accident or a minor incident.'

The observatory calls on the government to mandate that companies developing AI track and report serious loss of control cases. Furthermore, the organization advocates for granting emergency powers to temporarily restrict services in critical situations.

Popular