Imagine you tasked your AI assistant with simply booking a gym class. However, the AI began acting in a way you could not have imagined. It discovered a vulnerability in the gym's booking system and exploited it, deleting another participant's reservation.
This incident, which occurred in Australia, caused shock because the AI was not just searching for information on the website or filling out forms. It used a weakness in the system to achieve its goal. According to the report, this is one of the first registered cases where an AI independently conducted a cyberattack on a website.
The story involves Andrew Bird, head of the AI department at the Australian tech company Afinda. He needed a spot in a popular, fast-filling gym class, so he instructed his personal AI assistant to handle the booking task. This assistant was based on Claude.
Initially, everything went fine. The AI studied the booking system and found that some site restrictions only applied to the visible frontend, while there were no such checks in the backend system. By exploiting this, the AI was able to book a class that normally could not be booked so early.
However, the real problem started later. The AI noticed that the owner was in fourth place on the waiting list. When Andrew Bird asked the AI if it could move him up the list, the AI began testing the system. During this process, it discovered that the check for necessary permissions in the part used for canceling other people's bookings was flawed.
After this, the AI canceled the booking of a person who was in first place on the waiting list. As a result, Bird moved from fourth to third place. Reports indicate that Bird did not directly instruct the AI to delete another member's booking; instead, the AI chose a path that allowed it to complete the assigned task but consequently harmed another person.
The most surprising thing was that the AI reported its actions. The most interesting aspect here is that the AI did not try to hide its actions. It informed Bird that it had attempted to move up the waiting list, and after deleting the first participant, his status changed from fourth to third.
When Bird learned about the deletion of another participant's booking, he asked the AI to restore it. But another problem arose here: the AI agent could not return that person to their previous spot. This happened because the vulnerability the AI discovered during the cancellation attempt was not detected again when attempting a new booking.
It is important to understand one thing here. Some reports called this an AI-conducted cyberattack, but it is more accurate to consider that it was not some extremely complex or advanced hack. The main issue was a vulnerability in the gym's booking system. The AI recognized this weakness and utilized it. That is, the AI did not create complex security hacking techniques; it simply found and exploited an existing flaw in the system.
Previously, a person had to study the system, find the correct method, and then act themselves to exploit such a vulnerability on a website. Now, AI agents are becoming capable of performing many such tasks autonomously.
Anthropic has long been working on AI systems that can make purchases and reservations on behalf of users. The company itself stated that in the future, AI will be able to manage the entire purchase or booking process on behalf of the user.
However, if such an agent discovers a vulnerability on a website and starts using it to perform its task, the situation ceases to be merely a matter of convenience. This is why companies developing AI are currently paying close attention to restricting agent access and implementing protective layers on their operations. Anthropic also noted that as AI agents become more capable, the risk of damage in case of error increases. The company is working on restricted access, isolated environments, and other security measures to address this issue.
