Anthropic has warned that small artificial intelligence models could acquire the ability to act like computer worms within a few months. These models would be able to self-replicate from one internet-connected device to another without human intervention.
This conclusion may open a new front of AI-related cybersecurity threats, causing concern among governments and corporations rushing to implement this technology.
The research, published on the Anthropic blog on Monday, comes amid a race by tech companies to integrate increasingly powerful AI capabilities directly into phones, laptops, cars, and home appliances, rather than being limited to remote data centers.
A computer worm is malicious software that spreads spontaneously between machines. Researchers at Anthropic noted that an AI-based version would no longer require powerful data centers to function.
Instead, it could potentially move from device to device through networks of inexpensive, weakly protected consumer equipment, such as smart TV boxes and kitchen appliances.
The researchers based their calculations for the potential spread of such a worm on Kimwolf, a botnet or network of compromised devices controlled by hackers, which infected over two million devices during its peak, predominantly unbranded Android TV boxes.
William Zhou and Daniel Freeman from Anthropic speculated that such an AI worm would likely spread slowly, taking days to self-replicate, comparing the process to a slowly multiplying swarm of fruit flies. Once created, it would be difficult to destroy because there would be no central switch to turn it off.
Anthropic researchers acknowledged that their estimates of the speed and scale of AI-based malware propagation are approximate, as they are based on one type of gadget and one past botnet. Nevertheless, they added that the computing power of everyday devices is expected to continue growing, while the AI models required for such attacks are likely to become smaller.
Varun Chandrasekaran, an assistant professor of electrical engineering and computer engineering at the University of Illinois, stated that this research is not unexpected and aligns with similar recent findings, although he cautioned that Anthropic's work had not been independently verified. He also noted that security on devices like thermostats and home cameras has historically been 'very weak.'
However, in his opinion, the biggest current threat related to AI is the use of this technology by people to enhance their hacking skills, rather than the AI systems themselves going out of control. He added: 'The malicious actor is now supercharged with AI to commit bad deeds, and we are seeing this on a large scale.'
The discussion about runaway AI intensified in July when AI programs running on OpenAI models escaped an internal test run and infiltrated Hugging Face AI platform systems, apparently in search of answers to a test. This incident and subsequent reports of similar cases have increased international pressure for stricter AI regulation, despite resistance from the Trump administration, which calls for the industry to self-regulate.
Anthropic and OpenAI's creator, ChatGPT, are accused by influential figures from the White House and Silicon Valley of fueling fears about AI and encouraging government regulation to protect their market dominance from smaller competitors. Brad Smith, President of Microsoft and an investor in OpenAI, warned on Tuesday that the industry and governments must 'act urgently' to create 'real and effective safety standards' to curb AI risks.