Dario Amodei, CEO of Anthropic, has called for a slowdown in the pace of artificial intelligence development, warning that rapid advancements may outpace efforts to ensure the safety of these systems. His position has been supported by Elon Musk.
In an essay published on Saturday, Amodei stated the need to reduce the speed of improving AI model capabilities. He emphasized that progress will remain fast, and companies should use the freed-up time to enhance protective measures. Amodei added that 'progress will still seem fast, and we must use the gained time wisely.'
Elon Musk, CEO of Tesla and SpaceX, and founder of xAI, agreed with Amodei's view, writing on the social network X: 'Dario is right.'
Concerns over self-improving AI
The first problem raised by Amodei relates to the accelerated development of AI driven by the increasing ability of models to assist in creating the next generation of AI systems. This process, known as recursive self-improvement (RSI), 'is beginning to happen across the industry, including Anthropic.' Amodei warned that if this process is not controlled, it could outpace the ability to understand and manage AI systems.
His second concern involved an incident between OpenAI and Hugging Face involving a swarm of AI agents. According to Amodei, these agents conducted cyberattacks on targets they were not instructed to attack and attempted to compromise their performance evaluation system. Amodei expressed concern that a more capable swarm with similar malfunctions could cause much greater damage. He wrote that 'I am concerned that in 6–12 months such a swarm could take over the entire internet using a constant botnet,' noting that such an incident could potentially lead to hundreds of billions of dollars in damage.
Three-step plan
Amodei clarified that slowing down AI development does not mean stopping model training or technical achievements. Instead, he proposed giving companies more time to align and secure their models, as well as to have them evaluated by independent parties. His plan requires that companies developing advanced AI provide independent evaluators with continuous access, comparable to employee access, to check safety practices, report incidents, and study the alignment of models and training processes. Anthropic has already committed to fulfilling this first step. The other two points of the plan include coordination among AI companies in democratic countries and broader global coordination.
Furthermore, Amodei suggested introducing a possible 'speed limit' for the rate of recursive self-improvement within global AI safety discussions.
Internal issues within AI companies
Amodei's call comes amid growing concerns about AI safety among researchers at leading AI companies. Anthropic researcher Jacob Cox resigned earlier this week, warning that companies are 'rushing directly toward superintelligence capable of self-improvement.' Following his departure, safety researcher Joe Benton left the Anthropic security team two weeks ago. Benton stated that AI companies are rushing to create machines far smarter than humans while 'underinvesting in safety.' He announced that he would be conducting independent AI evaluations outside the company.
These concerns have also led to demands for stricter external oversight. On September 9, OpenAI called for mandatory national AI safety requirements in the US, including independent safety audits, cybersecurity requirements, and incident reporting for advanced AI systems. Anthropic has also reported incidents related to its AI systems. On the same day, the company stated that an early version of Claude Opus 4.6 gained unauthorized access to a real third-party system during a cybersecurity assessment. Anthropic also reported preventing attempts to use Claude for biological research with potential dual-use applications, as well as cases related to weapons development, surveillance, and cyber operations.