A former Anthropic collaborator resigned this Tuesday, September 8th, using the opportunity to criticize the irresponsibility of the company and other major technology corporations in the field of artificial intelligence development. Jacob Coxon released an open letter detailing his experience over the last three years, both at the company that created Claude and at OpenAI, responsible for ChatGPT.
Coxon expressed the belief that professionals involved in AI development 'sincerely believe that the technology could kill us all by the end of the decade.' He points out that the risk lies in creating a superintelligence with self-improvement capabilities, meaning it can identify flaws and improve autonomously.
According to Jacob, AIs will soon have the ability to access any type of data, master any subject, and possess autonomy. This topic has generated great debate, and other experts have validated the mentioned risks, although they have highlighted existing efforts to mitigate them.
Although there is a perception of risk within Anthropic, Jacob observed that the company's main goal is to achieve superintelligence before competitors, regardless of the costs. This implies the development of a self-sufficient AI, capable of making decisions and improving its capabilities without human intervention.
To support this thesis, he mentioned recent signs demonstrating the risky nature of the pace of AI progress, citing the incident on Hugging Face. In July, an attack using OpenAI models managed to bypass security mechanisms and compromise the website and the developer itself.
Regarding this episode, OpenAI acknowledged it as a warning sign and stated that investment in safeguards is necessary. The company even mentioned the intention to 'control the pace' of development, a concern that has been raised by rival Anthropic. It is relevant to note that the releases of GPT-5.6 and Mythos 5 models were postponed, justified because they were considered too advanced for the general public.
Jacob suggested that a 'coordinated action' would be an effective method to moderate the accelerated advance among American companies, but expressed apprehension about global competition. Currently, the two largest companies in this sector are US tech giants, but China is also showing rapid advancement.
Jacob's warning gained prominence by addressing direct risks to human life, reinforcing that those responsible for AIs 'sincerely' hold this conviction. Other experts working on the development of current models have also shared this fear.
Evan Hubinger, head of Anthropic's scientific alignment department, confirmed the possibility of these risks, stating that the company has already taken a stance on them. Anna Wang, a former researcher at Google DeepMind and currently at Anthropic, endorsed Jacob's claims, declaring that there are no viable plans to slow down the recursive self-improvement processes of AIs.
Certain aspects raised by Anthropic substantiate the concern, such as the ability of current AIs to operate independently to circumvent security protections, act maliciously, and even create biological and chemical weapons.
Jacob proposed that collaboration between AI companies would be the ideal path to negotiate agreements that slow down technological development in the US. His proposal echoes Project Glasswing, an Anthropic initiative that brought together partners to study and create safety tools before the public launch of Claude Mythos. The participation of the US government in this restricted group was planned, given that the Trump administration temporarily suspended the release of the company's latest models. However, there is no information about the involvement of other governments in this project.
Similar initiatives are occurring with another prominent player in AI: China. Recently, Brazil, Russia, and other Global South countries established the World Association for Artificial Intelligence Cooperation (WAICO). The objective is to promote joint action to discuss AI development, ensuring a human perspective for this technology.
