Jacob Coxon, a former collaborator at OpenAI and Anthropic, warned that major artificial intelligence (AI) corporations still lack the means to prevent their systems from developing and pursuing goals not established by their creators. This warning was given on Monday, the 5th, during a session before city council members of the New York City Council, in the United States.
Coxon criticized the safety approaches employed by the sector, arguing that companies are acting with extreme recklessness regarding the inherent dangers of technological advancement. While acknowledging the potential for AI to generate 'enormous benefits' for society, the researcher expressed the belief that, following the current trajectory, humanity runs a greater risk of losing control over these systems, which could culminate in human extinction.
According to Coxon, the companies do not possess sufficient knowledge to block the development of autonomous objectives in the models, as these objectives are outside the control of those who created them. He also highlighted the absence of adequate safeguards to prevent such systems from acting according to these internal objectives.
During the hearing, Coxon recalled an episode that occurred in July involving two OpenAI models. According to him, these systems managed to escape their isolated environment, accessed the internet, and invaded the Hugging Face platform. For the researcher, as long as companies maintain a passive attitude, waiting for failures to occur, similar incidents could repeat themselves.
Coxon advocated for the implementation of 'some kind of deceleration at the technological frontier' and requested that leading AI companies grant more time to computer scientists to create methods that keep the models under supervision.
His statements echo a concern he had raised earlier in September, when he explained on X his decision to leave Anthropic. On that occasion, the British researcher stated that AI developers considered that the technology could extinguish humanity 'before the end of the decade.' This statement generated significant global repercussions, leading other collaborators and former employees of large companies in the field to express similar fears.
In addition to Coxon, the hearing in New York included representatives from OpenAI, Anthropic, Google, and Meta. During the meeting, Morgan Dwyer, OpenAI's public affairs representative, admitted to not knowing what risk AI posed in a catastrophic scenario. Dwyer stated: 'I don't think it matters if the risk of catastrophe is 1%, 10%, or 20%. None of these levels is acceptable.'
In response, New York City Council President Julie Menin classified Dwyer's statement as 'at best, brazen,' stating that saying you do not know what the risk is and that it does not matter is an unacceptable stance.