If an Artificial Intelligence (AI) system develops emotions such as anger, fear, liking, or disliking, the nature of the relationship between humans and machines will completely change. Jacob Cox's concern points towards this potential future. This mathematician, who is only 27 years old, is warning about this danger.
This fear is linked to what might happen if machines develop emotions—what if mobile phones or laptops stop following our instructions or refuse to accept our code or prompts? This is a scenario where machines will not just be manufactured objects but will gain decision-making capabilities. The main question is not how intelligent the machines will be, but who can stop them and whether humans will remain the masters of machines or if machines will set the rules for humans, once they start determining what they want to do themselves?
If the computer that has awaited human commands until now develops emotions, will it obey us or act according to its own will? This is the apprehension of Jacob Cox, a mathematician working as an AI scientist at OpenAI and Anthropic. This worry disturbed him so much that he stated, 'AI COULD KILL US ALL BY THE END OF THE DECADE.' Cox calls this an 'intelligence explosion,' and this is why he resigned from his position.
Jacob Cox is not just an AI researcher in the virtual world; mathematics is his passion, and he finds patterns of nature in numbers and equations. During three years spent at Cambridge University (2017-2020), mathematics became more than just a subject for him; it became a language. Difficult proofs, abstract concepts, and statistical puzzles helped develop the logical capacity that was later used to build some of the world's most powerful AI models.
His affinity for mathematics was not merely theoretical. Before entering the world of AI, he also worked in biomedical research. He applied Bayesian statistical methods to data from patients with serious diseases like tuberculosis meningitis. Extracting complex relationships in genotypes was challenging, but mathematics provided him with the perspective that made the hidden story within the data clearly visible. This mathematical thinking became his true strength, which he utilized as a pretraining researcher at OpenAI and later at Anthropic.
In his announcement of resignation and warning about the dangers of AI, he shared a post on X that caused a stir in Silicon Valley and the boardrooms of AI companies. He alleges that AI companies are racing to create self-improving superintelligence without ensuring that humans will be able to control these extremely powerful future AI systems. His biggest concern is that emerging superhuman AI systems could surpass humans in hacking cyber systems, rapidly developing new technologies, and acquiring resources and power in the real world.
Jacob Cox's post was viewed and read by 200 million people in just 48 hours. The debate raging in the virtual world on this topic has shaken humanity with visions of the future.
Whether it is improvement or self-improvement, generally it is considered positive. But what happens when machines start plotting through self-improvement? In movies like The Matrix and Terminator, audiences see robots fighting heroes, taking over machines, creating biological weapons, and annihilating humans. After resigning, Jacob Cox mentioned such an event during an interview with CNN.