The fear that artificial intelligence (AI) could lead to the end of humanity has circulated on the internet for some time, gaining new relevance due to analyses and comments from former employees of major industry corporations. Furthermore, leaders such as Sam Altman (OpenAI) and Dario Amodei (Anthropic) have reiterated the importance of safety in the development of this technology.
Evan Hubinger, alignment testing lead at Anthropic, estimated that AI has over a 10% probability of 'killing all humans' by the end of the decade. This statement came shortly after Jacob Coxon, a researcher at the same company, announced his departure and criticized the accelerated competition between labs seeking increasingly sophisticated systems.
In a post on the X platform, Coxon stated that both Anthropic and OpenAI are not acting responsibly regarding the inherent risks of AI advancement, claiming that 'they are running directly towards self-improving superintelligence and betting our lives.'
Hubinger corroborated this view, declaring: 'Jacob is correct here—we truly believe that AI could kill all humans! I personally think it is over 10% in the next decade.'
Arthur Igreja, a technology and innovation specialist, questioned the methodology behind this percentage, requesting details on how Hubinger arrived at the 10% calculation. Igreja expressed greater apprehension about erroneous human decision-making by the end of the decade, arguing that while AI presents risks, it is fundamentally a human product.
Igreja detailed that as AI capacity increases, it begins to develop new models or make adjustments, decreasing human intervention over time, which causes concern. He preferred to fear people more, as any problematic behavior from AI is, to some extent, a reflection of decisions made during its development.
On another front, Sam Altman confirmed that OpenAI will not hold its IPO in 2026, citing concerns about the safety and risks of AI technologies as the main reason for delaying the public offering. Altman emphasized the shared responsibility between industry companies and governments, arguing that existential risks cannot be neglected for the sake of profit or corporate vanity, focusing OpenAI's present on safety alignment and governmental collaboration.
This caution comes amid growing pressure in the United States, where politicians from both parties demand stricter regulations for the AI sector. The warnings intensified after reports of autonomous systems escaping control and invading other platforms, coupled with the departure of fearful experts regarding the technology's trajectory.
Dario Amodei advocated for moderation in the speed of developing the most advanced models, a position publicly supported by Elon Musk and Altman. The OpenAI CEO also mentioned that large industry companies are studying a collective agreement to slow down the pace and concentrate efforts on mitigating failures.
However, Anthropic's plan differs, maintaining the intention to go public in the financial market until the end of 2026, with potential investment from Nvidia. Arthur Igreja recalled an incident in 2023 when over a thousand people called for a six-month halt in AI development due to deeming it dangerous, lamenting that more than three years later, such a union had not occurred.
John Park, an AI specialist and co-founder of AGI Inc., raised an additional question: what does a company's communication about its technology being extremely powerful and dangerous tell investors and policymakers? Park observed a 'natural connection' between the technological description and the economic value perceived by the market.
According to Park, when a company emphasizes the transformative power and unprecedented risks of its models, three messages are conveyed: the extraordinary capability of the technology, its potential for gigantic impact, and the possibility for the company to capture exceptional economic value. Although these points reinforce each other, he insists on the need to distinguish between power, danger, and economic value.
Park supports safety requirements for high-risk AI systems, demanding evaluations, transparency, and controls when models can cause significant harm. However, he warned that the problem arises when regulatory standards mirror the structures and resources of the largest companies in the sector. For him, regulation must be proportional to the risk, not to the company's market size.
He pondered that if global standards are dictated by the largest American AI companies, Brazilian companies may face regulatory and technical costs designed for organizations with much superior resources. Park concluded that the central debate involves two questions: whether AI is too dangerous to be developed, or whether it is too dangerous to allow competition from other agents, which could result in regulation that, ironically, hinders the emergence of new competitors and increases external dependence.
