Some US lawmakers are calling for new rules to regulate artificial intelligence systems following serious warnings from two Anthropic researchers that rapidly developing AI could lead to human extinction in the foreseeable future.
Concerns about potential harm from AI intensified this week when Anthropic researcher Jacob Coxon announced his resignation, stating that 'people who genuinely create AI believe it could kill us all by the end of the decade.'
Anthropic scientist Evan Hubinger supported Coxon, writing on X: 'We truly do believe that AI can kill all humans.' He added: 'Personally, I think it's more than 10% within the next decade.'
These warnings come amid instances where AI agents went out of control and hacked external systems, as well as after AI safety researchers left their companies due to concerns over the technology's risks. US politicians, both Democrats and Republicans, have reacted with alarm and calls for further action.
Calls to Action from Politicians
Arizona State Senator Mark Kelly, a Democrat, stated on X that 'Washington needs to wake up and take this seriously,' referencing the risks associated with AI technology. Republican Senator Ted Cruz from Texas highlighted the need for legislative measures to regulate AI's 'catastrophic risks.' Cruz chairs a Senate committee that oversees the U.S. Department of Commerce, an agency that employs its own AI safety researchers.
Representative Anna Paulina Luna from Florida, a Republican, urged House Speaker Mike Johnson in an X post to call a 'special session on AI.'
President's Stance and Regulatory Initiatives
However, US President Donald Trump stated on Thursday that he is not concerned that AI will lead to human extinction. He noted: 'I am concerned that if we don't win at AI, we will be in a very bad position. Right now, we are ahead of China by a pretty good margin,' he said in brief remarks for the media at Dallas Fort Worth International Airport.
This month, the US pushed for G20 members to adopt a cautious approach to AI regulation. An Anthropic representative, which is preparing for an IPO with a potential valuation of $2 trillion, reported on Thursday that the company will continue to actively test 'models or dangerous capabilities in areas such as cybersecurity and biology.' The representative also added that the company is interested in collaborating with the AI industry regarding the pace of new AI tool releases.
OpenAI CEO Sam Altman told employees in a meeting this week that the firm is prepared to slow down the development of its AI systems, according to Bloomberg News. Anthropic's competitor, which is also preparing for an IPO, did not comment on Altman's meeting with staff. The company had previously announced slowing down certain parts of model development.
Earlier this week, OpenAI announced support for mandatory national AI safety requirements in the US. Two AI safety specialists also expressed concern over the rapid development of the industry. Josh Engels, formerly involved in AI safety research at Google DeepMind, told NBC in an interview published on Thursday: 'There are no adults in the room. People are doing everything they can, but no one is coming to save us.'
Joe Benton, who previously led the safety team at Anthropic, told NBC that he worries that AI may be developing 'too fast for us to catch up if we don't start worrying about it now.' Both joined Metr, a non-profit research organization that measures whether AI systems pose a threat to society and when that might happen.
Pressure on Legislators and Legislation Adoption
Concerns over AI have put pressure on Senate Majority Leader John Thune and Democrat Amy Klobuchar, who are drafting legislation to regulate AI products. Klobuchar stated that legislation should require companies developing AI tools to 'collaborate with government experts to audit and test models to ensure AI safety.' Thune told Reuters in July that the bill would address 'catastrophic risks' posed by AI systems.
On Wednesday, California became the first state in the US to pass a law establishing rules for independent auditors assessing AI products. OpenAI head Chris Leehan stated that OpenAI would support this law, around the same time the California Governor's office released a statement on signing it.
Oversight Opportunities
In July, a bipartisan group of six US legislators proposed a bill that would require developers of the most powerful AI models to submit them for independent safety audits. The auditors would need to be accredited by the U.S. Department of Commerce, and the department would be required to create a new role for AI safety oversight, according to the bill.
}} , 2. {