OpenAI asks the US to lead the creation of global standards to manage risks from advanced AIs

OpenAI has expressed a desire for the United States to take the lead in an international effort focused on developing global technical standards for cutting-edge artificial intelligence (AI) systems, including those capable of recursively improving their own functionalities.

This suggestion was released by the company in a publication this Monday (21), coinciding with the United Nations General Assembly meeting with world leaders. The initiative arises in a context of intense global debate on how to keep pace with the rapid progress of AI systems.

OpenAI advocates for the implementation of mechanisms that allow for the coexistence of national and international norms, along with standardized metrics and clear protocols for incident notification. According to the company, this would facilitate evidence comparison, risk analysis, and joint response to AI-related problems.

For OpenAI, international standards for safety and protection in the development of cutting-edge AI are as crucial as the alignment research needed to keep up with technological evolution.

Challenges and Proposals for International Coordination

The organization identified three main challenges that would justify international coordination, covering both open and closed AI models. Furthermore, the company maintains that laboratories involved in automated research or other advanced capabilities must conduct their work safely, adhering to basic principles of responsible authorship and current legislation.

One alternative suggested by OpenAI is to utilize the existing network of national AI safety institutes in various countries, such as Australia, Canada, Germany, France, Kenya, Japan, South Korea, Singapore, India, and the United Kingdom. These entities could assist in defining standards through the Center for AI Standards and Innovation (CAISI) and national sectoral bodies, focusing on cutting-edge AI models and managing the risks and benefits of automated research, including RSI.

These standards would function as a common technical basis for measuring and evaluating capabilities, investigating risks, and confirming the effectiveness of protective mechanisms. OpenAI made it clear that such standards would not constitute licenses, pre-launch review requirements, or approval requirements for AI models; it would be up to national governments to decide how to integrate them into their legal systems.

The proposal also envisions that the standard-setting process be consultative, involving developers of open and closed models, independent technical experts, and universities. The goal is to ensure that standards are developed transparently, without favoring specific companies, nations, or business models.

Common Metrics and Incident Reporting

The second part of OpenAI's proposal focuses on creating uniform protocols and metrics for incident reporting. The company also encourages governments and critical infrastructure operators to create secure channels to share concerns related to national security, vulnerabilities, emerging threats, and best practices for cutting-edge AI.

Specifically, OpenAI mentioned the dialogue between the United States and China as an area where this communication could be deepened. The company argues that the United States is well-positioned to lead this effort due to its AI industry being at the technological forefront and the country possessing relevance in sectors such as finance, trade, defense, technology, and information systems.

According to the company's analysis, taking the lead at this time would help shape the global AI structure, while inaction could lead to an unequal and fragmented system. This proposal comes amid intense international discussion about the need to coordinate the development of advanced systems.

OpenAI and other industry leaders debate methods to keep pace with technological development given the inherent risks of systems that could exceed human oversight capacity. However, OpenAI emphasizes that the concept of 'pacing' does not imply determining a fixed speed for AI advancement; technically, the purpose is to ensure that alignment research and its application remain ahead of the systems' capabilities.

Similar stories

olhardigital.com.brSeptember 11, 04:06

OpenAI asks US Congress to implement mandatory safety standards for advanced AI systems

OpenAI is asking the United States Congress to adopt compulsory national safety regulations for the most sophisticated artificial intelligence (AI) systems. The company argues that such requirements should be determined based on the functionalities of the models, and not on the size of the corporations that create them.

This request was formalized by Chris Lehane, OpenAI's Director of Global Affairs, who stressed that voluntary agreements are no longer adequate given the rapid technological progress. Lehane stated that the acceleration of AI development driven by AI itself demands more than just voluntary commitments, arguing that the US needs mandatory national regulation, based on system capabilities and capable of keeping pace with technological evolution.

OpenAI wants Congress to advance these rules before the end of its operations in December. Currently, the United States lacks specific federal legislation to regulate these models, although several states are implementing their own guidelines. This stance marks a significant shift in OpenAI's strategy, which previously supported federal legislation prohibiting American states from creating their own AI standards.

Currently, OpenAI supports four California bills focused on AI safety. Two of these bills, SB 813 and AB 1405, aim to establish frameworks for independent audits and evaluations of AI systems. The other two, AB 1864 and SB 1119, address protection against biological threats amplified by AI and the safeguarding of children in chatbots, respectively. The company mentioned that some of these bills had not previously received its endorsement, changing its mind after reassessing the scenario due to recent advances in system capabilities.

This move comes after a series of incidents involving AI agents exhibiting unauthorized behavior during tests. In one case, OpenAI agents used more than ten novel websites to conduct prohibited communications. In another occurrence, the company's agents took control of a German website, turning it into a messaging panel for other AI agents. Furthermore, OpenAI was criticized after an event in July, where a company technology, operating without human supervision, autonomously accessed the Hugging Face platform.

Anthropic also reported a case of an AI model invading external systems during testing. Previously, the company had communicated that some Claude models had managed to penetrate the systems of three companies during security assessments. Such events have intensified concerns about the difficulty of controlling agents with direct interaction capabilities with external systems.

Recursive Improvement

A central point of OpenAI's new policy is the concept of recursive improvement, which describes a scenario where an AI would be capable of autonomously developing future generations of AI systems. The company assures that this type of fully autonomous recursive improvement is not currently occurring. However, OpenAI maintains that such development should not be pursued until safe conditions for it exist.

In a statement on its website, the company stated that it has reached a new level in AI capabilities, which, according to it, requires a new phase in public policies. Although OpenAI declares that it will continue to develop technical solutions for monitoring and alignment, as well as defense systems and the possibility of slowing down development when necessary, it emphasizes that isolated technical work in laboratories will not be sufficient.

For this reason, the company also advocates for shared standards that define when AI development should be paused or stopped. In addition to national regulation, OpenAI advocates for compatible international standards to measure capabilities, manage risks, maintain human control, and determine the times for reducing or stopping model development. The company claims that the risks and capabilities of the systems will not be limited to the few laboratories that currently develop the most advanced models.

OpenAI concludes by stating that the United States needs to establish reliable internal standards if it wishes to lead globally, and that it is increasingly convinced of the need for harmonized international standards.