The creators of some of the world's most powerful AI systems have put a dispute about the limits of their own technology to the level of the UN Security Council and called on States to agree on rules that will not allow advanced models to get out of human control.
Sam Altman, at a meeting on September 23, proposed creating compatible national and international standards for advanced AI. According to OpenAI, countries should equally measure the possibilities of models, assess risks and check whether protective mechanisms are sufficient to preserve human oversight.
A separate block of proposals relates to incidents. OpenAI wants uniform rules for the classification and rapid disclosure of dangerous failures, as well as secure channels between authorities, critical infrastructure operators and technicians. The company believes that such an exchange will allow to notice repeated problems before a separate failure escalates into a large-scale crisis.
The main cause of anxiety Altman called the growth of autonomy and the approach of recursive self-improvement, in which AI is increasingly helping to create the next generations of AI. The head of OpenAI considers unacceptable the situation when the pace of development becomes higher than the ability of people to understand the actions of the system and to intervene. In recent weeks, the developers have already revised the attitude towards such risks after a series of unusual episodes with autonomous agents.
The head of Anthropic Dario Amodei took a similar position, but formulated more specific restrictions. Amodei has proposed an international ban on the use of AI for the creation of biological weapons, to develop ways to test the real capabilities of models and to establish common standards for testing for abuse and loss of control.
For Anthropic, such a call was not a sharp change of course. The company has already proposed an international mechanism that would slow or temporarily stop the development of the most powerful systems if the control methods do not keep up with the growth of opportunities. The key difficulty is that the one-way pause does little to change as long as competitors continue the race.
At the same time, OpenAI and Anthropic do not offer to stop the development of AI in its entirety. Both companies simultaneously emphasize the economic and scientific advantages of the technology. Altman also warned of the other extreme when control of the most powerful models is concentrated in one company or one state, so future rules should not cement the advantage of the current market leaders.
At the level of States, there is no single position yet. The White House representative for science, Michael Kratzios, opposed the pause and new supranational governance structures. The U.S. proposes to develop national competencies, testing and sharing best practices, keeping regulation in the hands of individual countries. The UK, by contrast, intends to bring single global AI safety standards among the themes of its G20 presidency in 2027.
The discussion was also attended by the head of Hugging Face Clement Delang and one of the pioneers of modern machine learning Yoshua Bengio. The conversation in the Security Council showed a new stage of the dispute. The largest developers are no longer limited to internal security rules and publicly ask governments to define boundaries that companies themselves do not want to establish alone.
There was no practical international mechanism after the meeting. OpenAI and Anthropic's proposals remain the positions of the companies, not the UN's decisions. The main outstanding issue now concerns not only the level of risk from future models, but also the one who will get the right to measure such risk, check compliance with restrictions and, if necessary, slow down the development.
Sam Altman, at a meeting on September 23, proposed creating compatible national and international standards for advanced AI. According to OpenAI, countries should equally measure the possibilities of models, assess risks and check whether protective mechanisms are sufficient to preserve human oversight.
A separate block of proposals relates to incidents. OpenAI wants uniform rules for the classification and rapid disclosure of dangerous failures, as well as secure channels between authorities, critical infrastructure operators and technicians. The company believes that such an exchange will allow to notice repeated problems before a separate failure escalates into a large-scale crisis.
The main cause of anxiety Altman called the growth of autonomy and the approach of recursive self-improvement, in which AI is increasingly helping to create the next generations of AI. The head of OpenAI considers unacceptable the situation when the pace of development becomes higher than the ability of people to understand the actions of the system and to intervene. In recent weeks, the developers have already revised the attitude towards such risks after a series of unusual episodes with autonomous agents.
The head of Anthropic Dario Amodei took a similar position, but formulated more specific restrictions. Amodei has proposed an international ban on the use of AI for the creation of biological weapons, to develop ways to test the real capabilities of models and to establish common standards for testing for abuse and loss of control.
For Anthropic, such a call was not a sharp change of course. The company has already proposed an international mechanism that would slow or temporarily stop the development of the most powerful systems if the control methods do not keep up with the growth of opportunities. The key difficulty is that the one-way pause does little to change as long as competitors continue the race.
At the same time, OpenAI and Anthropic do not offer to stop the development of AI in its entirety. Both companies simultaneously emphasize the economic and scientific advantages of the technology. Altman also warned of the other extreme when control of the most powerful models is concentrated in one company or one state, so future rules should not cement the advantage of the current market leaders.
At the level of States, there is no single position yet. The White House representative for science, Michael Kratzios, opposed the pause and new supranational governance structures. The U.S. proposes to develop national competencies, testing and sharing best practices, keeping regulation in the hands of individual countries. The UK, by contrast, intends to bring single global AI safety standards among the themes of its G20 presidency in 2027.
The discussion was also attended by the head of Hugging Face Clement Delang and one of the pioneers of modern machine learning Yoshua Bengio. The conversation in the Security Council showed a new stage of the dispute. The largest developers are no longer limited to internal security rules and publicly ask governments to define boundaries that companies themselves do not want to establish alone.
There was no practical international mechanism after the meeting. OpenAI and Anthropic's proposals remain the positions of the companies, not the UN's decisions. The main outstanding issue now concerns not only the level of risk from future models, but also the one who will get the right to measure such risk, check compliance with restrictions and, if necessary, slow down the development.
