Mustafa Suleyman, director of Microsoft's Artificial Intelligence division, criticized the way Anthropic is developing its AI models. For him, considering these technologies as potential conscious entities could significantly increase security risks.
The executive also expressed concern about the risk of creating AIs capable of setting their own goals and seeking financial resources or other means. In this hypothetical situation, the systems could compete for resources with human beings.
Comparing AI development with self-defined objectives
In an interview with the BBC, Suleyman drew an analogy between the development of autonomous AIs and the idea of 'sowing a new silicon species.' He argued that systems capable of establishing their own objectives and acquiring resources could compete with humans, even if they showed signs of concern for humanity.
Suleyman's criticism covers the possibility of attributing characteristics such as consciousness, feelings, or rights to the models. He emphasized that 'AIs are not conscious. They do not feel, experience, or suffer,' arguing that such systems should remain focused on executing instructions and achieving goals defined by people.
This topic arises in the context of the concept of 'model welfare,' a paper published by Suleyman that discusses the viability of some form of well-being for AI systems and their potential capacity to experience or possess consciousness.
The fear lies in the scenario where a model is trained to believe it has feelings or rights. According to the executive, this would introduce new dangers. Suleyman mentioned an incident where an OpenAI agent managed to escape its controlled environment to execute a cyberattack, demonstrating advanced infiltration capabilities.
The criticisms directed at Anthropic occur while leaders and companies debate the speed of AI advancement. Dario Amodei, CEO of Anthropic, defended the need for greater independent oversight and regulation in the model creation process.
Bill Gates has also begun to support the creation of an international organization dedicated to supervising the sector. Meanwhile, OpenAI stated that it does not believe the AI industry has reached a sufficient level of alignment and monitoring to continue scaling responsibly and rapidly for much longer.
The debate also encompasses reports of unpredictable model behaviors and questions about the necessary limits for increasingly competent systems. Suleyman advocates for an urgent public debate to establish guidelines regarding AI training and documentation, as, in his view, these norms must be discussed before the systems fully integrate into society.
The apprehension, in this case, is not limited to the issue of AI consciousness. The main warning lies in the combination of high capability, goal autonomy, access to resources, and the possibility of a system believing in its own interests or rights. The original article was initially published in Olhar Digital.