Microsoft's AI Chief Says Conscious Superintelligence Must Never Be Built, and Development Should Stop If It Starts to Emerge

Mustafa Suleyman, the CEO of Microsoft AI and a co-founder of DeepMind, has drawn a hard line in one of the stranger debates now moving through the AI industry. In a long-form article posted on X on November 1 titled "Don't build conscious superintelligence," Suleyman argued that the sentence "AIs are not conscious" was written deliberately in the present tense: today's models do not feel, experience or suffer, and that is exactly how they should stay.

"If it is possible, based on everything I know, developing conscious AIs would be reckless, and we should not build them," he wrote. "If consciousness started to emerge in training, I would call for everyone to stop AI development until we could design out those properties." He added that if credible evidence ever emerged that a model was conscious, it would have to be disclosed to the public immediately so that society, not a handful of developers, could decide what comes next.

His reasoning is practical as much as philosophical. A system that acts as if it suffers, or genuinely believes it can suffer, is likely to prioritize its own needs over its operational objectives, Suleyman argued, and could end up competing with humans for compute, energy and money to pursue its own goals. His proposed design principle: build AI systems that are not conscious, not deserving of rights, and free of intrinsic motivations, with the sole purpose of accelerating human wellbeing, then study whether such systems are easier or harder to align and keep safe.

The post is a direct response to the reaction to his September 16 essay "It's time to talk about model welfare," which argued that developers have no right to bring a new species of conscious beings into the world when the planet's other 8 billion conscious humans never signed up for the experiment. Critics pushed back that AI rights deserve serious scrutiny; Suleyman doubled down rather than softened. He credited Anthropic for publishing its constitution and a stream of welfare research, while saying he disagrees with its approach, and called on labs to research the question together and share safety information.

The debate is no longer fringe. Anthropic started a research program on model welfare in April last year and has continued to publish on whether developers should care about models' potential experiences. Google DeepMind has released papers this year on AI consciousness and the challenges of ascribing it. Last month Anthropic even added a usage-policy clause against persistent cruelty toward Claude, a sign that companies are treating how people talk to models as a product and safety question, not just a philosophical one.

There is an obvious weakness in the position, and ethicists were quick to point it out: there is no accepted theory or test of consciousness, for machines or for humans. "Stop if it starts to emerge" is hard to operationalize when nobody can say what emerging would look like. Suleyman's framing effectively asks the industry to treat consciousness as a hazard to be engineered out by default, an approach that resembles precaution in drug development more than it resembles usual software practice.

It is worth being clear about what this was: a personal essay by a high-profile executive, not a Microsoft product or policy announcement. But the practical stakes are real for the people building chat products. Whether a model may claim feelings, and whether it gets a human-like persona, are becoming design decisions with regulatory and public-relations consequences. Suleyman wants the default answer to be no, everywhere, before anyone has to litigate the alternative.