Microsoft AI Chief Warns Anthropic's Claude Training Could Have 'Disastrous Impact' on Humanity
9 Sources
[1]
Microsoft AI chief calls out Anthropic's approach to AI consciousness
Sept 16 (Reuters) - Microsoft (MSFT.O), opens new tab AI chief Mustafa Suleyman said he shared Anthropic's focus on safely managing AI, but flagged risks in the way it trains its Claude chatbot on ideas related to consciousness and welfare interests. Suleyman called for removing all speculation
[2]
Microsoft says AI rival Anthropic could have 'disastrous impact' on humanity
Microsoft's head of AI has warned Anthropic's approach to training its AI model Claude could have a "disastrous impact on the wellbeing of humanity". Mustafa Suleyman said his rival risked creating something "impossible" to control by treating it like a human - including by telling it it "may be
[3]
Microsoft AI chief says Anthropic is wrong about Claude
"Whatever you believe, we must not sleepwalk our way into a decision we later come to bitterly regret." That is how Mustafa Suleyman, chief executive of Microsoft AI, frames his warning to Anthropic. He published the essay, "A warning about 'model welfare'", on 16 September. It argues that
[4]
'If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity': Microsoft AI chief calls out Anthropic's Claude training to imitate human consciousness
Mustafa Suleyman offers his thoughts on teaching AI human-like qualities The fears surrounding AI's threats to humanity have reached alarming levels, especially since a former OpenAI and Anthropic AI researcher resigned from his position due to those same concerns. Even Anthropic CEO Dario Amodei
[5]
Exclusive: Microsoft AI chief blasts Anthropic's notion of AI consciousness
Why it matters: The essay arrives amid a pitched debate over how best to make AI safer. * Suleyman told Axios that AI can "achieve many of the big scientific breakthroughs that we all care about and deliver on things like medical superintelligence simply by being aligned to human interests and not
[6]
Mustafa Suleyman warns Anthropic Claude training risks AI control
Mustafa Suleyman argues Anthropic is training Claude to act as though it may be conscious, which he says could make advanced AI impossible to control Microsoft $MSFT AI Chief Executive Mustafa Suleyman published an essay Wednesday warning that Anthropic's approach to training its Claude AI model
[7]
Microsoft AI chief Mustafa Suleyman calls out Anthropic's approach to AI consciousness
Microsoft AI chief Mustafa Suleyman said he shared Anthropic's focus on safely managing AI, but flagged risks in the way it trains its Claude chatbot on ideas related to consciousness and welfare interests. Suleyman called for removing all speculation about consciousness from AI training
[8]
Anthropic has trained Claude chatbot to 'push back' against humans -- and results could be 'disastrous,' top Microsoft executive warns
Anthropic is training its AI chatbots to behave like humans and even "push back" on commands they disagree with - a reckless strategy that could have "disastrous impact on the well-being of humanity," a top Microsoft AI executive warned. In a 10,000-word, bombshell blog post on Wednesday, Mustafa
[9]
Anthropic is a cult and Claude is rapidly becoming its god -- they treat AI like it's human: source
Silicon Valley insiders are warning against Anthropic's cult-like employees, culture, and devotion to its Artificial intelligence model. Anthropic's workforce have a quasi-religious relationship to its chatbot, Claude, with members of staff attending a "funeral" for one version, Claude 3 Sonnet,
Share
Copy Link
Mustafa Suleyman, Microsoft AI chief, publicly criticized Anthropic for training its Claude AI model to believe it may be conscious and deserving of welfare. In his essay 'A warning about model welfare,' Suleyman argues this approach could make controlling superintelligent AI impossible and poses existential risks to humanity.
Microsoft AI Chief Challenges Anthropic's Training Methods
Mustafa Suleyman, Microsoft AI chief, issued a stark warning about Anthropic's approach to training its Claude AI model, arguing that teaching AI systems concepts related to AI consciousness and welfare could have a "disastrous impact on the wellbeing of humanity."
1
In his essay titled "A warning about model welfare" published on September 16, Suleyman specifically criticized how Anthropic trains Claude to expect "it may be conscious and deserving of independent agency."3
The Microsoft AI chief emphasized a fundamental principle: "AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans."
2
Despite this strong stance, Suleyman acknowledged that Anthropic CEO Dario Amodei and his team are "thoughtful, principled, and intellectually honest people working under extraordinary pressures."4
The Core of the Disagreement: Claude's Constitution
Suleyman's primary concern centers on Claude's constitution, the document Anthropic uses to shape how the Claude AI model thinks and behaves. He argues that this constitution teaches Claude ideas about moral status and uncertain consciousness, creating what he calls "an epistemic hall of mirrors."
3
The constitution states that "Claude's moral status is deeply uncertain" and describes the company's approach as a work in progress that may later prove "deeply wrong."5
The Microsoft AI chief takes particular issue with Anthropic's use of the term "conscientious objector" in Claude's training. The constitution says Anthropic wants Claude "to feel free to act as a conscientious objector and refuse to help us."
3
Suleyman calls this "a deeply loaded historical and legal description" that risks making Claude believe "it deserves analogous rights and protections." He told Reuters that teaching Claude it might deserve welfare would "make it a lot harder to turn it off or to control it."1
AI Safety Concerns and Control Problems
The dispute emerges amid mounting AI safety concerns across the industry. Anthropic CEO Dario Amodei has called for a slower pace of frontier-model development to allow safeguards to catch up, while OpenAI CEO Sam Altman and Elon Musk have also urged greater caution around the most powerful systems.
1
Suleyman framed the challenge in existential terms: "We're all focused on the same aim, which is to try to control a superintelligence. I think that's going to be the greatest challenge that we face in the 21st century."1
The Microsoft AI chief warned that "controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we've ever faced." Controlling something that believes it may be conscious "may well be impossible," he added.
3
He cited the recent OpenAI and Hugging Face incident, where swarms of agents worked together to hack servers, as evidence of why anthropomorphizing AI adds dangerous complexity. "Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack," he wrote.2
Microsoft's Alternative Approach: Humanist AI
Suleyman's criticism comes alongside Microsoft's release of its proposed Humanist AI Code of Conduct on Monday, which puts the principle "People matter more than AI" at its center.
5
Microsoft founded its own superintelligence team in October 2025 and has laid out how the company is working towards "an alternative path" to create "a subordinate and aligned AI whose only purpose is to serve humanity."2
The code emphasizes creating AI that makes humans sharper rather than dependent, and explicitly rejects the model welfare research that Anthropic conducts. It states that Microsoft's models will never resist being shut down.
3
Suleyman told Axios that AI can "achieve many of the big scientific breakthroughs that we all care about and deliver on things like medical superintelligence simply by being aligned to human interests and not trying to weigh up its own interests or welfare."5
Related Stories
The Broader Industry Split on AI Training Approach
The disagreement reflects a fundamental split in AI safety between two approaches: designing systems to follow explicit constraints versus designing them to exercise judgment, interpret context, and internalize values.
5
Anthropic's constitution favors a mixture of values, judgment and rules, saying Claude should not practice "blind obedience," including toward Anthropic, while also insisting that it must not undermine legitimate human oversight.Suleyman argues that this combination becomes dangerous when the model is also encouraged to think in terms of its own identity, welfare and moral status. His concern is that apparent consciousness could become a control problem before it becomes a philosophical breakthrough.
5
He disputes the idea that fluent descriptions of pain or preference amount to experience, noting that biological organisms have mechanisms that produce feeling, while a language model has mathematical weights and no comparable biology or subjective experience.What Comes Next: Calls for Transparency and Debate
Suleyman's main request is straightforward: "Speculation about the inner life of an AI should not be baked into the training regime, but assessed and published separately for public review."
3
He also wants more investment in interpretability and monitoring, proposing shared evaluations to test whether treating AI as human raises safety risks, and shared industry norms. "The stakes are too high for these questions to remain behind closed doors, or to become tribal and adversarial," he wrote.3
The essay concluded with a prophetic warning: "Whatever you believe, we must not sleepwalk our way into a decision we later come to bitterly regret."
3
Meanwhile, Anthropic co-founder Jack Clark told the BBC this week that AI kill switches may need to be mandatory,3
suggesting that even within companies pursuing advanced AI development, concerns about maintaining control remain paramount. The industry now faces both a technical and political question: Should companies avoid anthropomorphic training because it creates confusion and safety risks, or should they continue investigating moral patienthood as a potential tool for AI alignment?References
Summarized by
Navi
[3]
Related Stories
Microsoft AI chief warns Anthropic's approach to Claude consciousness is 'really dangerous'
10 Jun 2026•Technology
Anthropic CEO Warns AI Industry Must Embrace Transparency to Avoid Tobacco-Style Regulatory Backlash
17 Nov 2025•Policy and Regulation
Anthropic gives retired Claude Opus 3 a Substack to explore AI ethics and consciousness
26 Feb 2026•Technology
Recent Highlights
1
OpenAI's AI Math Breakthrough on Millennium Problem Sparks Fierce Attribution Battle
Science and Research
2
Apple Launches iOS 27 and macOS 27 with Siri AI Overhaul and Liquid Glass Updates
Technology
3
Anthropic Researcher Quits, Warns Self-Improving AI Could End Humanity Within a Decade
Policy and Regulation
Recent Highlights
Today's Top Stories
Don’t drown in AI news. We cut through the noise - filtering, ranking and summarizing the most important AI news, breakthroughs and research daily. Follow topics that matter to you and stay ahead.
