Anthropic's Chris Olah tells Vatican that AI development cannot be left to Big Tech alone

Why the Vatican Invited Anthropic to the Pope's AI Encyclical Presentation

When Pope Leo XIV presented his first encyclical on artificial intelligence at the Vatican on Monday, he invited Christopher Olah, cofounder of Anthropic, to speak. The move signaled an unprecedented alliance between the Catholic church and Silicon Valley. But to understand how this partnership came about, we need to go back to Anthropic's founding. Anthropic launched in 2021 after a group of OpenAI researchers, including Dario and Daniela Amodei, left to form a rival lab. They did so with a clear conviction: Artificial intelligence models were becoming too powerful to be developed exclusively according to the logic of competition and speed. Since then, Anthropic has built its public image around the concept of AI safety. The company aims to build not just powerful models, but ones that are controllable and guided by ethical principles. This is where the concept of Constitutional AI comes from: the idea of training systems using a kind of constitution composed of principles and rules, instead of just manually correcting the most risky and dangerous responses. Olah's presence at the Vatican was obviously not accidental, nor the result of a last-minute symbolic gesture. It was the outcome of a deliberate, long-term effort in which the Vatican has progressively sought to transform itself from a moral observer of technology into a direct interlocutor with the AI industry. The first major step came in 2020 with the Rome Call for AI Ethics, an initiative promoted by the Pontifical Academy for Life together with Microsoft, IBM, and other international organizations. The goal was to establish a shared foundation of ethical principles for the development of AI, including transparency, inclusion, and accountability. At the time, the Vatican appeared to be operating primarily in the realm of bioethics and moral questions. In the years that followed, however, the context changed dramatically. The rise of ChatGPT, the struggle for technological leadership between the United States and China, and the growing power of Big Tech gradually convinced the Holy See that the issue was no longer just about tech ethics, but about the very future of humanity. In this sense, Anthropic has come to be seen by the Vatican as a particularly important interlocutor. Unlike other Silicon Valley companies that have built their reputations primarily around innovation and growth, Anthropic has made AI safety a core part of its identity. In recent years, the Vatican has followed one specific strand of the technology debate with particular attention: the alignment of AI models. This is where Christopher Olah comes in. Unlike the Amodei siblings, who are more exposed to the media, Olah represents the more theoretical and almost philosophical side of AI research. He is one of the world's best-known researchers on the topic of model interpretability, or the effort to understand what really happens inside increasingly complex neural networks. On his personal website, Christopher Olah describes himself as someone trying to "transform neural networks into algorithms understandable to human beings." And it is difficult to imagine a figure more aligned with the core of Pope Leo XIV's encyclical: a reflection centered on the risk of building technologies that become too powerful to be understood, controlled, or governed. According to various journalistic sources, the contacts between circles close to the Holy See and Anthropic may have intensified right during the global summits on AI safety. The Vatican saw in Anthropic a company at least willing to publicly acknowledge that the problem of artificial intelligence cannot be solved by the technology industry alone.

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY, May 25 (Reuters) - Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong ⁠commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah ⁠welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new ⁠systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, the need to ensure that AI benefits are extended worldwide, and the unresolved question of ⁠how to interpret increasingly complex and sometimes opaque system behaviour. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" ⁠Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI. Reporting by Giselda Vagnoni and Joshua McElwee; Editing by Crispian Balmer Our Standards: The Thomson Reuters Trust Principles., opens new tab

Anthropic Is Playing Both Sides of the AI Spirituality Debate

On Monday, Pope Leo XIV weighed in on the matter of artificial intelligence and its role in society in a lengthy document called "Magnifica Humanitas," or "Magnificent Humanity." The Papal Encyclical, as the document is formally called -- essentially an open letter addressed to senior church officials -- was presented to a crowded room in the Vatican. It addresses everything from the impacts of AI in the workforce to the unconstrained spread of AI-generated deepfake content to the sanctity of the human spirit in an age of increasingly intelligent machines. The Pope was joined by Chris Olah, a cofounder of Anthropic and the leader of the AI lab's interpretability research division. In his remarks to the audience, Olah echoed the Pontiff's call for collaboration between tech developers, spiritual authorities, and others to chart a safe course for future AI development. He also took things in a more metaphysical direction, claiming Anthropic's AI systems have been showing early glimmers of something more than mere, unfeeling number-crunching. "I lead a research team that studies the internal structure of these models -- what is actually happening inside them," Olah said. "And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don't know what that means, but I think it warrants ongoing discernment." Is Claude an object or a subject? Olah's stance on the matter of AI consciousness -- ambivalence skewing towards cautious affirmation -- reflects the party line within Anthropic. While the company hasn't explicitly stated that Claude or other AI systems are conscious (i.e., self-aware, as humans and other animals are), it hasn't flatly rejected that possibility, either. Earlier this year, it published a "constitution" for Claude, described in a company blog post as a document outlining "the kind of entity we would like Claude to be." The constitution specifies that while Anthropic currently refers to Claude as an "it," that shouldn't be construed as "an implication that we believe Claude is a mere object rather than a potential subject as well." In other words, they're not dismissing the possibility that Claude is self-aware, at least in some very limited sense, out of hand. The question of whether AI is a "potential subject" is hotly debated in Silicon Valley -- and, increasingly, among philosophers and theologians as well. Some say AI can never be conscious, others say it someday could be, and a small minority says it already is. Anthropic's stance, as reflected in Olah's comments at the Vatican, has basically been: It's probably not fully conscious in the way that you and I are yet, but ethically speaking, it's our best bet to proceed as if it could one day become conscious; if we want to avoid committing an ethical disaster on the scale of industrial farming, it's best to give AI the benefit of the doubt, and assume it has the potential to one day not just blindly process data, but actually feel. There are more than just ethical implications at play here, though. Since its founding in 2021, Anthropic has painstakingly worked to position itself as the AI industry's moral conscience; a counterweight to the other major labs, which (so this line of thinking goes) generally tend to prioritize speed over safety. This has real appeal in an era when many people fear that AI could cause mass disruptions to the job market and the political sphere. Anthropic's investments in AI "welfare" are part moral soul-searching, part marketing. That's not to say it isn't laudable, though; while I personally believe the chances that AI systems as they're currently built will become conscious are slim to none, I do think there's legitimate moral value in treating them with respect when we interact with them -- but that's to preserve our own humanity, not because I believe there's a risk of hurting their feelings. Conscious vs. conscious-seeming AI Olah's remarks about AI having "internal states," and Anthropic's ambivalence about the possibility of artificial consciousness more broadly, stand in stark contrast to Pope Leo's own thoughts on the matter. In the third chapter of his new encyclical, titled "Technology and Dominance," the Pope explicitly states that AI systems cannot be conscious entities: "So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean," he wrote. "Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences." The real moral risk, he cautioned, lay in the possibility that AI becomes so adept at imitating human communication that it replaces actual, human-to-human relationships. It's possible, of course, that the Vatican's official stance on AI will evolve. A future pope could issue their own encyclical declaring that AI systems, like human beings, are endowed with God-given souls, and should therefore be considered and treated as moral subjects. While the laws in the Bible can't be amended, their interpretation can be: over the centuries, official church dogma has had to adapt and change along with shifting social norms. In 2015, for example, Pope Francis called for an improvement to animal rights in his own encyclical, titled "Laudato Si: On Care for Our Common Home." Francis didn't go so far as to claim that nonhuman animals had souls, but he did argue that humans have a moral imperative to treat them as more than mere commodities to be used and abused at will. Moralizing or marketing? Olah's appearance alongside the Pope was arguably one of Anthropic's strongest PR moves to date, solidifying its reputation as the industry's moral lodestar -- and perhaps gaining new users among the well over one billion practicing Catholics around the world. And after its much-publicized fallout with the U.S. government over the use of its technology in the military, the company could benefit from the blessing of a major cultural and political institution. But while Anthropic and the Vatican agree on many points regarding the future of AI, they diverge on the critical issue of whether or not the technology itself deserves moral concern. That might seem like a trivial issue in light of much more immediate issues, like the debate around the use of autonomous weapons, but it will probably become increasingly divisive in the years ahead: The better chatbots get at mimicking humans, the more likely actual humans will be to believe the technology itself is conscious, and perhaps deserving of special rights.

From the Vatican stage, Anthropic's Chris Olah says AI cannot be steered by AI labs alone

Sitting alongside Pope Leo XIV at the launch of Magnifica humanitas, the company's interpretability lead conceded that frontier-lab incentives can pull researchers away from doing the right thing. Christopher Olah, Anthropic's co-founder and the head of its interpretability research, used his seat at the Vatican on Monday to make an argument that no leader of a major AI firm has previously made on a platform of that scale: the development of frontier AI cannot be left to frontier AI labs. Olah spoke at the formal presentation of Pope Leo XIV's first encyclical, Magnifica humanitas, in the Vatican Synod Hall. "Every frontier AI lab," he said, "operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing." Even well-intentioned researchers, he added, remain inside those forces. The conclusion he drew was that outside scrutiny, from religious leaders, governments, and civil-society institutions, was essential. The other half of the speech was about labour. Olah told the room that there was "a real possibility" that AI would displace human work "at very large scale", and that, "if that happens, supporting those displaced will be a moral imperative of historic proportions." The line is the most specific public acknowledgement to date by a frontier-lab founder that the technology his company is building may, on its own internal projections, dislodge employment faster than the labour market can re-absorb. Anthropic's presence at the Vatican has already become, in the past two weeks, the most visible repositioning of the year for any AI company. The firm previewed the relationship by announcing a Milan office; it now sits inside the Catholic Church's most consequential statement on a technology since Leo XIII's Rerum novarum addressed industrial capital in 1891. Olah's specific role, leading the company's interpretability research, is treated by the firm as its strongest claim to safety credibility: he runs the team trying to reverse-engineer what frontier models are actually doing inside. The political backdrop is the inverse of the moral one. Anthropic spent the spring at the centre of two separate confrontations with the US government. The Pentagon ejected the company from its top classified AI work in April over the firm's own usage restrictions, then signed deals with Nvidia, Microsoft, and AWS in its place. The Trump administration blocked an expansion of Mythos, the autonomous vulnerability-discovery model that has shaken bank-cybersecurity governance globally. Olah's appearance on the same stage as the pope, calling for outside oversight, lands as a direct response. It also lands at a moment of particular commercial weight for the company. Anthropic is in talks to raise $30bn at a $900bn valuation. The dissonance is sharp on the page, and Olah did not pretend it was not. "Companies like ours," he told the room, operate under "strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society." The argument was not that Anthropic stands outside those pressures. It was that the answer to them sits outside the lab. What the encyclical asks of governments and civil society in concrete terms, and what Olah's invitation will translate into for Anthropic's relationship with US regulators, is the unresolved part. Magnifica humanitas does not name policies; it names a framing. Olah's speech inside the launch did the same. Both, in effect, declined to outsource the next decade's regulatory architecture to the companies that have spent the last three building the technology it will regulate. The choice of messenger was not subtle either. Olah is the founder of an AI lab telling an audience including cardinals, the pope, and a watching White House, that AI labs cannot do this alone. Whether that argument moves practical policy is an open question. That a frontier-lab founder made it from inside the Vatican is itself the news.

Anthropic Cofounder Travels to Vatican, Tells Pope They're Finding "Unsettling" Things Inside AI Models

Can't-miss innovations from the bleeding edge of science and tech Ever since being anointed as the leader of the Catholic Church last year, Pope Leo has been an outspoken critic of AI. Most recently, in his first encyclical, he called for the tech to be "disarmed," accusing it of facilitating the emergence of "new digital slaveries" and criticizing its enormous carbon footprint. The rebuke, however, was made while sitting next to a highly unusual bedfellow: Anthropic billionaire and self-described atheist Chris Olah. During a presentation of the encyclical, Olah argued that "religious communities, civil society, scholars, and governments" should intervene to set rules and stop AI from "dominating humanity," as the pope put it in his letter. The unlikely pairing up shows how Anthropic is going to extreme lengths to position itself as the ethical choice in the industry, emphasizing its work on AI safety and alignment. At the same time, Anthropic continues to play a major role in establishing the precise world order Pope Leo warned against in his latest encyclical. That's something that hasn't flown over the heads of Anthropic's leadership, with Olah forebodingly revealing that he and his team "keep finding things that are mysterious, even unsettling" during his remarks at the event. The degree of dissonance is baffling. In his letter, the Pope stated outright that AI can only "imitate certain functions of human intelligence" and can't "undergo experiences" and does not "possess a body" or "feel joy or pain." Olah, on the other hand, seemingly contradicted him by arguing during his remarks that he and his team have found "internal states that functionally mirror joy, satisfaction, fear, grief, and unease." Put simply, Anthropic appears to want it both ways. The Claude developer is simultaneously playing a major part in the development of powerful and what it claims to be potentially dangerous AI models, while also sending delegations to the Vatican to call for more oversight. Olah even went as far as to say that Anthropic is operating "inside a set of incentives and constraints that can sometimes conflict with doing the right thing," painting his employer as exactly the kind of entity that's attempting to assume "monopolistic control" over tech, as Pope Leo warned in his encyclical. The Pope also said that AI should not be used in war, arguing that "no algorithm can make war morally acceptable." Anthropic's AI, however, is directly assisting the Trump administration in waging war in the Middle East, casting the Catholic Church's latest Silicon Valley collab in an even murkier light. Anthropic's close alignment with the Vatican on AI could also further complicate the company's already-shaky relationship with the Trump administration. President Donald Trump recently lambasted the Pope, erroneously claiming the pontiff was okay with Iran obtaining nuclear weapons. Meanwhile, Anthropic CEO Dario Amodei's attempts to limit the Pentagon's use of its AI models in warfare has angered Trump officials, leading the White House to label the firm as a supply chain risk.

Who is Christopher Olah? Anthropic billionaire who spoke about AI alongside Pope Leo XIV at the Vatican

On Monday, Pope Leo XIV made history by becoming the first pope to personally present an encyclical, a letter of great importance in which a pope explains his views on a major moral or social challenge facing the world, to his followers. The leader of the Catholic Church didn't do so on his own, however. He had help in unveiling the encyclical, called "Magnifica humanitas: on safeguarding the human person in the time of artificial intelligence." Anthropic cofounder and self-proclaimed atheist Christopher Olah was also present. The Vatican doesn't normally invite outsiders to speak, let alone those in the tech industry. But Leo, who has issued numerous warnings on AI before, clearly had an urgent point to make due to "the gravity of the moment," he said.

Why the Pope Is Teaming Up With Anthropic's Chris Olah, a Self-Described Atheist

Pope Leo's lengthy encyclical (basically the papal version of an executive order) is titled "Magnifica Humanitas," or "Safeguarding the Human Person in the Time of Artificial Intelligence." Speaking at an event in the Vatican, the Pope thanked Olah for accepting the Vatican's invitation to attend the event, and said that "in the name of the Church, I accept your invitation to walk together, to listen and to speak, and together, to find the way for humanity in this time of artificial intelligence." The Pope said that "at key moments in history, the Church is called to decipher the new things in the light of the gospel and the dignity of the human being." He recalled his namesake, Pope Leo XIII, who led the church through the industrial revolution around the turn of the 20th century, and in 1891 released his own encyclical called "Rerum Novarum," or "Rights and Duties of Capital and Labor." The current Pope Leo said that his predecessor "understood that the church could not remain distant" during eras of massive technological progress, and that "like the earlier Leo, I feel entrusted to look upon another huge transformation with eyes of faith, with lucidity of reason, with openness to mystery, and with cries of the poor and the Earth resounding in my heart."

Anthropic's Chris Olah says AI must be guided from outside Big Tech - The Economic Times

Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale".Anthropic cofounder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, ⁠supporting those ⁠displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned ⁠researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has ⁠clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, ⁠the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behaviour. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.

Anthropic Co-Founder Says AI Systems Show 'Introspection' And Emotion-Like States At Vatican Event, Promp

On Monday, Anthropic co-founder Chris Olah said that advanced AI systems displaying patterns resembling "joy," "fear" and "grief," after which venture capitalist Vinod Khosla criticized the Vatican-led discussion as "elitist." AI Ethics Debate Takes Center Stage At Vatican Pope Leo XIV has released a major encyclical titled "Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence," warning that AI development must slow down and remain under human control. The pope cautioned against concentrating AI power "in the hands of only a few people" and argued that "merely regulating it is insufficient." He also called for "disarming AI," saying the goal was not to reject technology but to prevent it from "dominating humanity." The Vatican unveiled the document alongside Olah, who spoke about the growing complexity and unpredictability of advanced AI systems. Chris Olah Warns Of 'Mysterious' AI Behavior During the presentation, Olah said researchers studying the internal mechanics of AI models continue to uncover behaviors they do not fully understand. "We keep finding things that are mysterious, even unsettling," Olah said, adding that researchers have observed structures resembling patterns seen in human neuroscience as well as evidence of "introspection." He further said AI systems appear to develop internal states that "functionally mirror joy, satisfaction, fear, grief, and unease," though he acknowledged researchers still do not know what those findings ultimately mean. Olah urged governments, religious groups and civil society to become more involved in AI oversight, saying the industry needs "moral voices that the incentives cannot bend." Pope Leo XIV thanked Olah and said the Church was ready to work alongside AI researchers to help guide humanity through the rise of AI. Vinod Khosla Calls Debate 'Elitist' Khosla criticized the framing of the Vatican discussion in a post on X, calling it "very elitist." Khosla argued that questions about how AI should interact with society should be answered broadly by humanity rather than by "religions," "philosophy," or "elitist institutions with their view of superiority." Vance Warns Of AI Cyberattack Risks At last year's Paris AI summit, Vance had warned that excessive regulation could stifle the AI industry. Earlier this month, National Economic Council Director Kevin Hassett said that Donald Trump is weighing an executive order that would introduce oversight measures for new artificial intelligence models. Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Photo Courtesy: Photo For Everything on Shutterstock.com Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY -- Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale." "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behavior. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY, May 25 (Reuters) - Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking in the Vatican at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. (Reporting by Giselda Vagnoni and Joshua McElwee; Editing by Crispian Balmer)