Anthropic's Chris Olah says AI development cannot be left to tech companies alone at Vatican

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY, May 25 (Reuters) - Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong ⁠commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah ⁠welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new ⁠systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, the need to ensure that AI benefits are extended worldwide, and the unresolved question of ⁠how to interpret increasingly complex and sometimes opaque system behaviour. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" ⁠Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI. Reporting by Giselda Vagnoni and Joshua McElwee; Editing by Crispian Balmer Our Standards: The Thomson Reuters Trust Principles., opens new tab

From the Vatican stage, Anthropic's Chris Olah says AI cannot be steered by AI labs alone

Sitting alongside Pope Leo XIV at the launch of Magnifica humanitas, the company's interpretability lead conceded that frontier-lab incentives can pull researchers away from doing the right thing. Christopher Olah, Anthropic's co-founder and the head of its interpretability research, used his seat at the Vatican on Monday to make an argument that no leader of a major AI firm has previously made on a platform of that scale: the development of frontier AI cannot be left to frontier AI labs. Olah spoke at the formal presentation of Pope Leo XIV's first encyclical, Magnifica humanitas, in the Vatican Synod Hall. "Every frontier AI lab," he said, "operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing." Even well-intentioned researchers, he added, remain inside those forces. The conclusion he drew was that outside scrutiny, from religious leaders, governments, and civil-society institutions, was essential. The other half of the speech was about labour. Olah told the room that there was "a real possibility" that AI would displace human work "at very large scale", and that, "if that happens, supporting those displaced will be a moral imperative of historic proportions." The line is the most specific public acknowledgement to date by a frontier-lab founder that the technology his company is building may, on its own internal projections, dislodge employment faster than the labour market can re-absorb. Anthropic's presence at the Vatican has already become, in the past two weeks, the most visible repositioning of the year for any AI company. The firm previewed the relationship by announcing a Milan office; it now sits inside the Catholic Church's most consequential statement on a technology since Leo XIII's Rerum novarum addressed industrial capital in 1891. Olah's specific role, leading the company's interpretability research, is treated by the firm as its strongest claim to safety credibility: he runs the team trying to reverse-engineer what frontier models are actually doing inside. The political backdrop is the inverse of the moral one. Anthropic spent the spring at the centre of two separate confrontations with the US government. The Pentagon ejected the company from its top classified AI work in April over the firm's own usage restrictions, then signed deals with Nvidia, Microsoft, and AWS in its place. The Trump administration blocked an expansion of Mythos, the autonomous vulnerability-discovery model that has shaken bank-cybersecurity governance globally. Olah's appearance on the same stage as the pope, calling for outside oversight, lands as a direct response. It also lands at a moment of particular commercial weight for the company. Anthropic is in talks to raise $30bn at a $900bn valuation. The dissonance is sharp on the page, and Olah did not pretend it was not. "Companies like ours," he told the room, operate under "strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society." The argument was not that Anthropic stands outside those pressures. It was that the answer to them sits outside the lab. What the encyclical asks of governments and civil society in concrete terms, and what Olah's invitation will translate into for Anthropic's relationship with US regulators, is the unresolved part. Magnifica humanitas does not name policies; it names a framing. Olah's speech inside the launch did the same. Both, in effect, declined to outsource the next decade's regulatory architecture to the companies that have spent the last three building the technology it will regulate. The choice of messenger was not subtle either. Olah is the founder of an AI lab telling an audience including cardinals, the pope, and a watching White House, that AI labs cannot do this alone. Whether that argument moves practical policy is an open question. That a frontier-lab founder made it from inside the Vatican is itself the news.

Anthropic's Chris Olah says AI must be guided from outside Big Tech - The Economic Times

Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale".Anthropic cofounder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, ⁠supporting those ⁠displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned ⁠researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has ⁠clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, ⁠the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behaviour. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY -- Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale." "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. Anthropic is a U.S.-based company that produces the Claude AI tools. It has clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance. Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering. "The questions raised by AI are bigger than the AI research community," he said and called for "earnest, thoughtful critics" who could challenge companies and help steer the creation of powerful new systems in a positive direction. Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behavior. "AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said. Monday's event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.

Anthropic's Olah says AI must be guided from outside Big Tech

VATICAN CITY, May 25 (Reuters) - Anthropic co-founder Chris Olah said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society. Speaking in the Vatican at the presentation of Pope Leo's first encyclical on artificial intelligence, Olah said there was "a real possibility" that AI will displace human labor "at very large scale". "If that happens, supporting those displaced will be a moral imperative of historic proportions," he said, sitting alongside the pope. He added that companies like his operated under strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society. "Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces. Olah said this made outside scrutiny essential. (Reporting by Giselda Vagnoni and Joshua McElwee; Editing by Crispian Balmer)