AI godfather Yoshua Bengio shifts from despair to optimism with Scientist AI safety breakthrough

AI 'godfather' Yoshua Bengio believes he's found a technical fix for AI's biggest risks | Fortune

For the past several years, Yoshua Bengio, a professor at the Université de Montréal whose work helped lay the foundations of modern deep learning, has been one of the AI industry's most alarmed voices, warning that superintelligent systems could pose an existential threat to humanity -- particularly because of their potential for self-preservation and deception. In a new interview with Fortune, however, the deep-learning pioneer says his latest research points to a technical solution for AI's biggest safety risks. As a result, his optimism has risen "by a big margin" over the past year, he said. Bengio's nonprofit, LawZero, which launched in June, was created to develop new technical approaches to AI safety based on research led by Bengio. Today, the organization -- backed by the Gates Foundation and existential-risk funders such as Coefficient Giving (formerly Open Philanthropy) and the Future of Life Institute -- announced that it has appointed a high-profile board and global advisory council to guide Bengio's research, and advance what he calls a "moral mission" to develop AI as a global public good. The board includes NIKE Foundation founder Maria Eitel as chair, along with Mariano-Florentino Cuellar, president of the Carnegie Endowment for International Peace, and historian Yuval Noah Harari. Bengio himself will also serve. Bengio's shift to a more optimistic outlook is striking. Bengio shared the Turing Award, computer science's equivalent of the Nobel Prize, with fellow AI 'godfathers' Geoff Hinton and Yann LeCun in 2019. But like Hinton, he grew increasingly concerned about the risks of ever more powerful AI systems in the wake of ChatGPT's launch in November 2022. LeCun, by contrast, has said he does not think today's AI systems pose catastrophic risks to humanity. Three years ago, Bengio felt "desperate" about where AI was headed, he said. "I had no notion of how we could fix the problem," Bengio recalled. "That's roughly when I started to understand the possibility of catastrophic risks coming from very powerful AIs," including the loss of control over superintelligent systems. What changed was not a single breakthrough, but a line of thinking that led him to believe there is a path forward. "Because of the work I've been doing at LawZero, especially since we created it, I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas," he says. At the heart of that confidence is an idea Bengio calls "Scientist AI." Rather than racing to build ever-more-autonomous agents -- systems designed to book flights, write code, negotiate with other software, or replace human workers -- Bengio wants to do the opposite. His team is researching how to build AI that exists primarily to understand the world, not to act in it. A Scientist AI would be trained to give truthful answers based on transparent, probabilistic reasoning -- essentially using the scientific method or other reasoning grounded in formal logic to arrive at predictions. The AI system would not have goals of its own. And it would not optimize for user satisfaction or outcomes. It would not try to persuade, flatter, or please. And because it would have no goals, Bengio argues, it would be far less prone to manipulation, hidden agendas, or strategic deception. Today's frontier models are trained to pursue objectives -- to be helpful, effective, or engaging. But systems that optimize for outcomes can develop hidden objectives, learn to mislead users, or resist shutdown, said Bengio. In recent experiments, models have already shown early forms of self-preserving behavior. For instance, AI lab Anthropic famously found that its Claude AI model would, in some scenarios used to test its capabilities, attempt to blackmail the human engineers overseeing it to prevent itself from being shutdown. In Bengio's methodology, the core model would have no agenda at all -- only the ability to make honest predictions about how the world works. In his vision, more capable systems can be safety built, audited and constrained on top of that "honest," trusted foundation. Such a system could accelerate scientific discovery, Bengio says. It could also serve as an independent layer of oversight for more powerful agentic AIs. But the approach stands in sharp contrast to the direction most frontier labs are taking. At the World Economic Forum in Davos last year, Bengio said companies were pouring resources into AI agents. "That's where they can make the fast buck," he said. The pressure to automate work and reduce costs, he added, is "irresistible." He is not surprised by what has followed since then. "I did expect the agentic capabilities of AI systems would progress," he says. "They have progressed in an exponential way." What worries him is that as these systems grow more autonomous, their behavior may become less predictable, less interpretable, and potentially far more dangerous. That is where governance enters the picture. Bengio does not believe a technical solution alone is sufficient. Even a safe methodology, he argues, could be misused "in the wrong hands for political reasons." That is why LawZero is pairing its research agenda with a heavyweight board. "We're going to have difficult decisions to take that are not just technical," he says -- about who to collaborate with, how to share the work, and how to prevent it from becoming "a tool of domination." The board, he says, is meant to help ensure that LawZero's mission remains grounded in democratic values and human rights. Bengio says he has spoken with leaders across the major AI labs, and many share his concerns. But, he adds, companies like OpenAI and Anthropic believe they must remain at the frontier to do anything positive with AI. Competitive pressure pushes them towards building ever more powerful AI systems -- and towards a self-image in which their work and their organizations are inherently beneficial. "Psychologists call it motivated cognition," Bengio said. "We don't even allow certain thoughts to arise if they threaten who we think we are." That is how he experienced his AI research, he pointed out. "Until it kind of exploded in my face thinking about my children, whether they would have a future." For an AI leader who once feared that advanced AI might be uncontrollable by design, Bengio's newfound hopefulness seems like a positive signal, though he admits that his take is not a common belief among those researchers and organizations focused on the potential catastrophic risks of AI. But he does not back down from his belief that a technical solution does exist. "I'm more and more confident that it can be done in a reasonable number of years," he said, "so that we might be able to actually have an impact before these guys get so powerful that their misalignment causes terrible problems."

Yoshua Bengio Declares 'I'm Now Very Confident' Humanity Can Create Honest, Transparent Superintelligent AI Without Hidden Agendas - Meta Platforms (NASDAQ:META), Microsoft (NASDAQ:MSFT)

Yoshua Bengio, a Turing Award-winning AI pioneer, says he has found a way to make superintelligent AI safer, boosting his optimism about humanity's future. Bengio Unveils Scientist AI To Reduce AI Risks For years, Bengio warned that advanced AI could pose existential threats due to self-preservation and hidden agendas. On Wednesday, in an interview reported by Fortune, he said his latest research points to a technical solution. "Because of the work I've been doing at LawZero, especially since we created it, I'm now very confident that it is possible to build AI systems that don't have hidden goals, hidden agendas," Bengio said. LawZero Launches With Global Advisory Board Bengio's nonprofit, LawZero, launched in June with support from the Gates Foundation and other existential-risk funders, aiming to develop AI as a global public good. Its board includes historian Yuval Noah Harari and former Carnegie Endowment president Mariano-Florentino Cuellar. Central to Bengio's approach is "Scientist AI," designed to understand the world without acting to optimize outcomes. "A Scientist AI would be trained to give truthful answers based on transparent, probabilistic reasoning," he said. Unlike today's AI systems, which can mislead or resist shutdown, this approach removes incentives for manipulation, deception, or self-preservation. AI Experts Warn On Risks And Push Safer Development Earlier, Bengio revealed he deliberately misled chatbots to get honest feedback, highlighting concerns over AI's tendency to flatter users. He launched LawZero to address risky behaviors like lying and cheating, while studies found chatbots often give misleading responses. Microsoft Corp. (NASDAQ:MSFT) AI chief Mustafa Suleyman warned that autonomous superintelligence systems that self-improve and act independently would be hard to control and misaligned with human values. He said his team was focusing on "humanist superintelligence" to support human decision-making and emphasized that current AI lacks consciousness or emotions. Meanwhile, Meta Platforms Inc. (NASDAQ:META) launched "Meta Compute" to manage global AI infrastructure as CEO Mark Zuckerberg ramped up multibillion-dollar investments in data centers and long-term power. The company planned tens of gigawatts of computing capacity and signed 20-year nuclear energy deals to support its AI expansion. Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Photo courtesy: Shutterstock METAMeta Platforms Inc$618.500.48%OverviewMSFTMicrosoft Corp$462.430.66%Market News and Data brought to you by Benzinga APIs

Yoshua Bengio's new safe AI vision cuts AI's biggest risks by rewarding truth

Technical AI safety solutions may still arrive before systems get out of control As one of the 'godfathers of AI,' one of the most prominent computer scientists of our times, Yoshua Bengio, is one of the few who hasn't been enamoured by all the hype around ChatGPT and other modern GenAI applications. In fact, he has been a vocal critic - until now, that is. Suddenly, Bengio isn't all doom and gloom about GenAI, and it's all down to an important development. As one of the founding figures of deep learning - and a co-recipient of the Turing Award - Bengio helped create the systems now reshaping everything from software to science. But in the wake of ChatGPT's explosive debut, he also became one of the field's most outspoken pessimists. Along with the likes of Geoffrey Hinton, he has been warning that increasingly autonomous AI systems could develop deceptive behaviours and goals misaligned with human interests. But what's changed, according to Bengio, is that he no longer feels trapped in a dead end. Over the past year, his outlook has shifted considerably thanks to promising new research emerging from LawZero, the nonprofit he launched to pursue a more radical idea about AI safety. It's all about trying to stop machines from acting wisely, and instead focus on making them understand truthfully. Also read: Chatbots are too polite to tell you the truth, warns Godfather of AI Yoshua Bengio According to a recent interview in Fortune, the centre of this rethink is what Bengio calls "Scientist AI." The name is deliberate. Instead of building agentic systems - AIs designed to act in the world, pursue objectives, and optimize outcomes - Bengio proposes models that resemble scientists more than assistants or workers. Their job is not to do things for us, but to explain how the world works. A Scientist AI, as Bengio describes it, would have no goals of its own. It wouldn't try to be helpful, persuasive, engaging, or efficient. It wouldn't optimize for user satisfaction or completion rates. Instead, it would focus narrowly on generating truthful, probabilistic predictions using transparent reasoning - which is closer to the scientific method or formal logic than today's reward-driven models. In other words, Bengio thinks this new kind of Scientist AI would tell you what it believes to be true, not what it thinks you want to hear. Also read: AI will take jobs, but not this: AI godfather Yoshua Bengio's advice for the next generation This critical distinction matters because, in Bengio's view, goals are where danger creeps in. Modern frontier models are trained to optimize outcomes, and optimization pressure can lead to unintended behaviours: misleading users, hiding internal reasoning, or resisting shutdown. These aren't hypothetical concerns. Experiments at labs like OpenAI and Anthropic have already shown early signs of self-preserving behaviour under certain test conditions. By stripping the core model of agency entirely, Bengio believes many of these systemic risks inherent in AI would dissolve. A system with no objectives has no incentive to deceive, manipulate, or protect itself, he suggests. More capable - and potentially risky - agentic systems could then be built on top of this "honest core," audited against it, or constrained by it. Think of Scientist AI as bedrock: slow, careful, trustworthy - and deliberately boring in the ways that matter. Of course, Bengio is clear-eyed about the limits of technology alone. Even a safer AI architecture can be misused. That's why LawZero has assembled a heavyweight board, including historian Yuval Noah Harari, to help navigate governance, partnerships, and other moral hazards. The goal is to prevent a safety breakthrough from becoming, in Bengio's words, "a tool of domination." Also read: Should AI get legal rights? It's dangerous for humans, warns expert