AI Pioneer Yoshua Bengio Warns of Potential Risks and Power Concentration in Advanced AI Systems

Godfather of AI Warns of Powerful People Who Want Humans "Replaced by Machines"

"Intelligence gives power. So who's going to control that power?" One of the world's leading AI pioneers is warning not only that the increasingly powerful technology could harm the world, but that the people in charge of it may be equally dangerous. In an interview with CNBC, computer science luminary Yoshua Bengio said that members of an elite tech "fringe" want AI to replace humans. The head of the University of Montreal's Institute for Learning Algorithms, Bengio was among the public signatories of the "Right to Warn" open letter penned by leading AI researchers at OpenAI who claim they're being silenced about the technology's dangers. Along with famed experts Yann LeCun and Geoffrey Hinton, he's sometimes referred to as one of the "Godfathers of AI." "Intelligence gives power. So who's going to control that power?" the preeminent machine learning expert told the outlet during the One Young World Summit in Montreal. "There are people who might want to abuse that power, and there are people who might be happy to see humanity replaced by machines," Bengio claimed. "I mean, it's a fringe, but these people can have a lot of power, and they can do it unless we put the right guardrails right now." Because AI systems cost billions of dollars to build, the deep learning expert noted, the financial power aspect can't be ignored. "Very few organizations and very few countries will be able to do it," the University of Montreal AI lab leader said. "That's already the case." "There's going to be a concentration of power: economic power, which can be bad for markets; political power, which could be bad for democracy; and military power, which could be bad for the geopolitical stability of our planet," he continued. Currently, experts disagree on whether artificial general intelligence, or human-level AI, can be reached and when it might happen. If it happens before global policy catches up, Bengio suggests, we're in trouble. "If it's five years, we're not ready," he concluded, "because we don't have methods to make sure that these systems will not harm people or will not turn against people." While far from the first AI expert to offer warnings about the dangers of AI, Bengio has unusual gravitas; after all, he helped create it.

AI systems could 'turn against humans': Tech pioneer Yoshua Bengio warns of artificial intelligence risks

Professor Yoshua Bengio, at the One Young World Summit in Montreal, Canada, on Friday, Sept. 20, 2024 Famed computer scientist Yoshua Bengio -- an artificial intelligence pioneer -- has warned of the nascent technology's potential negative effects on society and called for more research to mitigate its risks. Bengio, a professor at the University of Montreal and head of the Montreal Institute for Learning Algorithms, has won multiple awards for his work in deep learning, a subset of AI that attempts to mimic the activity in the human brain to learn how to recognize complex patterns in data. But he has concerns about the technology and warned that some people with "a lot of power" may even want to see humanity replaced by machines. "It's really important to project ourselves into the future where we have machines that are as smart as us on many counts, and what would that mean for society," Bengio told CNBC's Tania Bryer at the One Young World Summit in Montreal. Machines could soon have most of the cognitive abilities of humans, he said -- artificial general intelligence (AGI) is a type of AI technology that aims to equal or better human intellect. "Intelligence gives power. So who's going to control that power?" he said. "Having systems that know more than most people can be dangerous in the wrong hands and create more instability at a geopolitical level, for example, or terrorism." A limited number of organizations and governments will be able to afford to build powerful AI machines, according to Bengio, and the bigger the systems are, the smarter they become. "These machines, you know, cost billions to be built and trained [and] very few organizations and very few countries will be able to do it. That's already the case," he said. "There's going to be a concentration of power: economic power, which can be bad for markets; political power, which could be bad for democracy; and military power, which could be bad for the geopolitical stability of our planet. So, lots of open questions that we need to study with care and start mitigating as soon as we can."

Why an AI Pioneer Worries the Could Turn Against Us

We should listen to Bengio. Not least because in 2018 he won the Association for Computing Machinery "Turing Award," often considered the Nobel Prize for computing. In the 1990s Bengio worked alongside prominent AI thinkers Geoffrey Hinton and Yann LeCunn, researching and developing many core principles and machine learning protocols that went into the AI systems that you've probably already used today. Bengio spoke to CNBC at the summit, held by a nonprofit that aims at "Empowering and developing young leaders to build a fair, sustainable future for all." He outlined his worries about the way the direction of today's cutting-edge AI. He's well aware that developing next-generation AIs is going to be an expensive business, for example, with organizations like Google planning $100 billion-scale investments, and to Bengio this spells trouble. Not every country can afford this luxury, Bengio said, noting this could lead to a "concentration of power: economic power, which can be bad for markets; political power, which could be bad for democracy; and military power, which could be bad for the geopolitical stability of our planet." He's also sure that advanced, truly smart next-gen AIs will come online during a decades-long timescale, but if this happens as soon as five years, Bengio thinks we're just not ready. "We don't have methods to make sure that these systems will not harm people or will not turn against people," he said -- tapping into a vein of concerns about the difficulty of regulating AI's capabilities.