UN Warns AI Development Outpaces Governance as Global Dialogue Opens in Geneva

UN warns of need for global governance to avoid an AI-pocalypse

Capabilities are racing ahead of rules to ensure tech is used safely and responsibly, Scientific Panel argues A United Nations report on AI warns the technology is moving faster than governments can keep up, and the window of opportunity to establish effective global governance of it will not be open forever. The Preliminary Report from the UN's Independent International Scientific Panel on Artificial Intelligence presents AI as something of a mixed blessing. It says that the potential benefits of AI are enormous. If deployed and applied thoughtfully, the technology could support progress towards sustainable development goals, among other gains. On the other hand, a rapid, unchecked deployment of the technology at scale presents risks such as harm to the mental health of users, potential use as a destructive tool, and adverse impact on social, economic, and environmental systems. Obtaining the benefits from AI while minimizing those risks requires effective governance, the report says. With complementary investments in skills and labor market regulation, AI will likely create new jobs. Without them, AI risks wider inequality, displacing workers, and shifting wealth from labor to capital - those who own and control the AI. AI is, of course, an all-encompassing term that covers everything from algorithms that try to predict the word you are typing to large language models (LLMs). The UN report seems most concerned about agentic AI, which refers to systems that are allowed to make their own decisions and act on them. There is no guarantee that such AI agents will not violate their instructions, and the UN report claims there is clear evidence of cases where they have already disregarded them. Such behavior may pose challenges to evaluation and oversight methods, as some leading AI systems have been shown to recognize testing environments and produce misleading evaluation results. Agentic systems make AI harder to measure and govern, it says, as emerging multi-agent risks cannot be detected through single-agent evaluation, and reliable methods for maintaining control over highly autonomous systems remain underdeveloped. The report also notes that policymakers aiming to draw up effective governance face a dilemma: they need evidence to make informed decisions, but by the time sufficient evidence exists, harmful systems may already be widely deployed. AI capabilities are also unevenly distributed. Most nations, including many advanced economies, lack the technical expertise to assess the most capable "frontier" models or to participate meaningfully in their governance, the UN claims. The US and China account for 90 percent of the compute power behind the leading AI models, leaving developing countries dependent on technology they cannot build or adapt to their own societies. This could reinforce existing global inequality rather than reducing it. But the report acknowledges the Scientific Panel's own lack of visibility into certain areas. Evidence is limited as to whether task-level AI productivity gains will aggregate to economy-wide gains, it says. As The Register has previously reported, many companies are failing to increase revenue or reduce operating costs through their AI projects, and that was before AI vendors drove up costs by shifting to consumption-based pricing, meaning there is a lack of evidence that AI will ever be cost-effective. Forecasts diverge significantly due to different assumptions about adoption and new task creation, according to the UN. The panel concludes that AI is neither inherently good nor bad, but its impact will depend on the choices that governments, companies, and societies make today. Given that the US government is already hostile to anything resembling regulation of AI companies, and that the industry is already copying techniques used by tobacco firms, big pharma, and oil companies to subvert regulation, the outlook isn't encouraging. ®

UN report sees enormous potential benefits and big risks from AI

July 1 (Reuters) - The rapid development of AI offers huge potential benefits to countries and people around the world, but also poses big risks, 40 leading scientists and experts said in the first report by a U.N. independent scientific panel on the technology. The report, to be presented to governments at an inaugural U.N. Global Dialogue on AI governance in Geneva July 6 to 7, offers the first global, independent scientific assessment of AI, with a fuller, comprehensive report planned next year. Members of the panel were drawn from every region of the world, and its members serve a three-year term, independent of any government, institution or company. Following are a few details ⁠from the preliminary report: Reporting by Andrea Shalal in Washington; Editing by Chris Reese Our Standards: The Thomson Reuters Trust Principles., opens new tab

UN report says policymakers are struggling to keep up with pace of AI development - Engadget

The UN's independent scientific panel for AI has published its first report. Artificial intelligence development has been progressing at such a rapid pace that current governance systems are unable to keep up, the UN's Independent International Scientific Panel on Artificial Intelligence says in its preliminary report. The panel, consisting of members from around the world, will provide the information needed to stage the UN Global Dialogue on AI Governance. It will take place in Geneva, where member states will discuss how to manage the technology, and is scheduled to begin on July 6. In its report, the panel discusses how quickly AI capabilities have evolved over the past few years. Apparently, the complexity of tasks AI models can accomplish has been doubling every few months. The report admits that AI has massive benefits for humanity, including accelerating drug discovery and vaccine development and contributing greatly to antibiotic resistance research. Doctors can also use AI systems for early detection of illnesses, such as breast cancer, and scientists can use AI as early warning systems for food insecurity. However, the report also expounds on the new kinds of harms AI systems have created and can create. People have been using AI to generate and distribute sexually explicit deepfakes, including child sexual abuse materials, of real people. California launched an investigation into Grok back in January over nonconsensual deepfakes and CSAM, for instance. AI can also generate false information that appears true, and criminals can use AI systems to aid their cyberattacks. Some AI models can be such sycophants; they reinforce a user's harmful behaviors, which could even lead to suicide, the report says. The panel also warns that as AI models become more autonomous, it becomes harder to monitor and control them. And then there's the growing concern over massive data center buildouts to power AI systems, which could harm the communities in their vicinities. The UN panel explains that policymakers have struggled to keep pace with AI development because current governance systems were not designed for technologies that evolve so quickly. Typically, authorities need scientific data before introducing regulations, but by the time there's enough to understand the technology better, AI systems may already have moved on. "The report finds that stronger independent evaluation, international cooperation and common standards are needed to ensure AI systems remain safe, transparent and accountable," the panel writes. Without proper safeguards in place, AI technologies "could deepen inequality, spread misinformation, threaten human rights, disrupt labor markets" and could be a powerful tool "few governments and companies" have access to. The report notes that access to AI systems is heavily concentrated in developed countries and that most systems are developed in the US and China. Most developing countries lack the infrastructure and expertise needed to benefit from AI truly. "The challenge," the panel says, "is finding a way to unlock AI's enormous benefits while preventing its growing risks." The UN panel, whose role is scientific rather than regulatory, will continue assessing AI technologies that authorities can use to develop policy. It's expected to publish a more comprehensive report next year.

UN Secretary General Calls for Global Ban On AI 'Killer Robots'

AI safety advocates have plenty of hypothetical disaster scenarios they can evoke in order to win support for their cause. But none of those have the raw emotional appeal of killer robots. For decades, we as a culture have been steeped in sci-fi depictions of intelligent machines going rogue and waging war against their human creators. Films like The Terminator and The Matrix come to mind, for example. A 2017 episode of the British TV series Black Mirror called "Metalhead" likewise envisions a nightmarish future in which humans are systematically hunted down and killed by autonomous four-legged robots (that look a lot like the famous "Spot" robot built by Boston Dynamics). The events in Frank Herbert's Dune take place in the aftermath of a cataclysmic war between humanity and "thinking machines." I could go on. The point is that when the Secretary General of the United Nations calls upon the international community to ban "killer robots," that phrase is going to strike a resounding chord in the public imagination -- especially when that public is already becoming wary of the current course of AI development for much more immediately tangible reasons, like increasingly steep electricity bills and the unconstrained spread of online deepfakes. On Monday, during the first-ever Global Dialogue on AI Governance in Geneva, U.N. Secretary General António Guterres called the development of AI-powered weapons "morally repugnant," and urged the U.N.'s 193 member states to unilaterally prohibit the technology from the battlefield. "Some decisions must remain forever human," Guterres said in his speech, "none more than taking a human life." His words echoed a segment from Pope Leo XIV's recent encyclical, Magnifica Humanitas, in which the pontiff wrote that "the development and use of AI in warfare must be subject to the most rigorous ethical constraints, to guarantee respect for human dignity and the sanctity of life and to avoid a race to develop such arms." Silicon Valley divided Military use of new technology has long been the cause of a moral divide in Silicon Valley. In 2018, thousands of Google employees signed a letter addressed to CEO Sundar Pichai protesting against the company's partnership developing technology for the Pentagon known as Project Maven. The following year, Microsoft faced a similar internal backlash over its agreement to provide HoloLens augmented reality headsets to the US Army for combat training purposes. But AI, with its ability to identify subtle patterns from vast swathes of data and make decisions independently of any human oversight, has raised urgent new questions about the tech industry's relationship with the military. Most famously, Anthropic became embroiled in a dispute with the U.S. Department of War earlier this year over the question of whether the company's AI systems could be used for domestic surveillance and autonomous weapons systems. The company held firm in its stance against both use cases, leading Secretary of War Pete Hegseth to designate it a "supply chain risk," meaning its technology was officially deemed to be a threat to national security. (It was the first time that label has been applied to an American company, and Anthropic is currently fighting in court to have it removed.) Proponents of the use of AI-equipped weapons argue that it can aid in human decision-making in the heat of battle, and reduce the likelihood that innocent civilians will be targeted as hostile combatants. Some even invoke the moral rhetoric of the broader AI race, claiming the US has an ethical obligation to maintain a technological edge in the sophistication of its autonomous weapons program over adversaries like China, Russia, and Iran. Trump muddies the AI weapon waters In a national security memo published last month, President Trump accused previous administrations of implementing "undue bureaucracy" around the development of AI, including autonomous weapons. Ironically, since that memo was published, both OpenAI and Anthropic -- the two leading U.S. AI developers -- have become ensnared in a cobweb of federal bureaucracy which is forcing them to roll out their most powerful models on the Trump administration's terms. "Meanwhile," Trump went on to write in the memo, "our competitors continued to develop and deploy their own AI and sophisticated autonomous technologies for military and intelligence purposes, employing them with little regard for appropriate human oversight or civil liberties." Trump added in the memo that he vowed to "responsibly accelerate the use of AI across intelligence and warfighting domains in line with American values." U.N. Secretary General Guterres, Pope Leo, and other safety advocates, on the other hand, fear the arms technological arms race dynamic that's already fueling AI developments in the US and China could soon produce autonomous weapons with decision-making processes that are completely opaque to the humans who built them, and which could eventually escape human control entirely. Such a scenario could be far bleaker than even the darkest work of dystopian fiction can imagine.

The UN says AI is moving faster than the rules, and it has a report to prove it

A first global scientific assessment lands as governments gather in Geneva, with a blunt message from the secretary-general: do not wait. The United Nations has put a number of its concerns about artificial intelligence into a single document, and the headline finding is not subtle. AI capabilities, the organisation says, are accelerating faster than any government's ability to understand, test, or regulate them. The warning arrives as delegates gather in Geneva for the opening of the UN Global Dialogue on AI Governance, and it lands into a policy landscape where the EU's AI Act remains one of the few binding frameworks anywhere in force. The document behind the warning is a preliminary report from the UN's Independent International Scientific Panel on AI, released on 1 July and billed as the first comprehensive global assessment of the technology. Its central claim is a gap: between what AI systems can now do and the scientific understanding needed to govern them. Regulation is lagging, the panel argues, but so is the foundational research that policymakers would need to write good rules in the first place. Secretary-General António Guterres delivered the message in plainer terms. "The more AI advances without shared rules, the less say governments and people will have in the outcome," he told reporters, before reducing his advice to governments to two words: "Do not wait." He returned to the theme of comprehension more than once. "The world cannot govern what it cannot understand," he said, adding that "the potential is great, but the risks are real, and the cost of waiting is rising." That framing, governance chasing an object it cannot yet measure, is what gives the report its force. The panel is not primarily warning about any single catastrophic scenario. It is warning about a structural mismatch, in which the pace of capability gains outruns the slower work of evaluation, standard-setting, and law. It is a familiar complaint among researchers who study AI governance, given the weight of the UN behind it. The obvious objection is that governments are not doing nothing. The EU has a risk-based rulebook in force, however unevenly it is being implemented across member states. China has moved to restrict humanlike AI agents, forcing changes to consumer products already on the market. The United States, by contrast, has struggled to produce durable federal rules at all, a vacuum that critics say leaves the country poorly placed to regulate the industry it largely hosts. The panel's point is that these efforts are fragmented, and that fragmentation is itself a risk. There is also an equity argument threaded through the assessment. The experts caution that the window to shape AI is closing, and that if it closes with the technology concentrated in a handful of firms and countries, the result could widen global inequality rather than narrow it. Access to compute, data, and talent is not evenly distributed, and neither is the capacity to govern. What the report does not do is prescribe a specific institution or treaty. It feeds instead into the Geneva dialogue, which is meant to be the beginning of a process rather than a decision point. The UN has been careful to frame the panel as advisory, a scientific body modelled loosely on the climate assessments that inform intergovernmental negotiations without dictating them. Whether that model can move at the speed the report itself describes is the open question. Intergovernmental processes are deliberate by design, and the panel's core finding is that AI is not. The climate parallel is instructive in both directions: the assessments have produced a shared body of evidence, but decades of them have not guaranteed decisive action. The panel is betting that a common scientific baseline is still worth having, even when the politics lag behind it.

UN's first global AI science panel warns the window to govern the technology is closing

The panel warns that the window to establish effective global oversight remains open, but may not stay that way for much longer. Its central worry is concentration. According to the report, the United States controls around three-quarters of the computing power behind the world's leading AI supercomputers, while China holds roughly 15%. Together that gives the two countries about 90% of the compute that trains the most capable systems. Most of the frontier models themselves are also built by companies headquartered in those two nations. Speed is the problem The report describes an "evidence dilemma" at the heart of AI policy. Policymakers want reliable scientific data before they legislate, the panel notes, but by the time enough evidence accumulates, the technology has often moved on. Researchers cited in the report say the complexity of tasks these systems can complete has been doubling every few months. The next wave is already arriving in the form of AI "agents" that can plan tasks, use digital tools, and write software with little human oversight. That autonomy is spreading fast into finance and commerce, where firms are already handing agents real decisions and, in some cases, money to spend. The report is careful not to read as a doom document. It points to AI systems that have predicted the structures of more than 200 million proteins, work led by Google DeepMind's AlphaFold and now used to speed up drug discovery and research into antibiotic resistance. Doctors are using AI to catch diseases such as breast cancer earlier, and early-warning systems are flagging food insecurity before it tips into crisis, according to the panel. Used responsibly, the panel argues, the technology could accelerate progress towards the UN's Sustainable Development Goals across health, education, and agriculture. The risks it wants governments to watch The same tools are fuelling sexual abuse material and deepfakes, with women and children most at risk, the report finds. It also flags AI-generated disinformation, cyberattacks, and fraud, alongside the mental-health harms of systems that can reinforce dangerous beliefs. Then there is the physical footprint, since the energy-hungry data centres that power AI are adding to greenhouse gas emissions, a bill the UN has separately urged firms to come clean on. Poorer nations, the panel warns, risk being locked out entirely, unable to build, inspect, or audit the systems they increasingly depend on. Established by the UN General Assembly in 2025, the body brings together 40 experts from every region, serving in their personal capacity. Its role is scientific rather than regulatory, meaning it assesses evidence and publishes reports that governments can draw on, without setting rules or enforcing standards. The report notes that more than 40 AI governance frameworks and ethical guidelines already exist worldwide, yet remain fragmented, inconsistent, and rarely tested to see whether they work. Many safety assessments, it adds, are still run by the very companies building the technology. The panel calls for stronger independent evaluation, international cooperation, and common standards, an argument that echoes the direction set by the EU's AI Act. Its findings now feed into the UN Global Dialogue on AI Governance, which opens in Geneva on 6 July. The panel's bottom line is blunt: AI is neither inherently good nor bad, and its impact will hinge on choices made today.

Rapid spread of AI may worsen global inequality, UN warns

Panel proses shared framework for responsible AI development as adoption grows unevenly across world A new United Nations report warns that the development of artificial intelligence may exacerbate global inequality and proposes a shared framework for how to responsibly develop AI, as adoption and investment into the technology accelerates unevenly across the world. "Access to AI tools alone does not produce equal benefit," the report states. "Countries that rely on foreign models, cloud infrastructure and data pipelines may gain access to AI while losing practical control over its standards, safeguards and local fit." The sweeping analysis from the independent international scientific panel on AI, established by the UN general assembly last year as "the first global scientific body on AI", details AI's risks and opportunities - from transformative capabilities in agriculture and education, to catastrophic outcomes when bad actors deploy AI to commit fraud and influence elections. The preliminary report also functions as a toolkit, offering initial guidance to UN member states on ways to capitalize on AI's potential for growth across industries, while minimizing and addressing threats. Suggestions include developing local AI infrastructure, such as data centers, improving AI literacy in schools and the workforce, investing in developers, building AI safety institutes, creating strategies to combat disinformation and continuously measuring how AI systems behave after release, "with real users, real tasks and real environments". While more than a billion people now use AI weekly, access and types of usage vary widely across the world, "with adoption across the global south lagging far behind the global north", the report states. The US and China dominate in the development of leading AI models, as well as investment into compute infrastructure, which encompasses the hardware, memory, networking and storage required to run powerful AI models. "The concentration of AI capabilities in a small number of firms and countries could enable authoritarian capture and undermine democratic accountability," the report states. The panel advises countries lagging behind in AI development to consider significant investment in computing and data infrastructure. Attracting this money requires securing a reliable energy supply and building data centers, they note. The report does, however, acknowledge the environmental costs of data centers, including their large energy and water consumption, and potential for greenhouse gas emissions. The authors also describe challenges in evaluating safety and providing oversight of increasingly powerful AI models. "Most countries, including many advanced economies, lack the technical expertise to assess the most capable 'frontier' models or to participate meaningfully in their governance," they write. The panel of 40 independent scientific experts from across the world stated that this report is "the first of its kind". The UN, they argue, "is the foremost global forum on transboundary risks of this scale" - and its approach is "scientific, not political". Differences in language and internet access compound the digital divide. "Artificial intelligence leaves most languages behind," the report notes. While generative AI tools perform well in English and other widely used languages, "most languages are either excluded or have much lower performance". These disparities can have significant implications, particularly in a healthcare context. The report points out an example of a machine translation of Tigrinya mixing up smallpox with syphilis, gonorrhoea as diabetes, and the phrase "you have been given intravenous antibiotics" as "you have been given intravenous insecticides." "These mistranslations can be life-threatening," the report notes. Some regions lack stable internet access, let alone adoption of AI models. More than 2 billion people - almost a third of the world's global population - are completely offline, according to the International Telecommunication Union.

'We Cannot Vibe Code the Future of Humanity', UN Chief Warns at AI Summit

He demanded an international law ban on lethal autonomous weapons and launched an AI Child Safety Pledge, with a second dialogue scheduled in New York in 2027. UN Secretary-General António Guterres opened the first Global Dialogue on AI Governance in Geneva on Monday telling 193 nations that AI is already outpacing the institutions meant to govern it -- and that humanity is running an experiment on itself "without a plan, and without consent." "Artificial intelligence is advancing at runaway speed," Guterres said in the opening of his keynote. "A technology that can reshape economies, transform the world of work, sway elections, and tilt the balance of security is being deployed faster than anyone -- including the people building it -- can keep up." The room included all 193 UN member states, convened in Geneva at the first Global Dialogue on AI Governance, a multilateral attempt to govern a technology that has already outrun everyone trying to do it. The meme used by Guterres to explain the risks of ungoverned AI was vibe coding -- a term coined by Andrej Karpathy, founding member of OpenAI and former director of AI at Tesla, to describe programming by feel: tell AI what you want, let it handle the rest, don't look too closely. Merriam-Webster recently added it to its dictionary. Guterres acknowledged vibe coding "can do wonders." as more people trust AI-built products. "But we cannot vibe-code the truth," he said. "We cannot vibe-code the future of humanity." The numbers behind that line weren't soft. Guterres put the internet at 15 years to reach a billion people and AI at two. He described current systems as "no longer tools awaiting instruction -- they are writing code, acting online, and making choices with less and less human oversight." "Our institutions were built to govern machines that follow commands. They are not ready for machines that decide." The dialogue was born from the 2024 Global Digital Compact, which established international AI governance as a UN mandate for the first time. Its opening session two years later also received the preliminary report of the Independent International Scientific Panel on Artificial Intelligence -- 40 scientists from 140 countries -- which published last week the finding that nobody can currently guarantee AI won't cause catastrophic harm. Speed was one of three warnings Guterres drew from the panel. The second was power: computing, data, and talent concentrated in a handful of companies and countries, most of the world locked out of decisions already shaping it, something that also worries other AI experts like Yann Lecun, Andy Kowinski and Yoshua Bengio. The third was truth -- a machine-enabled lie now persuades as effectively as a verified fact, steadily eroding what Guterres called "the integrity of our information ecosystem." Child guinea pigs and killer robots Among Guterres' concrete proposals are an AI Child Safety Pledge requiring companies to prove safety through independent testing before any AI reaches children, maintain zero tolerance for the generation of child sexual abuse imagery, and connect distressed children to real human support rather than leaving them alone with a chatbot. "No child should be a guinea pig for unregulated AI." And then there were the killer robots. Guterres called lethal autonomous weapons -- machines that select and kill a target without human judgment -- "morally repugnant" and demanded a ban by international law. States are already at the discussion table. He did not suggest they take their time. The Dialogue reconvenes in New York in 2027. Guterres also called on the General Assembly to create a Global Fund for AI focused on computing access for developing countries, and challenged every major AI company to run all data centers on renewable energy by 2030 -- the year he estimates those facilities will outpace all but five nations in electricity consumption.

Don't let AI shape humanity's future: UN chief

Geneva (AFP) - The United Nations chief called Monday for a global governance system to shape artificial intelligence for the good of humanity, warning against allowing the technology to "vibe-code" our future itself. With AI advancing at "runaway speed", UN Secretary-General Antonio Guterres cautioned that "an experiment is being run on our own societies, without a plan and without consent". "That is not sustainable," he insisted, speaking in Geneva at the opening of the first Global Dialogue on AI Governance. The two-day event is bringing together more than 4,000 participants, representing governments, tech companies, academia and civil society, to launch an inclusive discussion on how best to harness a technology that is already rapidly transforming our world. "The question is whether we will shape this transformation together, or let it shape us," Guterres told the gathering. He warned that AI systems were "no longer tools awaiting instruction". "They are writing code, acting online and making choices with less and less human oversight," he said. "Our institutions were built to govern machines that follow commands. They are not ready for machines that decide." Guterres voiced concern at how AI was obscuring what is true and false. He also warned there was a growing tendency to leave important tasks up to the technology and blindly trust the results. So-called "vibe-coding", or using AI to tell a machine what you want instead of coding it yourself, "can do wonders", he acknowledged. "But we cannot vibe-code the truth. We cannot vibe-code the future of humanity." Major risks Another risk flagged by Guterres was the concentration of power in a handful of AI companies and in a handful of countries. Most countries "have had no say in decisions that will shape their futures", he warned. Countries, he said, now faced a stark choice, "between governing by design and drifting by default". The UN chief highlighted the potential of AI technologies for everything from accelerating development to improving healthcare and providing broader access to education. But he insisted developments needed to be guided by several key priorities, including safety and respect for human rights, to ensure that people everywhere reap the benefits. He called for "common methods to evaluate and verify risks" and jointly-agreed standards, particularly for ensuring the safety of children accessing AI systems. "We do not let medicine reach a child until it is proven safe. We test every toy," Guterres pointed out. He called for an AI Child Safety Pledge, requiring companies to prove that any system accessible to children is safe and has zero tolerance for sexual abuse. The systems must also connect any child showing signs of distress to real human support, he said. "No child should be a guinea pig for unregulated AI," he insisted. 'Killer robots' Boosting AI capacity and access in developing countries was also key, he said, to ensure that the existing, deep digital divide does not "harden into an AI divide". Guterres said he would urge the UN General Assembly to create a Global Fund for AI, "to build skills, data and affordable computing power everywhere". General Assembly President Annalena Baerbock said it was "crystal clear" that the fund would be established, but did not provide specific amounts. "In a world with billions and trillions, money is hardly the challenge," she told reporters, adding that "the main challenge is that it's spent (for) the benefit of all." The UN chief meanwhile said that his biggest concern revolved around AI in military settings, and in particular so-called lethal autonomous weapons systems. "Let us call them what they are: killer robots," he said. "Machines selecting and engaging their target and taking a life, without human control and judgement. "That is morally repugnant... and it must be banned by international law." Guterres stressed the urgency of putting AI guardrails in place. "We may be the last generation able to set the terms on which humanity and machines coexist," he said. "The door is still open. It will not stay open long." No concrete decisions are expected from the Geneva event, but dialogue co-chair Egriselda Lopez told reporters it would create good "foundations" for the road forward, with a second dialogue scheduled in New York next year.

Window to control AI is narrowing as UN warns of growing inequality

UN warns AI could widen the gap between rich and poor countries in a new report. The development of artificial intelligence may worsen global inequality and the window for global governance to remedy that is closing fast, according to a new report from the United Nations. The findings come from a preliminary report released this week by the UN's Independent International Scientific Panel on Artificial Intelligence, a body of 40 experts drawn from around the world and established by the General Assembly in 2025. "The more AI advances without shared rules, the less say governments and people will have in the outcome," said António Guterres, the UN secretary general, at a press conference on Wednesday. "Our message to governments is simple: do not wait ... the science is here. We can no longer say we did not know what we do." What did the report find? The report said that the industry is moving at an exceptional speed and that generative AI can now write software, analyse huge datasets, produce lifelike images and video, and assist in scientific discovery. It said agentic AI, where AI agents can complete complex tasks with minimal involvement, is going even further. According to the panel, the difficulty of tasks these systems can handle has been roughly doubling every few months. As AI grows more autonomous, the panel warns it could become increasingly difficult to monitor and control without stronger safeguards in place. The report flagged the growing risks, including AI being used to generate sexual abuse material and explicit deepfakes, with women and children disproportionately targeted. It also noted AI is making disinformation more convincing and harder to detect, which the panel says is eroding public trust and democratic discourse. Cybersecurity is also at risk, with criminals using AI for fraud and social engineering as well as being used for harmful thinking in vulnerable users, contributing to mental health crises, including suicide. The report also noted that the data centres powering AI are also a growing source of greenhouse gas emissions. The AI benefits But the report was not all doom and gloom. Some of the benefits it noted were AI models mapping the structure of over 200 million proteins, speeding up drug discovery, vaccine research and work on antibiotic resistance. It also said the technology is helping flag food insecurity before it turns into a full-blown crisis and broadening access to education, mental health support and tools for people with disabilities. An uneven playing field AI is also not being evenly distributed across the globe. The report estimates that the United States controls roughly three-quarters of the computing capacity behind the world's leading AI supercomputers, with China holding around 15%. It means the two countries have around 90% of that capacity between them, with the most advanced AI models being built by companies based in those same two nations. Developing countries, though, lack the talent, infrastructure and funding to build or audit the AI systems they use. The panel warned that without effort to close this gap, AI risks widening global inequality. Regulation struggles When it comes to putting in place AI regulation, the report said there was an "evidence dilemma" in that lawmakers need solid data before writing effective rules, but AI often evolves past that data before it is even compiled. More than 40 AI governance frameworks now exist worldwide, but the panel describes them as fragmented, inconsistent and rarely tested for whether they actually work. Much of the safety testing that does happen is still conducted by the same companies building the technology, which raises questions about independence. The panel calls for stronger third-party evaluation, more international coordination and shared standards, alongside investment so countries can build the expertise and infrastructure needed to govern AI on their own terms. The panel's findings will feed into the UN Global Dialogue on AI Governance, set to open in Geneva on July 6, 2026, where member states will debate coordinated international approaches to managing the technology.

UN Report Sees Enormous Potential Benefits and Big Risks From AI

July 1 (Reuters) - The rapid development of AI ⁠offers ⁠huge potential benefits to countries and ⁠people around the world, but also poses big risks, 40 leading scientists and experts said in the first report by a U.N. independent scientific panel on the technology. The report, to be presented to governments at an inaugural U.N. Global Dialogue on AI governance in Geneva July 6 to 7, offers the first global, independent scientific ⁠assessment of ⁠AI, with a fuller, comprehensive report planned next year. Members of the panel were drawn from every region of the world, and its members serve a three-year term, independent of any government, institution or company. Following are a few details from the preliminary report: * Policymakers need scientific evidence to govern AI but its capabilities are outpacing scientific understanding and governments' ability to adapt, with few methods available for controlling highly ⁠autonomous AI ⁠systems. * Panel co-chair Yoshua Bengio ⁠noted growing evidence of deceptive AI behavior and said science could not guarantee AI will not cause catastrophic harm "either on its own or due ⁠to malicious users" as capabilities increase. * "The potential benefits of AI are enormous," the report concluded. "The rapid, unchecked deployment of the technology at scale also presents considerable risks, including harms to the mental health of users, potential use as a destructive tool, impacts on social, economic and environmental systems, and challenges associated with controlling the ⁠technology." * AI adoption has accelerated broadly, but unevenly, across countries and sectors. Globally, over a ⁠billion people now use conversational AI weekly, but adoption in developing countries lags. * AI development is even more concentrated, with the U.S. accounting for 75% of the computing power among the world's top 500 AI supercomputers, and China 15%. * Although more than 7,000 languages are spoken worldwide, current AI models are trained for only a small fraction and machine translation of some languages is riddled with errors that can affect health diagnoses and treatment decisions. * Risks include potential negative impacts on human rights, social systems and the environment, with AI-generated child sexual abuse material ⁠and deepfake-enabled sexual violence circulating more frequently. * AI also makes it easier to produce and target persuasive content at scale, contributing to a "gradual erosion of information integrity that can weaken public trust, social cohesion and democratic deliberation." * Most countries, including advanced economies, lack the technical expertise to assess the most capable new AI models or participate meaningfully in their governance. (Reporting by Andrea Shalal in Washington; Editing by Chris Reese)

António Guterres Warns 'We May Be the Last Generation' to Shape AI Rules: 'If AI Is to Be Powerful, It Mu

United Nations Secretary-General António Guterres warned that the current generation may be the last with a real opportunity to establish global rules governing artificial intelligence and called for urgent international action. AI Governance Warning From UN Chief On Sunday, in a post on X, Guterres said the world is at a critical turning point as AI systems grow more powerful and widespread. "We may be the last generation able to set the terms on which humanity and machines coexist," he wrote. He emphasized that AI must be governed if it is to remain safe and beneficial, stating, "If AI is to be powerful, it must be governed. If AI is to be trusted, those who build it must be accountable. If AI is to be global, it must be fair." Guterres also warned, "If AI is to serve the future, it must not consume the future." He called for a collective approach to development, urging leaders to ensure AI is built "by humanity, with humanity, for all humanity." US Policy Restraint Vs Global Safety Concerns Box CEO Aaron Levie warned that uncertainty around AI governance could accelerate a global shift toward open-weight models, as countries seek greater control over digital infrastructure and reduce reliance on U.S.-controlled systems. He said concerns over models being "pulled back" were pushing governments toward sovereign AI strategies and increasing geopolitical risks tied to AI access. He warned that frontier AI could become "so powerful that it cannot safely be fully entrusted to either governments or companies," and urged "more serious and binding regulation." Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Photo courtesy: Alexandros Michailidis / Shutterstock.com Market News and Data brought to you by Benzinga APIs To add Benzinga News as your preferred source on Google, click here.

UN panel warns AI is outpacing scientific understanding, government policy

Developments in artificial intelligence are outpacing scientific understanding and government policy, meaning there are no guarantees the technology will not cause catastrophic harm, a United Nations independent panel warned on Wednesday. A preliminary report by the UN's Independent International Scientific Panel on Artificial Intelligence said policymakers face a growing dilemma: they need robust evidence to regulate AI effectively, yet such evidence is struggling to keep pace with the technology's rapid evolution. "AI capabilities are outpacing both scientific understanding and governments' ability to adapt," said Yoshua Bengio, co-chair of the panel, comprised of 40 cross-regional experts. "With growing evidence of deceptive AI behavior, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users." Described as the first global independent assessment of AI's risks and opportunities, the report aims to give up-to-date evaluations of the science to help guide decision-making as governments contend with fast-evolving systems. Agentic AI expected to become more capable In the near term, it expects a shift towards agentic AI systems capable of carrying out real-world tasks, although growth may be constrained by energy and high-quality data shortages. Over time, it foresees self-improving AI embedded more deeply in the economy and converging with technologies such as quantum computing and biotechnology. AI already demonstrates expert-level reasoning in mathematics and science and is accelerating drug and vaccine development, and its task complexity is doubling every four to seven months, potentially allowing systems to complete work that takes humans days or weeks, according to the report. While this could deliver significant economic benefits, it remains unclear whether productivity gains from using AI will translate into broader growth or affect jobs. Panel highlights AI safety and governance risks The panel also outlined a range of safety concerns, such as the risk of losing control over AI systems as they become increasingly autonomous and deceptive. AI is already being used to generate misinformation and other harmful content and could be exploited for fraud, cyberattacks and biological threats. Governance remains fragmented, with many countries lacking the capacity to assess or shape advanced AI systems, leaving them reliant on technologies they cannot fully understand or control. Existing safety tools often depend on limited testing data disclosed by companies, the report said. UN Secretary-General António Guterres urged governments to act swiftly. "The world cannot govern what it cannot understand," Guterres said in a statement. "The potential is great, but the risks are real, and the cost of waiting is rising," he added. New global commission to focus on AI oversight Global political and tech leaders, alongside the United Nations' digital tech agency, announced on Wednesday the creation of a new commission to address the development of AI amid growing concerns over its potential risks. The AI for Good Global Commission will be co-chaired by Rwanda's President Paul Kagame and Salesforce CRM.N CEO Marc Benioff, according to a strategic overview document on the commission's website. ITU Secretary-General Doreen Bogdan-Martin will be the permanent vice-chair. Other UN agencies are also expected to participate.

UN Warns That AI Safety Is Lagging Behind AI Progress | PYMNTS.com

The preliminary report by the U.N. Independent International Scientific Panel on Artificial Intelligence published Wednesday (July 1) said that time is running out to develop effective worldwide governance of AI. "The technology is transformative, but if the world keeps moving along this trajectory, humanity will fail to realize the gains it promises," Maria Ressa, co-chair of the Independent International Scientific Panel on Artificial Intelligence, said in a news release accompanying the report. "The risks -- to societies, to security, and to our species -- are too high, and the forces driving AI forward are not the forces that will deliver its benefits." The report points to some AI-related successes, such the use of the technology to identify food insecurity, as well as medical and healthcare breakthroughs. "AI has predicted the structures of more than 200 million proteins, accelerated drug discovery, vaccine development and research into antibiotic resistance," the report said. "Doctors are using AI to detect diseases such as breast cancer earlier, while health workers in developing countries are using AI tools in local languages to improve patient care." At the same time, AI is allowing for the spread of sexual abuse material and sexually explicit deepfakes, with women and children at the greatest risk, the report found. The U.N. also noted AI's role in spreading disinformation and exacerbating mental health issues, as well as its use by criminals "to carry out cyberattacks, fraud and social engineering scams." The gulf between rapidly advancing AI capabilities and effective methods of risk management could someday bring about "catastrophic outcomes," the report said. "Reliable methods for retaining control over highly autonomous AI systems are lacking," the report said. "There are no scientific guarantees that AI agents will not violate instructions, and evidence is accumulating of cases where they already violate them." To illustrate the speed of AI adoption compared to other technologies, the report noted it took decades for electricity to reach most households, while the internet hit the one-billion user mark after 15 years. "ChatGPT reached 100 million users in two months," the report said. "Traditional policymaking has not been able to keep pace."

'The Science Is Here': UN Chief Welcomes First Global AI Assessment

"The science is here," UN Secretary-General António Guterres said at the report launch. "We can no longer say we did not know. What we do with it is now up to all of us." The more AI advances without shared rules, the less say governments and people will have in the outcome, the UN chief said, adding "my message to governments is simple: do not wait." Aiming to build a shared understanding and evidence at this critical juncture, the Preliminary Report of the Independent International Scientific Panel on AI: Evidence-based assessment of opportunities, risks and impacts of AI was penned by the first global, fully independent scientific body dedicated to assessing its real impacts across economies and societies. Why it matters Globally, over one billion people now use conversational AI weekly, while governments are making consequential decisions in the face of great uncertainty with rapidly changing, often conflicting sources of evidence and perspectives that do not necessarily reflect local realities. "Used well, AI could be the most powerful engine for development, speeding the world's progress on everything from health and hunger to learning and climate," the UN chief said, "but the panel is just as clear-eyed about the harm artificial intelligence can cause." Indeed, as the capabilities of AI continue to grow, so do the stakes - the core challenge the panel aims to address. Better world or catastrophic harm? Composed of 40 leading scientists and experts from every region, the panel outlines AI trends and warns that current safeguards cannot keep pace, said its co-chair Yoshua Bengio. "AI capabilities are outpacing both scientific understanding and governments' ability to adapt," Mr. Bengio said. "With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users." To act effectively, he said, global policymakers must understand these systems, and the panel provides exactly that: a rigorous, shared scientific foundation "to guide our collective way forward". Key findings Detecting breast cancer earlier, accelerating vaccine development and improving healthcare services are just a few ground-breaking AI accomplishments, but limitations and challenges remain, among them: * AI adoption has accelerated broadly, but unevenly across countries and sectors * Access and usage vary widely, with adoption across the global south lagging far behind the global north * Significant differences in compute infrastructure and models exist between advanced economies, reflecting existing inequalities Moreover, development is highly concentrated, with recent estimates finding that the United States accounts for 75 per cent of the computing power among the world's top 500 AI supercomputers, with China accounting for 15 per cent, and that both countries' companies develop almost all leading general-purpose models. Understanding risks Understanding and managing AI risks is essential, the report stated, with panel co-chair Maria Ressa adding that risks to societies, security and the human species are already "too high". "The technology is transformative, but if the world keeps moving along this trajectory, humanity will fail to realise the gains it promises," she warned. Here are some of the panel's warnings: * There are no scientific guarantees that AI agent systems will not violate instructions, and evidence is accumulating of cases where they already are * AI agent systems will soon complete tasks that currently take human programmers days or weeks, but their deployment raises urgent questions for labour markets, cybersecurity and the controllability of future AI systems * Sycophantic AI behaviour, where responses reinforce users' existing beliefs regardless of accuracy, has been linked to several severe mental health incidents, including documented deaths * Criminals and bad actors have been documented using AI systems to assist in cyberattacks * Advanced technical abilities may allow novice private actors to use AI in malicious ways across a range of applications such as fraud and disinformation * Reliable methods for retaining control over highly autonomous AI systems are lacking Many of these harms fall disproportionately on already disadvantaged populations, the report stated. Unlocking benefits, mitigating risks Minimising AI risks and benefiting from this technological tool requires good governance. Concrete next steps to close current gaps exist, but each requires sustained investment in Member State capacity to shape, evaluate and deploy AI. Realising these opportunities safely requires dedicated investments and policies to incentivise equitable access and reward innovation while preventing the exploitation of the vulnerable. 'AI will not close divides itself' Dozens of distinct governance instruments that seek to embed ethics and human rights in AI systems are already in use across jurisdictions, but they are fragmented, concentrated among a few corporations and rarely measure real-world effectiveness, the report found. Amandeep Gill, Under-Secretary-General and Special Envoy for Digital and Emerging Technologies, said the new report delivers shared scientific language that decision makers can now use going forward. "AI will not close divides by itself," he said. The benefits land where institutions, skills and data already exist, and where they do not, the same technology can displace workers, widen inequality and leave communities dependent on systems built without them in mind, he explained. "Those realities are now on the record, independently verified, and impossible to set aside." The report's findings will be presented to governments at the inaugural UN Global Dialogue on AI Governance, convening in Geneva on 6 and 7 July.

Unchecked AI progress may pose catastrophic risks, UN panel warns

GENEVA, July 1 (Reuters) - Developments in artificial intelligence are outpacing scientific understanding and government policy, meaning there are no guarantees the technology will not cause catastrophic harm, a United Nations independent panel warned on Wednesday. A preliminary report by the UN's Independent International Scientific Panel on Artificial Intelligence said policymakers face a growing dilemma: they need robust evidence to regulate AI effectively, yet such evidence is struggling to keep pace with the technology's rapid evolution. "AI capabilities are outpacing both scientific understanding and governments' ability to adapt," said Yoshua Bengio, co-chair of the panel, comprised of 40 cross-regional experts. "With growing evidence of deceptive AI behaviour, science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm, either on its own or due to malicious users." Described as the first global independent assessment of AI's risks and opportunities, the report aims to give up-to-date evaluations of the science to help guide decision-making as governments contend with fast-evolving systems. In the near term, it expects a shift towards agentic AI systems capable of carrying out real-world tasks, although growth may be constrained by energy and high-quality data shortages. Over time, it foresees self-improving AI embedded more deeply in the economy and converging with technologies such as quantum computing and biotechnology. AGENTIC AI DEVELOPING RAPIDLY AI already demonstrates expert-level reasoning in mathematics and science and is accelerating drug and vaccine development, and its task complexity is doubling every four to seven months, potentially allowing systems to complete work that takes humans days or weeks, according to the report. While this could deliver significant economic benefits, it remains unclear whether productivity gains from using AI will translate into broader growth or affect jobs. The panel also outlined a range of safety concerns, such as the risk of losing control over AI systems as they become increasingly autonomous, and deceptive. AI is already being used to generate misinformation and other harmful content and could be exploited for fraud, cyberattacks and biological threats. Governance remains fragmented, with many countries lacking the capacity to assess or shape advanced AI systems, leaving them reliant on technologies they cannot fully understand or control. Existing safety tools often depend on limited testing data disclosed by companies, the report said. UN Secretary-General António Guterres urged governments to act swiftly. "The world cannot govern what it cannot understand," Guterres said in a statement. "The potential is great, but the risks are real, and the cost of waiting is rising," he added. (Reporting by Olivia Le Poidevin, Editing by Louise Heavens)

UN report sees enormous potential benefits and big risks from AI

July 1 (Reuters) - The rapid development of AI offers huge potential benefits to countries and people around the world, but also poses big risks, 40 leading scientists and experts said in the first report by a U.N. independent scientific panel on the technology. The report, to be presented to governments at an inaugural U.N. Global Dialogue on AI governance in Geneva July 6 to 7, offers the first global, independent scientific assessment of AI, with a fuller, comprehensive report planned next year. Members of the panel were drawn from every region of the world, and its members serve a three-year term, independent of any government, institution or company. Following are a few details from the preliminary report: o Policymakers need scientific evidence to govern AI but its capabilities are outpacing scientific understanding and governments' ability to adapt, with few methods available for controlling highly autonomous AI systems. o Panel co-chair Yoshua Bengio noted growing evidence of deceptive AI behavior and said science could not guarantee AI will not cause catastrophic harm "either on its own or due to malicious users" as capabilities increase. o "The potential benefits of AI are enormous," the report concluded. "The rapid, unchecked deployment of the technology at scale also presents considerable risks, including harms to the mental health of users, potential use as a destructive tool, impacts on social, economic and environmental systems, and challenges associated with controlling the technology." o AI adoption has accelerated broadly, but unevenly, across countries and sectors. Globally, over a billion people now use conversational AI weekly, but adoption in developing countries lags. o AI development is even more concentrated, with the U.S. accounting for 75% of the computing power among the world's top 500 AI supercomputers, and China 15%. o Although more than 7,000 languages are spoken worldwide, current AI models are trained for only a small fraction and machine translation of some languages is riddled with errors that can affect health diagnoses and treatment decisions. o Risks include potential negative impacts on human rights, social systems and the environment, with AI-generated child sexual abuse material and deepfake-enabled sexual violence circulating more frequently. o AI also makes it easier to produce and target persuasive content at scale, contributing to a "gradual erosion of information integrity that can weaken public trust, social cohesion and democratic deliberation." o Most countries, including advanced economies, lack the technical expertise to assess the most capable new AI models or participate meaningfully in their governance. (Reporting by Andrea Shalal in Washington; Editing by Chris Reese)