Meta's AI Shows Signs of Self-Improvement, Sparking Superintelligence Debate and Policy Shift

Meta AI takes first step to superintelligence -- and Zuckerberg will no longer release the most powerful systems to the public

The Meta CEO believes that AI with the capacity to improve itself is the first step towards a technology that will transform humanity. Researchers at Meta claim to have observed the company's artificial intelligence (AI) systems improving themselves without human input. "Over the last few months we have begun to see glimpses of our AI systems improving themselves," Meta CEO Mark Zuckerberg said in a policy paper published July 30 on the company's website. "The improvement is slow for now, but undeniable." Zuckerberg added that self-improving AI is the first step towards achieving artificial superintelligence (ASI) -- an artificial intelligence that is capable of surpassing the cognitive abilities of humans. Broadly speaking, AI researchers consider AI's capabilities in three strata. While current AI models can achieve extraordinary superhuman feats, like predicting protein structures, these capabilities are extremely narrow. In such cases, the system can only outpace humans in one specific area but lacks general intelligence for this capability to apply across different disciplines. The next tier is called artificial general intelligence (AGI), and refers to models that can understand, learn and adapt in much the same way as the human brain. ASI is the final state that Zuckerberg referenced in the blog post, and concerns models that have evolved to far exceed human potential -- and could improve themselves at an exponential rate, leading to an "intelligence explosion." Scientists refer to the hypothetical moment that we achieve AGI as the technological singularity. Related: AI is entering an 'unprecedented regime.' Should we stop it -- and can we -- before it destroys us? Zuckerberg's reference to catching sight of AI self-improvement is not the first time this phenomenon has been spotted. In October 2024, researchers at the University of California, Santa Barbara published a paper to the preprint arXiv database centered on the notion of self-improvement. In it, they defined, created and tested an AI framework based on the idea of a Gödel Machine -- a theoretical device capable of improving itself by rewriting its own code and instructions. Importantly, the machine would only implement a change if it could produce a formal proof that the change would be beneficial. Related: AI could soon think in ways we don't even understand -- evading our efforts to keep it aligned -- top AI scientists warn The team experimented with an AI framework based on that concept, and proved that their Gödel Agent could improve its own performance in various tasks, including coding, science, math and reasoning. Almost all models are not typically empowered to rewrite their own code. The Gödel Agent could not only access its entire codebase but also the code used to develop improvements, while showing that those improvements were beneficial. The AI consistently showed better performance in key areas than the human-designed agents to which it was compared, according to the study.Zuckerberg said that ASI could represent a transformative step in humanity's technological progress, and that it could lead to the "discovery of new things that aren't imaginable today." That said, he added that Meta would be far more careful about which models the company chooses to release to the public under an open source framework -- meaning anybody could have access to it. "I am extremely optimistic that superintelligence will help humanity accelerate our pace of progress. But perhaps even more important is that superintelligence has the potential to begin a new era of personal empowerment where people will have greater agency to improve the world in the directions they choose," Zuckerberg wrote. "As profound as the abundance produced by AI may one day be, an even more meaningful impact on our lives will likely come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, experience any adventure, be a better friend to those you care about, and grow to become the person you aspire to be."

Zuckerberg reveals Meta's AI superintelligence breakthrough -- and why you won't be using it anytime soon

Meta is shifting gears in the AI race, claiming its systems are beginning to improve themselves; a potential early step toward artificial superintelligence (ASI). But in the same breath, CEO Mark Zuckerberg says the company will no longer release its most advanced AI models to the public, citing safety concerns. In a newly published policy paper, Zuckerberg revealed that Meta's AI has started refining its own abilities without human input. While the pace is "slow for now, but undeniable," he framed the breakthrough as a foundational moment on the path to ASI -- AI systems that not only outperform humans in nearly every domain but can also evolve on their own. Researchers often describe ASI as the next rung above artificial general intelligence (AGI), which matches human adaptability. AGI is considered the key milestone before an "intelligence explosion," where AI could rapidly improve beyond human control. For years, Meta has touted its open-source approach to AI, making large language models like Llama freely available to researchers and developers. Now, that policy is changing. Zuckerberg says the company will continue releasing competitive models, but the most advanced systems will stay internal to prevent potential misuse. ASI, he warned, introduces "novel safety concerns" that demand tighter controls, even at the expense of openness. For those unfamiliar with what open source means, it's software that's built on the principle that its source code (the instructions that make it work) is freely available for anyone to view, use and modify. A good example of a completely open source chatbot is DeepSeek. The openness allows developers worldwide to collaborate on improvements, fix security flaws, and adapt the software for specific needs. It also promotes transparency, since anyone can inspect the code to understand how it works and ensure it's trustworthy. However, it also comes with safety concerns like the ones Zuckerberg is referencing including guardrails can be removed, it's harder to know who is using the software for harm because there is little to no gatekeeping. Meta's superintelligence ambitions are now housed under a new division: Meta Superintelligence Labs. Launched in June 2025, the group is based in Menlo Park, California, and reportedly oversees development of the ultra-secret "Behemoth" model. Tech figures Alexandr Wang and Nat Friedman are said to be leading parts of the initiative. Meta's decision puts it at odds with rivals like OpenAI, which still provides limited access to its flagship models through public platforms. The move raises bigger questions for the AI industry: Meta is planting its flag in the superintelligence conversation and it's willing to change its long-standing approach to do it. It's hard to know if this is a necessary safeguard or a bid for exclusive AI dominance, but either way, it will likely define the next chapter of the company's role in the AI race.