Anthropic CEO Sets Ambitious Goal to Decode AI Models by 2027

Anthropic CEO wants to open the black box of AI models by 2027 | TechCrunch

Anthropic CEO Dario Amodei published an essay Thursday highlighting how little researchers understand about the inner workings of the world's leading AI models. To address that, he's set an ambitious goal for Anthropic to reliably detect most model problems by 2027. Amodei acknowledges the challenge ahead. In "The Urgency of Interpretability," the CEO says Anthropic has made early breakthroughs in tracing how models arrive at their answers -- but emphasizes that far more research is needed to decode these systems as they grow more powerful. "I am very concerned about deploying such systems without a better handle on interpretability," Amodei wrote in the essay. "These systems will be absolutely central to the economy, technology, and national security, and will be capable of so much autonomy that I consider it basically unacceptable for humanity to be totally ignorant of how they work." Anthropic is one of the pioneering companies in mechanistic interpretability, a field that aims to open the black box of AI models and understand why they make the decisions they do. Despite the rapid performance improvements of the tech industry's AI models, we still have relatively little idea how these systems arrive at decisions. For example, OpenAI recently launched new reasoning AI models, o3 and o4-mini, that perform better on some tasks, but also hallucinate more than its other models. The company doesn't know why it's happening. "When a generative AI system does something, like summarize a financial document, we have no idea, at a specific or precise level, why it makes the choices it does -- why it chooses certain words over others, or why it occasionally makes a mistake despite usually being accurate," Amodei wrote in the essay. Anthropic co-founder Chris Olah says that AI models are "grown more than they are built," Amodei notes in the essay. In other words, AI researchers have found ways to improve AI model intelligence, but they don't quite know why. In the essay, Amodei says it could be dangerous to reach AGI -- or as he calls it, "a country of geniuses in a data center" -- without understanding how these models work. In a previous essay, Amodei claimed the tech industry could reach such a milestone by 2026 or 2027, but believes we're much further out from fully understanding these AI models. In the long term, Amodei says Anthropic would like to, essentially, conduct "brain scans" or "MRIs" of state-of-the-art AI models. These checkups would help identify a wide range of issues in AI models, including their tendencies to lie, seek power, or other weakness, he says. This could take five to ten years to achieve, but these measures will be necessary to test and deploy Anthropic's future AI models, he added. Anthropic has made a few research breakthroughs that have allowed it to better understand how its AI models work. For example, the company recently found ways to trace an AI model's thinking pathways through, what the company call, circuits. Anthropic identified one circuit that helps AI models understand which U.S. cities are located in which U.S. states. The company has only found a few of these circuits, but estimates there are millions within AI models. Anthropic has been investing in interpretability research itself, and recently made its first investment in a startup working on interpretability. In the essay, Amodei called on OpenAI and Google DeepMind to increase their research efforts in the field. Amodei even calls on governments to impose "light-touch" regulations to encourage interpretability research, such as requirements for companies to disclose their safety and security practices. In the essay, Amodei also says the U.S. should put export controls on chips to China, in order to limit the likelihood of an out-of-control, global AI race. Anthropic has always stood out from OpenAI and Google for its focus on safety. While other tech companies pushed back on California's controversial AI safety bill, SB 1047, Anthropic issued modest support and recommendations for the bill, which would have set safety reporting standards for frontier AI model developers. In this case, Anthropic seems to be pushing for an industry-wide effort to better understand AI models, not just increasing their capabilities.

Anthropic wants to decode AI by 2027

Anthropic CEO Dario Amodei published an essay on Thursday highlighting the limited understanding of the inner workings of leading AI models and set a goal for Anthropic to reliably detect most AI model problems by 2027. Amodei acknowledges the challenge ahead, stating that while Anthropic has made early breakthroughs in tracing how models arrive at their answers, more research is needed to decode these systems as they grow more powerful. "I am very concerned about deploying such systems without a better handle on interpretability," Amodei wrote, emphasizing their central role in the economy, technology, and national security. Anthropic is a pioneer in mechanistic interpretability, aiming to understand why AI models make certain decisions. Despite rapid performance improvements, the industry still has limited insight into how these systems arrive at decisions. For instance, OpenAI's new reasoning AI models, o3 and o4-mini, perform better on some tasks but hallucinate more than other models, with the company unsure why. Amodei notes that AI researchers have improved model intelligence but don't fully understand why these improvements work. Anthropic co-founder Chris Olah says AI models are "grown more than they are built." Amodei warns that reaching AGI without understanding how models work could be dangerous and believes we're further from fully understanding AI models than achieving AGI, potentially by 2026 or 2027. Anthropic aims to conduct "brain scans" or "MRIs" of state-of-the-art AI models to identify issues, including tendencies to lie or seek power. This could take five to 10 years but will be necessary for testing and deploying future models. The company has made breakthroughs in tracing AI model thinking pathways through "circuits" and identified one circuit that helps models understand U.S. city locations within states. Anthropic has invested in interpretability research and recently made its first investment in a startup working on the field. Amodei believes explaining how AI models arrive at answers could present a commercial advantage. He called on OpenAI and Google DeepMind to increase their research efforts and asked governments to impose "light-touch" regulations to encourage interpretability research. Amodei also suggested the U.S. should impose export controls on chips to China to limit the likelihood of an out-of-control global AI race. Anthropic has focused on safety, issuing modest support for California's AI safety bill, SB 1047, which would have set safety reporting standards for frontier AI model developers. Anthropic is pushing for an industry-wide effort to better understand AI models, not just increase their capabilities. The company's efforts and recommendations highlight the need for a collaborative approach to AI safety and interpretability.