The Evolution of AI: From ChatGPT to Reasoning Models and Beyond

We Interrupt Your Chatbot Conversation to Bring You This Ad

Member of the board, UCLA Daily Bruin Alumni Network; advisory board, Center for Ethical Leadership in the Media As part of its goal to pivot away from its nonprofit origins and become a for-profit business within two years, OpenAI last week announced a $200 a month "pro tier" that will give subscribers unlimited access to a faster version of its most powerful ChatGPT engine -- called o1 -- as well as the voice mode that lets you talk to the advanced chatbot (which is capable of humanlike reasoning). The news about ChatGPT Pro, the first of 12 new product announcements to be delivered each work day starting Dec. 5, was shared via a 15-minute livestream called 12 Days of OpenAI, hosted by CEO Sam Altman. And it came the day after Altman told The New York Times at the newspaper's DealBook summit that ChatGPT has seen explosive growth since it was launched two years ago and is now hosting 300 million weekly active users -- that's three times as many as it was hosting as of November 2023. Video of the 35-minute interview is here and it's worth a watch, given Altman's belief that we'll see a general artificial intelligence (think Jarvis from the Marvel movies) "sooner than most people think" -- in a few years -- and given his optimism that someone else will figure out how to make sure it doesn't harm humanity. Altman also said he isn't concerned that his OpenAI co-founder Elon Musk, who's now suing the company and has started his own AI company, called xAI, might sway the incoming Trump administration in a way that will harm Musk's competitors (Musk, in a revised lawsuit filed in November, is asking the courts to stop OpenAI's transition to a for-profit company). "I believe pretty strongly that Elon will do the right thing and that it would be profoundly unAmerican to use political power to the degree that Elon has it to hurt competitors and advantage his own businesses," Altman said. "I'm not that worried about it." But back to the quest to turn into a for-profit company: Days before all this news hit, OpenAI CFO Sarah Friar told The Financial Times that the company is considering adding advertisements to its chatbot as it looks for new ways to make money. Though it doesn't have "active plans" right now to serve up ads as you prompt ChatGPT, Friar told the FT that OpenAI planned to be "thoughtful about when and where we implement [ads]." She also pointed to the ad expertise of OpenAI's top executives, including Chief Product Officer Kevin Weil, who built ad-supported products at Instagram and X. "The good news with Kevin Weil at the wheel with product is that he came from Instagram. He knows how this [introducing ads] works," Friar said in the FT interview. What does this mean for you? I'd say enjoy your ad-free chatbot experience while it lasts -- not just with ChatGPT but with other currently free AI services as well. As the FT adds, Google and Meta led the way in capitalizing on their massive user bases -- that's you and me -- with advertising, and OpenAI needs the money. The success of ChatGPT led revenue to surge to about $4 billion on an annualized basis, the FT notes, but it adds that the high cost of running data- and power-intensive AI systems means OpenAI will spend more than it's taking in and may "burn through more than $5 billion of cash." Here are the other doings in AI worth your attention. While death and taxes are among life's certainties, something that remains less definite is what the weather will be on any given day -- despite the best forecasts of meteorologists. That's why the news that Google DeepMind's latest AI model may be the "best yet" at predicting the weather, according to the MIT Technology Review, gets a shout-out in this week's nod to AI for the public good. Called GenCast, DeepMind's new model "was trained on 40 years of weather data (1979 to 2018) and then generated a forecast for 2019," the MIT Review noted. "In its predictions, it was more accurate than the current best forecast, the Ensemble Forecast, ENS, 97% of the time, and it was better at predicting wind conditions and extreme weather like the path of tropical cyclones." You can learn more about GenCast in an overview published in Nature that explains why knowing whether it'll rain or snow matters. "Every day," the overview says, "people, governments and other organizations around the world rely on accurate weather forecasts to make many key decisions -- whether to carry an umbrella, when to flee an approaching tropical cyclone, how to plan the use of renewable energy in a power grid, or how to prepare for a heatwave." In other Google news, the company published a blog item recapping the seven AI-related news announcements it made in November. And though it was interesting to read how people are using Google Lens to find products and comparison shop for holiday presents, the bit that caught my attention is how the company has expanded its AI-flood forecasting model to cover "100 countries in areas where 700 million people live." Another AI use case for the public good. Popular music-streaming service Spotify closes out each year with a wrap-up of the year's most popular songs, artists and podcasts, based on the content consumed by its more than 250 million subscribers from around the world. It's called "Spotify Wrapped," and we learned that Sabrina Carpenter's Espresso was the top song (on Apple Music, it was Not Like Us by Kendrick Lamar, CNET's Ty Pendlebury notes). Spotify also sends individual users rundowns of their listening habits for the year. And this year's data-lite Spotify roundups included a new feature that uses Google's NotebookLM AI technology to create a "Wrapped AI Podcast," in which two AI bots discuss your listening habits based on your song preferences -- and "flatter" your taste in music, Pendlebury adds. If that sounds a little weird and cringe-inducing, it's because it is, and I'm not the only one who thinks so. Vox calls it a "bizarre addition" that feels "both like listening to a doctor go through your bloodwork results and a psychic vaguely supposing facts about your life." Forbes called the AI podcast a "bad idea" and said, "Listening to a couple of AI bots mindlessly spitting out empty observations while mimicking the tempo of human speech is a soul-destroying experience." All that prompted me to ask Google for a list of the most popular songs about screwing up. It sent me to this list of "20 Songs About Messing Up." You're welcome. If you want to hear your personalized podcast and decide for yourself, go to the top menu of Spotify's mobile app and select Wrapped > AI Podcast. "It's worth noting that it's short -- there's no music -- and it's simply the two 'hosts' reading your stats," Pendlebury adds. Enjoy? President-elect Donald Trump picked tech investor, former PayPal executive and friend of Elon Musk David Sacks as his "White House AI and Crypto Czar," putting him in charge of US efforts to promote the technologies. Sacks, who made some of his fortune after selling Yammer to Microsoft for $1.2 billion in 2012, is a former Trump critic, saying the ex-president "had disqualified himself from being a candidate at a national level" after the Capitol riots on Jan. 6, CNBC notes. But with expectations that AI and crypto will be big moneymakers for tech investors, Sacks became a "major Trump booster" earlier this year, CNBC said. Though chatbots like ChatGPT are "now acing nearly every math test they encounter," AI still isn't smarter at math than humans, at least not yet, Science reports. That's the take after a "tech research institute called Epoch AI rounded up 60 expert mathematicians to raise the bar with the most challenging math test they could muster," Science said. "Leading models correctly answered fewer than 2% of the questions, showing just how far they are from disrupting the field" for mathematicians. If you're looking for insight into how AI scientists are thinking about AI, the University of Pennsylvania Press is offering a collection of essays from noted experts as a free PDF that's worth the download. Called Realizing the Promise and Minimizing the Perils of AI for Science and the Scientific Community, it includes thoughts from internet pioneer Vint Cerf, who offers this caution: "These systems produce the illusion of ­human discourse and are often extremely convincing, even when completely wrong. We are learning to use them in myriad ways but should be wary of being misled by the glib responses to our prompts."

The creator of ChatGPT's voice wants to build the tech from "Her," minus the dystopia

Alexis Conneau thinks a lot about the movie "Her." For the last several years, he's obsessed over trying to turn the film's fictional voice technology, Samantha, into a reality. Conneau even uses a picture of Joaquin Pheonix's character in the movie as his banner on Twitter. With ChatGPT's Advanced Voice Mode, a project Conneau started at OpenAI after doing similar work at Meta, he kind of did it. The AI system natively processes speech, and talks back much like a human. Now, he has a new startup, WaveForms AI, that's trying to build something better. Conneau spends a good chunk of time thinking about how to avoid the dystopia shown in that movie, he told TechCrunch in an interview. "Her" was a science fiction film about a world where people develop intimate relationships with AI systems, instead of other humans. "The movie is a dystopia, right? It's not a future we want," said Conneau. "We want to bring that technology - which now exists and will exist - and we want to bring it for good. We want to do precisely the opposite of what the company in that movie does." Building the tech, minus the dystopia that comes with it, seems like a contradiction. But Conneau intends to build it anyways, and he's convinced his new AI startup will help people "feel the AGI" with their ears. On Monday, Conneau launched WaveForms AI, a new audio LLM company training its own foundation models. It's aiming to release AI audio products in 2025 that compete with offerings from OpenAI and Google. The startup raised $40 million in seed funding, it announced on Monday, led by Andreessen Horowitz. Conneau says Marc Andreessen - who previously wrote that AI should be part of every aspect of human life - has taken a personal interest in his endeavor. It's worth noting that Conneau's obsession with the movie "Her" may have landed OpenAI in trouble at one point. Scarlett Johansson sent a legal threat to Sam Altman's startup earlier this year, ultimately forcing OpenAI to take down one of ChatGPT's voices that strongly resembled her character in the film. OpenAI denied ever trying to replicate her voice. But it's undeniable how much the movie has influenced Conneau. "Her" was clearly science fiction when it was released in 2013 -- at the time, Apple's Siri was quite new and very limited. But today, the technology feels scarily within reach. AI companionship platforms like Character.AI reach millions of users weekly who just want to talk with its chatbots. The sector is emerging as a popular use case for generative AI -- despite occasionally tragic and unsettling outcomes. You can imagine how someone typing with a chatbot all day would love the chance to speak with it too, especially using tech as convincing as ChatGPT's Advanced Voice Mode. The CEO of WaveForms AI is wary of the AI companionship space, and it's not the core of his new company. While he thinks people will use WaveForms' products in new ways - such as talking to an AI for 20 minutes in the car to learn about something - Conneau says he wants the company to be more "horizontal." "[WaveForms AI] can be that teacher that inspires, you know, maybe that teacher that you wouldn't have in your life, at least, your physical life," said the CEO. In the future, he believes talking to generative AI will be a more common way to interact with all kinds of technology. That may include talking to your car, talking to your computer, and WaveForms aims to supply the "emotionally intelligent" AI that facilitates it all. "I don't believe in the future where human-to-AI interaction replaces human-to-human interaction," said Conneau. "If anything, it's going to be complementary." He says AI can learn from the mistakes of social media. For instance, he thinks AI shouldn't optimize for "time spent on platform," a common metric of success for social apps that can promote unhealthy habits, like doomscrolling. More broadly, he wants to make sure WaveForms' AI is aligned with the best interests of humans, calling this "the most important work you could do." Conneau says OpenAI's name for his project, "Advanced Voice Mode," doesn't really do justice to how different the technology is from ChatGPT's regular voice mode. The old voice mode was really just translating your voice into text, running it through GPT-4, and then converting that text back into speech. It was a somewhat hacked together solution. However, with Advanced Voice Mode, Conneau says that GPT-4o is actually breaking down the audio of your voice into tokens (apparently, every second of audio is equal to roughly three tokens) and running those tokens directly through an audio-specific transformer model. That, he explained, is what enables Advanced Voice Mode to have such low latency. One claim that gets thrown around a lot when talking about AI audio models is that they can supposedly "understand emotions." Much like text-based LLMs are based on patterns found in heaps of text documents, audio LLMs do the same thing with audio clips of humans talking. Humans label these clips as "sad" or "excited" so that AI models recognize similar voice patterns when they hear you say it, and even respond back with emotional intonations of their own. So it's less that they "understand emotions" and more that they systematically recognize audio qualities that humans associate with those emotions. Making AI more personable, not smarter Conneau is betting that generative AI today doesn't need to get significantly smarter than GPT-4o to create better products. Instead of improving the underlying intelligence of these models, like OpenAI is with o1, WaveForms is simply trying to make AI better to talk to. "There will be a market of people [using generative AI] who will just choose the interaction that is the most enjoyable for them," said Conneau. That's why the startup is confident it can develop its own foundational models -- ideally, smaller ones that will be less expensive and faster to run. That's not a bad bet given recent evidence that the old AI scaling laws are slowing down. Conneau says his former co-worker at OpenAI, Ilya Sutskever, often talked to him about trying to "feel the AGI" - essentially, using a gut feeling to assess whether we've reached superintelligent AI. The CEO of WaveForms is convinced that achieving AGI will be more of a feeling, instead of reaching some sort of benchmark, and audio LLMs will be the key to that feeling. "I think you'll be able to feel the AGI a lot more when you can talk to it, when you can hear the AGI, when you can actually talk to the transformer itself," said Conneau, repeating comments he made to Sutskever over dinner. But as startups make AI better to talk to, they clearly also have a responsibility to figure out how to make sure people don't get addicted. Although, Andreessen Horowitz general partner Martin Casado, who helped lead the investment in WaveForms, says it's not necessarily a bad thing if people are talking to AI more often. "I can go talk to a random person on the internet, and that person can bully me, that person can take advantage of me... I can talk to a video game which could be arbitrarily violent, or I could talk to an AI," said Casado in an interview with TechCrunch. "I think it's an important question study. I will not be surprised if it turns out that [talking to AI] is actually preferable." Some companies may consider someone developing a loving relationship with your AI as a marker of success. But from a societal standpoint, it also could be seen as a marker of total failure, much like the movie "Her" tried to depict. That's the tightrope that WaveForms now has to walk.

ChatGPT's second birthday: What will gen AI (and the world) look like in another 2 years?

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More It is now just over two years since the first appearance of ChatGPT on November 30, 2022. At the time of its launch, OpenAI viewed ChatGPT as a demonstration project designed to learn how people would make use of the tool and the underlying GPT 3.5 large language model (LLM). A LLM is a model based on the transformer architecture first introduced by Google in 2017, which uses self-attention mechanisms to process and generate human-like text across tasks like natural language understanding. It was more than a successful demonstration project! OpenAI was as surprised as anyone by the rapid uptake of ChatGPT, which reached one hundred million users within two months. Although perhaps they should not have been so surprised. Futurist Kevin Kelly, also the co-founder of Wired, advised in 2014 that "the business plans of the next 10,000 startups are easy to forecast: Take X and add AI. This is a big deal, and now it's here." Kelly said this several years before ChatGPT. Yet, this is exactly what has happened. Equally remarkable is his prediction in the same Wired article that: "By 2024, Google's main product will not be search but AI." It could be debated if this is true, but it might soon be. Gemini is Google's flagship AI chat product, but AI pervades its search and likely every other one of its products, including YouTube, TensorFlow and AI features in Google Workspace. The bot heard around the world The headlong rush of AI startups that Kelly foresaw really gained momentum after the ChatGPT launch. You could call it the AI big bang moment, or the bot heard around the world. And it jumpstarted the field of generative AI -- the broad category of LLMs for text and diffusion models for image creation. This reached the heights of hype, or what Gartner calls "The Peak of Inflated Expectations" in 2023. The hype of 2023 may have diminished, but only by a little. By some estimates, there are as many as 70,000 AI companies worldwide, representing a 100% increase since 2017. This is a veritable Cambrian explosion of companies pursuing novel uses for AI technology. Kelly's 2014 foresight about AI startups proved prophetic. If anything, huge venture capital investments continue to flow into startup companies looking to harness AI. The New York Times reported that investors poured $27.1 billion into AI start-ups in the U.S. in the second quarter of 2024 alone, "accounting for nearly half of all U.S. start-up funding in that period." Statista added: "In the first nine months of 2024, AI-related investments accounted for 33% of total investments in VC-backed companies headquartered in the U.S. That is up from 14% in 2020 and could go even higher in the years ahead." The large potential market is a lure for both the startups and established companies. Hype does not equal use, at least not immediately A recent Reuters Institute survey of consumers indicated individual usage of ChatGPT was low across six countries, including the U.S. and U.K. Just 1% used it daily in Japan, rising to 2% in France and the UK, and 7% in the U.S. This slow uptake might be attributed to several factors, ranging from a lack of awareness to concerns about the safety of personal information. Does this mean AI's impact is overestimated? Hardly, as most of the survey respondents expected gen AI to have a significant impact on every sector of society in the next five years. The enterprise sector tells quite a different story. As reported by VentureBeat, industry analyst firm GAI Insights estimates that 33% of enterprises will have gen AI applications in production next year. Enterprises often have clearer use cases, such as improving customer service, automating workflows and augmenting decision-making, which drive faster adoption than among individual consumers. For example, the healthcare industry is using AI for capturing notes and financial services is using the technology for enhanced fraud detection. GAI further reported that gen AI is the leading 2025 budget priority for CIOs and CTOs. What's next? From gen AI to the dawn of superintelligence The uneven rollout of gen AI raises questions about what lies ahead for adoption in 2025 and beyond. Both Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman suggest that artificial general intelligence (AGI) -- or even superintelligence -- could appear within the next two to 10 years, potentially reshaping our world. AGI is thought to be the ability for AI to understand, learn and perform any intellectual task that a human being can, thereby emulating human cognitive abilities across a wide range of domains. Sparks of AGI in 2025 As reported by Variety, Altman said that we could see the first glimmers of AGI as soon as 2025. Likely he was talking about AI agents, in which you can give an AI system a complicated task and it will autonomously use different tools to complete it. For example, Anthropic recently introduced a Computer Use feature that enables developers to direct the Claude chatbot "to use computers the way people do -- by looking at a screen, moving a cursor, clicking buttons and typing text." This feature allows developers to delegate tasks to Claude, such as scheduling meetings, responding to emails or analyzing data, with the bot interacting with computer interfaces as if it were a human user. In a demonstration, Anthropic showcased how Claude could autonomously plan a day trip by interacting with computer interfaces -- an early glimpse of how AI agents may oversee complex tasks. In September, Salesforce said it "is ushering in the third wave of the AI revolution, helping businesses deploy AI agents alongside human workers." They see agents focusing on repetitive, lower-value tasks, freeing people to focus on more strategic priorities. These agents could enable human workers to focus on innovation, complex problem-solving or customer relationship management. With features like Computer Use capabilities from Anthropic and AI agent integration by Salesforce and others, the emergence of AI agents is becoming one of the most anticipated innovations in the field. According to Gartner, 33% of enterprise software applications will include agentic AI by 2028, up from less than 1% in 2024, enabling 15% of day-to-day work decisions to be made autonomously. While enterprises stand to gain significantly from agentic AI, the concept of "ambient intelligence" suggests an even broader transformation, where interconnected technologies seamlessly enhance daily life. In 2016, I wrote in TechCrunch about ambient intelligence, as a "digital interconnectedness to produce information and services that enhance our lives. This is enabled by the dynamic combination of mobile computing platforms, cloud and big data, neural networks and deep learning using graphics processing units (GPUs) to produce artificial intelligence (AI)." At that time, I said that connecting these technologies and crossing the boundaries necessary to provide seamless, transparent and persistent experiences in context will take time to realize. It is fair to say that eight years later, this vision is on the cusp of being realized. The five levels of AGI Based on OpenAI's roadmap, the journey to AGI involves progression through increasingly capable systems, with AI agents (level 3 out of 5) marking a significant leap toward autonomy. Altman stated that the initial impact of these agents will be minimal. Although eventually AGI will "be more intense than people think." This suggests we should expect substantial changes soon that will require rapid societal adjustments to ensure fair and ethical integration. How will AGI advances reshape industries, economies, the workforce and our personal experience of AI in the years to come? We can surmise that the near-term future driven by further AI advances will be both exciting and tumultuous, leading to both breakthroughs and crises. Balancing breakthroughs and disruptions Breakthroughs could span AI-enabled drug discovery, precision agriculture and practical humanoid robots. While breakthroughs promise transformative benefits, the path forward is not without risks. The rapid adoption of AI could also lead to significant disruptions, notably job displacement. This displacement could be large, especially if the economy enters a recession, when companies look to shed payroll but remain efficient. If this were to occur, social pushbacks on AI including mass protests are possible. As the AI revolution progresses from generative tools to autonomous agents and beyond, humanity stands on the cusp of a new era. Will these advancements elevate human potential, or will they present challenges we are not yet prepared to face? Likely, there will be both. In time, AI will not just be part of our tools -- it will seamlessly integrate into the fabric of life itself, becoming ambient and reshaping how we work, connect and experience the world.

The GPT Era Is Already Ending

This week, OpenAI launched what its chief executive, Sam Altman, called "the smartest model in the world" -- a generative-AI program whose capabilities are supposedly far greater, and more closely approximate how humans think, than those of any such software preceding it. The start-up has been building toward this moment since September 12, a day that, in OpenAI's telling, set the world on a new path toward superintelligence. That was when the company previewed early versions of a series of AI models, known as o1, constructed with novel methods that the start-up believes will propel its programs to unseen heights. Mark Chen, then OpenAI's vice president of research, told me a few days later that o1 is fundamentally different from the standard ChatGPT because it can "reason," a hallmark of human intelligence. Shortly thereafter, Altman pronounced "the dawn of the Intelligence Age," in which AI helps humankind fix the climate and colonize space. As of yesterday afternoon, the start-up has released the first complete version of o1, with fully fledged reasoning powers, to the public. (The Atlantic recently entered into a corporate partnership with OpenAI.) On the surface, the start-up's latest rhetoric sounds just like hype the company has built its $157 billion valuation on. Nobody on the outside knows exactly how OpenAI makes its chatbot technology, and o1 is its most secretive release yet. The mystique draws interest and investment. "It's a magic trick," Emily M. Bender, a computational linguist at the University of Washington and prominent critic of the AI industry, recently told me. An average user of o1 might not notice much of a difference between it and the default models powering ChatGPT, such as GPT-4o, another supposedly major update released in May. Although OpenAI marketed that product by invoking its lofty mission -- "advancing AI technology and ensuring it is accessible and beneficial to everyone," as though chatbots were medicine or food -- GPT-4o hardly transformed the world. Read: The AI boom has an expiration date But with o1, something has shifted. Several independent researchers, while less ecstatic, told me that the program is a notable departure from older models, representing "a completely different ballgame" and "genuine improvement." Even if these models' capacities prove not much greater than their predecessors', the stakes for OpenAI are. The company has recently dealt with a wave of controversies and high-profile departures, and model improvement in the AI industry overall has slowed. Products from different companies have become indistinguishable -- ChatGPT has much in common with Anthropic's Claude, Google's Gemini, xAI's Grok -- and firms are under mounting pressure to justify the technology's tremendous costs. Every competitor is scrambling to figure out new ways to advance their products. Over the past several months, I've been trying to discern how OpenAI perceives the future of generative AI. Stretching back to this spring, when OpenAI was eager to promote its efforts around so-called multimodal AI, which works across text, images, and other types of media, I've had multiple conversations with OpenAI employees, conducted interviews with external computer and cognitive scientists, and pored over the start-up's research and announcements. The release of o1, in particular, has provided the clearest glimpse yet at what sort of synthetic "intelligence" the start-up and companies following its lead believe they are building. The company has been unusually direct that the o1 series is the future: Chen, who has since been promoted to senior vice president of research, told me that OpenAI is now focused on this "new paradigm," and Altman later wrote that the company is "prioritizing" o1 and its successors. The company believes, or wants its users and investors to believe, that it has found some fresh magic. The GPT era is giving way to the reasoning era. Last spring, I met Mark Chen in the renovated mayonnaise factory that now houses OpenAI's San Francisco headquarters. We had first spoken a few weeks earlier, over Zoom. At the time, he led a team tasked with tearing down "the big roadblocks" standing between OpenAI and artificial general intelligence -- a technology smart enough to match or exceed humanity's brainpower. I wanted to ask him about an idea that had been a driving force behind the entire generative-AI revolution up to that point: the power of prediction. The large language models powering ChatGPT and other such chatbots "learn" by ingesting unfathomable volumes of text, determining statistical relationships between words and phrases, and using those patterns to predict what word is most likely to come next in a sentence. These programs have improved as they've grown -- taking on more training data, more computer processors, more electricity -- and the most advanced, such as GPT-4o, are now able to draft work memos and write short stories, solve puzzles and summarize spreadsheets. Researchers have extended the premise beyond text: Today's AI models also predict the grid of adjacent colors that cohere into an image, or the series of frames that blur into a film. The claim is not just that prediction yields useful products. Chen claims that "prediction leads to understanding" -- that to complete a story or paint a portrait, an AI model actually has to discern something fundamental about plot and personality, facial expressions and color theory. Chen noted that a program he designed a few years ago to predict the next pixel in a grid was able to distinguish dogs, cats, planes, and other sorts of objects. Even earlier, a program that OpenAI trained to predict text in Amazon reviews was able to determine whether a review was positive or negative. Today's state-of-the-art models seem to have networks of code that consistently correspond to certain topics, ideas, or entities. In one now-famous example, Anthropic shared research showing that an advanced version of its large language model, Claude, had formed such a network related to the Golden Gate Bridge. That research further suggested that AI models can develop an internal representation of such concepts, and organize their internal "neurons" accordingly -- a step that seems to go beyond mere pattern recognition. Claude had a combination of "neurons" that would light up similarly in response to descriptions, mentions, and images of the San Francisco landmark. "This is why everyone's so bullish on prediction," Chen told me: In mapping the relationships between words and images, and then forecasting what should logically follow in a sequence of text or pixels, generative AI seems to have demonstrated the ability to understand content. The pinnacle of the prediction hypothesis might be Sora, a video-generating model that OpenAI announced in February and which conjures clips, more or less, by predicting and outputting a sequence of frames. Bill Peebles and Tim Brooks, Sora's lead researchers, told me that they hope Sora will create realistic videos by simulating environments and the people moving through them. (Brooks has since left to work on video-generating models at Google DeepMind.) For instance, producing a video of a soccer match might require not just rendering a ball bouncing off cleats, but developing models of physics, tactics, and players' thought processes. "As long as you can get every piece of information in the world into these models, that should be sufficient for them to build models of physics, for them to learn how to reason like humans," Peebles told me. Prediction would thus give rise to intelligence. More pragmatically, multimodality may also be simply about the pursuit of data -- expanding from all the text on the web to all the photos and videos, as well. Read: OpenAI's big reset Just because OpenAI's researchers say their programs understand the world doesn't mean they do. Generating a cat video doesn't mean an AI knows anything about cats -- it just means it can make a cat video. (And even that can be a struggle: In a demo earlier this year, Sora rendered a cat that had sprouted a third front leg.) Likewise, "predicting a text doesn't necessarily mean that [a model] is understanding the text," Melanie Mitchell, a computer scientist who studies AI and cognition at the Santa Fe Institute, told me. Another example: GPT-4 is far better at generating acronyms using the first letter of each word in a phrase than the second, suggesting that rather than understanding the rule behind generating acronyms, the model has simply seen far more examples of standard, first-letter acronyms to shallowly mimic that rule. When GPT-4 miscounts the number of r's in strawberry, or Sora generates a video of a glass of juice melting into a table, it's hard to believe that either program grasps the phenomena and ideas underlying their outputs. These shortcomings have led to sharp, even caustic criticism that AI cannot rival the human mind -- the models are merely "stochastic parrots," in Bender's famous words, or supercharged versions of "autocomplete," to quote the AI critic Gary Marcus. Altman responded by posting on social media, "I am a stochastic parrot, and so r u," implying that the human brain is ultimately a sophisticated word predictor, too. Altman's is a plainly asinine claim; a bunch of code running in a data center is not the same as a brain. Yet it's also ridiculous to write off generative AI -- a technology that is redefining education and art, at least, for better or worse -- as "mere" statistics. Regardless, the disagreement obscures the more important point. It doesn't matter to OpenAI or its investors whether AI advances to resemble the human mind, or perhaps even whether and how their models "understand" their outputs -- only that the products continue to advance. OpenAI's new reasoning models show a dramatic improvement over other programs at all sorts of coding, math, and science problems, earning praise from geneticists, physicists, economists, and other experts. But notably, o1 does not appear to have been designed to be better at word prediction. According to investigations from The Information, Bloomberg, TechCrunch, and Reuters, major AI companies including OpenAI, Google, and Anthropic are finding that the technical approach that has driven the entire AI revolution is hitting a limit. Word-predicting models such as GPT-4o are reportedly no longer becoming reliably more capable, even more "intelligent," with size. These firms may be running out of high-quality data to train their models on, and even with enough, the programs are so massive that making them bigger is no longer making them much smarter. o1 is the industry's first major attempt to clear this hurdle. When I spoke with Mark Chen after o1's September debut, he told me that GPT-based programs had a "core gap that we were trying to address." Whereas previous models were trained "to be very good at predicting what humans have written down in the past," o1 is different. "The way we train the 'thinking' is not through imitation learning," he said. A reasoning model is "not trained to predict human thoughts" but to produce, or at least simulate, "thoughts on its own." It follows that because humans are not word-predicting machines, then AI programs cannot remain so, either, if they hope to improve. More details about these models' inner workings, Chen said, are "a competitive research secret." But my interviews with independent researchers, a growing body of third-party tests, and hints in public statements from OpenAI and its employees have allowed me to get a sense of what's under the hood. The o1 series appears "categorically different" from the older GPT series, Delip Rao, an AI researcher at the University of Pennsylvania, told me. Discussions of o1 point to a growing body of research on AI reasoning, including a widely cited paper co-authored last year by OpenAI's former chief scientist, Ilya Sutskever. To train o1, OpenAI likely put a language model in the style of GPT-4 through a huge amount of trial and error, asking it to solve many, many problems and then providing feedback on its approaches, for instance. The process might be akin to a chess-playing AI playing a million games to learn optimal strategies, Subbarao Kambhampati, a computer scientist at Arizona State University, told me. Or perhaps a rat that, having run 10,000 mazes, develops a good strategy for choosing among forking paths and doubling back at dead ends. Read: Silicon Valley's trillion-dollar leap of faith Prediction-based bots, such as Claude and earlier versions of ChatGPT, generate words at a roughly constant rate, without pause -- they don't, in other words, evince much thinking. Although you can prompt such large language models to construct a different answer, those programs do not (and cannot) on their own look backward and evaluate what they've written for errors. But o1 works differently, exploring different routes until it finds the best one, Chen told me. Reasoning models can answer harder questions when given more "thinking" time, akin to taking more time to consider possible moves at a crucial moment in a chess game. o1 appears to be "searching through lots of potential, emulated 'reasoning' chains on the fly," Mike Knoop, a software engineer who co-founded a prominent contest designed to test AI models' reasoning abilities, told me. This is another way to scale: more time and resources, not just during training, but also when in use. Here is another way to think about the distinction between language models and reasoning models: OpenAI's attempted path to superintelligence is defined by parrots and rats. ChatGPT and other such products -- the stochastic parrots -- are designed to find patterns among massive amounts of data, to relate words, objects, and ideas. o1 is the maze-running rodent, designed to navigate those statistical models of the world to solve problems. Or, to use a chess analogy: You could play a game based on a bunch of moves that you've memorized, but that's different from genuinely understanding strategy and reacting to your opponent. Language models learn a grammar, perhaps even something about the world, while reasoning models aim to use that grammar. When I posed this dual framework, Chen called it "a good first approximation" and "at a high level, the best way to think about it." Reasoning may really be a way to break through the wall that the prediction models seem to have hit; much of the tech industry is certainly rushing to follow OpenAI's lead. Yet taking a big bet on this approach might be premature. For all the grandeur, o1 has some familiar limitations. As with primarily prediction-based models, it has an easier time with tasks for which more training examples exist, Tom McCoy, a computational linguist at Yale who has extensively tested the preview version of o1 released in September, told me. For instance, the program is better at decrypting codes when the answer is a grammatically complete sentence instead of a random jumble of words -- the former is likely better reflected in its training data. A statistical substrate remains. François Chollet, a former computer scientist at Google who studies general intelligence and is also a co-founder of the AI reasoning contest, put it a different way: "A model like o1 ... is able to self-query in order to refine how it uses what it knows. But it is still limited to reapplying what it knows." A wealth of independent analyses bear this out: In the AI reasoning contest, the o1 preview improved over the GPT-4o but still struggled overall to effectively solve a set of pattern-based problems designed to test abstract reasoning. Researchers at Apple recently found that adding irrelevant clauses to math problems makes o1 more likely to answer incorrectly. For example, when asking the o1 preview to calculate the price of bread and muffins, telling the bot that you plan to donate some of the baked goods -- even though that wouldn't affect their cost -- led the model astray. o1 might not deeply understand chess strategy so much as it memorizes and applies broad principles and tactics. Even if you accept the claim that o1 understands, instead of mimicking, the logic that underlies its responses, the program might actually be further from general intelligence than ChatGPT. o1's improvements are constrained to specific subjects where you can confirm whether a solution is true -- like checking a proof against mathematical laws or testing computer code for bugs. There's no objective rubric for beautiful poetry, persuasive rhetoric, or emotional empathy with which to train the model. That likely makes o1 more narrowly applicable than GPT-4o, the University of Pennsylvania's Rao said, which even OpenAI's blog post announcing the model hinted at, stating: "For many common cases GPT-4o will be more capable in the near term." Read: The lifeblood of the AI boom But OpenAI is taking a long view. The reasoning models "explore different hypotheses like a human would," Chen told me. By reasoning, o1 is proving better at understanding and answering questions about images, too, he said, and the full version of o1 now accepts multimodal inputs. The new reasoning models solve problems "much like a person would," OpenAI wrote in September. And if scaling up large language models really is hitting a wall, this kind of reasoning seems to be where many of OpenAI's rivals are turning next, too. Dario Amodei, the CEO of Anthropic, recently noted o1 as a possible way forward for AI. Google has recently released several experimental versions of Gemini, its flagship model, all of which exhibit some signs of being maze rats -- taking longer to answer questions, providing detailed reasoning chains, improvements on math and coding. Both it and Microsoft are reportedly exploring this "reasoning" approach. And multiple Chinese tech companies, including Alibaba, have released models built in the style of o1. If this is the way to superintelligence, it remains a bizarre one. "This is back to a million monkeys typing for a million years generating the works of Shakespeare," Emily Bender told me. But OpenAI's technology effectively crunches those years down to seconds. A company blog boasts that an o1 model scored better than most humans on a recent coding test that allowed participants to submit 50 possible solutions to each problem -- but only when o1 was allowed 10,000 submissions instead. No human could come up with that many possibilities in a reasonable length of time, which is exactly the point. To OpenAI, unlimited time and resources are an advantage that its hardware-grounded models have over biology. Not even two weeks after the launch of the o1 preview, the start-up presented plans to build data centers that would each require the power generated by approximately five large nuclear reactors, enough for almost 3 million homes. Yesterday, alongside the release of the full o1, OpenAI announced a new premium tier of subscription to ChatGPT that enables users, for $200 a month (10 times the price of the current paid tier), to access a version of o1 that consumes even more computing power -- money buys intelligence. "There are now two axes on which we can scale," Chen said: training time and run time, monkeys and years, parrots and rats. So long as the funding continues, perhaps efficiency is beside the point. The maze rats may hit a wall, eventually, too. In OpenAI's early tests, scaling o1 showed diminishing returns: Linear improvements on a challenging math exam required exponentially growing computing power. That superintelligence could use so much electricity as to require remaking grids worldwide -- and that such extravagant energy demands are, at the moment, causing staggering financial losses -- are clearly no deterrent to the start-up or a good chunk of its investors. It's not just that OpenAI's ambition and technology fuel each other; ambition, and in turn accumulation, supersedes the technology itself. Growth and debt are prerequisites for and proof of more powerful machines. Maybe there's substance, even intelligence, underneath. But there doesn't need to be for this speculative flywheel to spin.

New Beginnings: A Conversation with Mira Murati

she oversaw product teams building ChatGPT, DALL·E, Sora, and contributed to advancements in ai safety, ethics, And I could tell you, I did a story on Microsoft recently and unprompted, any number of people told me how important she was Prior to joining opening OpenAI, she managed the product you left OpenAI with a very generous and diplomatic note. I know from our prep you're not gonna be talking a lot about Can you tell us anything about what you're up to next? I'm not going to share much about what I'm doing next And yeah, generally I would totally ignore the noise and obsession about who is leaving the labs and so on and let's focus on the actual substance of things. But what I'm excited about is, you know, quite similar And I really think that we're sort of at this beginning how our civilization co-evolve with the development of science and technology as our knowledge deepens. It was a company that where people had a shared vision, humanity really was on this quest to take, you know, and before that we had college level performance. And before that, just a couple of years before that we had high school level performance. that has a capability to learn how to perform at human level even if it's not something that happens within a couple and we believed in this, what you call spiritual mission Whereas now we've made enough progress that we can kind for how AI would really advanced transportation. and particularly how it would change our relationship in exploring virtual reality, augmented reality, and it was his essay on Singularity where he talks about, you know, this is sort of likens our era to a time where, where the change is so transformational it would be the most important thing that I would do. You know, you mentioned, you know, the VR company you work or if this conference were taking place like six years ago, all people would be talking about was the metaverse, right, and I don't think any of the sessions here are about the metaverse, you know, that I thought it would happen in that particular time. I was more curious to understand this next human-machine interface and augmented reality are have definitely advanced a lot, since then. And yeah, I think we will definitely see great technologies, and they're somehow saying that at this moment, you know, which was, you know, kind of like an astounding leap. So I think one interesting observation is that people get, to come in our society's ability to adapt to more change. But in terms of whether there is a plateau or not, let's consider where the progress came from. And a lot of the progress today has come from, you know, increasing the size of the neural networks, increasing the amount of data, increasing the amount that as you increase all of these things predictably leads of different data code and images and video and so on. So we've seen a lot of advancement coming from that. we're just starting to see the rise of more agentic systems. So I expect there is going to be a lot of progress there. but I, I'm quite optimistic that the progress will continue. people are exploring things like synthetic data this year companies are spending a billion dollars and next year that goes up by a factor of 10 to 10 billion to AGI level systems is not just about capability It's about figuring out the entire social infrastructure in which these systems are going to be operated in. because this technology is not intrinsically good or bad, they got more excited about the building AGI part I think kind of like the market dynamics have pushed everyone in the industry to really innovate in that vector. I would say that civilization needs to coexist harmoniously Some people have said that the as existential threats because people aren't really building the safety stuff now, market alignment on the short term safety questions And so I think a lot of effort will actually is already in the understanding of what these systems are capable of, why we haven't been able to get rid of the Hallucinations? something like, These things are always tied together and you almost cannot distinguish, it's impossible to distinguish the lion from the lamb. And I think hallucinations are like that where it gives you, where you need very accurate information in, you know, But it's still something that we need to figure out. Some people have suggested, you know, you talked earlier But it seems to me that the more we go down this path, the more valuable, the trustworthy information is, that talked about that if models are trained on, you know, which seems to put a premium on like human created You know, it winds up to be some sort of licensing things for the best, most trustworthy models, which then sort of, I guess limits its world models. How are we going to eventually deal with this IP issue There is the aspect of, you know, how the laws evolve and figuring out and innovating perhaps in business models and understanding, doing more research and understanding how specific data contribution And another layer is definitely the research on the data like our reinforcement learning with human feedback or you're doing reinforcement learning from AI feedback, and requires a lot of human feedback or synthetic data. which can match and exceed some human capabilities and how civilization co-evolves with this technology. I think it's entirely up to us, the institutions, the structures we put into place, the level of investment, the work that we do, and really how we move forward the entire ecosystem. and constrain the actions of any specific individuals. or individual to bring AGI to the entire civilization.

Anthropic's Dario Amodei: Democracies must maintain the lead in AI

Dario Amodei has worked in the world's most advanced artificial intelligence labs: at Google, OpenAI, and now Anthropic. At OpenAI, Amodei drove the company's core research strategy, building its GPT class of models for five years, until 2021 -- a year before the launch of ChatGPT. After quitting over differences about the future of the technology, he founded Anthropic, the AI start-up now known for its industry-leading chatbot, Claude. Anthropic was valued at just over $18bn earlier this year and, last month, Amazon invested $4bn, taking its total to $8bn -- its biggest-ever venture capital commitment. Amazon is working to embed Anthropic's Claude models into the next-generation version of its Alexa speaker. Amodei, who co-founded Anthropic with his sister Daniela, came to artificial intelligence from biophysics, and is known for observing the so-called scaling laws -- the phenomenon whereby AI software gets dramatically better with more data and computing power. In this conversation with the FT's Madhumita Murgia, he speaks about new products, the concentration of power in the industry, and why an "entente" strategy is central to building responsible AI. Madhumita Murgia: I want to kick off by talking about your essay, Machines of Loving Grace, which describes in great depth the ways in which AI could be beneficial to society. Why choose to outline these upsides in this detail right now? Dario Amodei: In a sense, it shouldn't be new because this dichotomy between the risks of AI and the benefits of AI has been playing out in the world for the last two or three years. No one is more tired of it than me. On the risk side . . . I've tried to be specific. On the benefits side there, it's very motivated by techno-optimism, right? You'll see these Twitter posts with developers talking about "build, build, build" and they'll post these pictures of gleaming cities. But there's been a real lack of concreteness in the positive benefits. MM: There are a lot of assumptions when people talk about the upsides. Do you feel that there was a bit of fatigue from people . . . never being [told] what that could actually look like? DA: Yeah, the upside is being explained in either very vague, emotive terms, or really extreme. The whole singularity discourse is . . . "We're all going to upload ourselves to the cloud and whatever problem you have, of course, AI will instantly solve it". I think it is too extreme and it lacks texture. Can we actually envision a world that is good, that people want to live in? And what are the specific things that will get better? And what are the challenges around them? If we look at things like cancer and Alzheimer's, there's nothing magic about them. There's an incredible amount of complexity, but AI specialises in complexity. It's not going to happen all at once. But -- bit by bit -- we're going to unravel this complexity that we couldn't deal with before. MM: What drew you to the areas that you did pick, like biology, neuroscience, economic development and work? DA: I looked at the places that could make the most difference to human life. For me, that really pointed to biology and economic development. There are huge parts of the world where these inventions that we've developed in the developed world haven't yet propagated. I wanted to target what immediately occurred to me as some of the biggest predictors and determiners of how good life is for humans. MM: In an ideal world, what would you like to spend Anthropic's time on in 2025? DA: Two things: one would be mechanistic interpretability, looking inside the models to open the black box and understand what's inside them. I think that's the most exciting area of AI research right now, and perhaps the most societally important. And the second would be applications of AI to biology. One reason that I went from biological science to AI is I looked at the problems of biology and . . . they seemed almost beyond human scale, almost beyond human comprehension -- not that they were intellectually too difficult, but there was just too much information, too much complexity. It is my hope, like some other people in the field -- I think Demis Hassabis is also driven in this way too -- to use AI to solve the problems of science and particularly biology, in order to make human life better. Anthropic is working with pharmaceutical companies and biotech start-ups [but] it's very much at the "how can we apply Claude models right now?" level. I hope we start in 2025 to really work on the more blue-sky, long-term ambitious version of that -- both with companies and with researchers and academics. MM: You've been instrumental in pushing forward the frontiers of AI technology. It's been five months since Sonnet 3. 5, your last major model came out. Are people using it in new ways to some of the older models? DA: I'll give an example in the field of coding. I've seen a lot of users who are very strong coders, including some of the most talented people within Anthropic who have said previous models weren't useful to [them] at all. They're working on some hard problem, something very difficult and technical, and they never felt that previous models actually saved them time. It's just like if you're working with another human: if they don't have enough of the skill that you have, then collaborating with them may not be useful. But I saw a big change in the number of extremely talented researchers, programmers, employees . . . for whom Sonnet 3.5 was the first time that the models were actually helpful to them. Another thing I would point to is Artifacts: a tool on the consumer side of Claude. [With it,] you can do back-and-forth development. You can have this back-and-forth where you tell the model: "Make a video game for me where the main character looks like this, and the environment looks like this". And, then, they'll make it. [But] you can go back and talk to it and say: "I don't think my main character looks right. He looks like Mario. I want him to look more like Luigi." Again, it shows the collaborative development between you and the AI system. MM: Has this led to revenue streams or business models you're excited about? Do you think there are new products that you can envision coming out of it, based on these new capabilities? DA: Yes. While we have a consumer product, the majority of Anthropic's business has come from selling our model to other businesses, via an API on which they build these products. So I think our general position in the ecosystem has been that we're enabling other companies to build these amazing products and we've seen lots of things that have been built. For example, last month, we released a capability called "Computer Use" to developers. Developers can build on top of this capability: you can tell it, "book me a reservation at this restaurant" or "plan a trip for this day", and the model will just directly use your computer. It'll look at the screen. It'll click at various positions on the mouse. And it will type in things using the keyboard. It's not a physical robot, but it's able to type in . . . automate and control your computer for you. Within a few days of when we released it, people had released versions that control an iPhone screen and Android screen, Linux, Mac. MM: Is that something you would release as its own product? The word being thrown around everywhere these days is an agent. You could have your own version of that, right? DA: Yes, I can imagine us directly making a product that would do this. I actually think the most challenging thing about AI agents is making sure they're safe, reliable and predictable. It's one thing when you talk to a chatbot, right? It can say the wrong thing. It might offend someone. It might misinform someone. Of course, we should take those things seriously. But making sure that the models do exactly what we want them to do becomes much more highlighted when we start to work with agents. MM: What are some of the challenges? DA: As a thought experiment, just imagine I have this agent and I say: "Do some research for me on the internet, form a hypothesis, and then go and buy some materials to build [some]thing, or, make some trades undertaking my trading strategy." Once the models are doing things out there in the world for several hours, it opens up the possibility that they could do things I didn't want them to do. Maybe they're changing the settings on my computer in some way. Maybe they're representing me when they talk to someone and they're saying something that I wouldn't endorse at all. Maybe they're taking some action on another set of servers. Maybe they're even doing something malicious. So, the wildness and unpredictability needs to be tamed. And we've made a lot of progress with that. It's using the same methods that we use to control the safety of our ordinary systems, but the level of predictability you need is substantially higher. I know this is what's holding it up. It's not the capabilities of the model. It's getting to the point where we're assured that we can release something like this with confidence and it will reliably do what people want it to do; when people can actually have trust in the system. Once we get to that point, then we'll release these systems. MM: Yes, the stakes are a lot higher when it moves from it telling you something you can act on, versus acting on something for you. DA: Do you want to let a gremlin loose in the internals of your computer to just change random things? You might never know what changed those things. To be clear, I think all these problems are solvable. But these are the practical challenges we face when we design systems like this. MM: So when do you think we get to a point of enough predictability and mundanity with these agents that you'd be able to put something out? DA: This is an early product. Its level of reliability is not all that high. Don't trust it with critical tasks. I think we'll make a lot of progress towards that by 2025. So I would predict that there will be products in 2025 that do roughly this, but it's not a binary. There will always still be tasks that you don't quite trust an AI system to do because it's not smart enough or not autonomous enough or not reliable enough. I'd like us to get to the point where you can just give the AI system a task for a few hours -- similar to a task you might give to a human intern or an employee. Every once in a while, it comes back to you, it asks for clarification, and then it completes the task. If I want to have a virtual employee, where I say go off for several hours, do all this research, write up this report -- think of a management consultant or a programmer -- people [must have] confidence that it'll actually do what you said it would do, and not some crazy other thing. MM: There's been talk recently about how these capabilities are perhaps plateauing, and we're starting to see limits to the current techniques, in what is known as the "scaling law". Are you seeing evidence of this, and looking at alternative ways in which to scale up intelligence in these models? DA: I've been in this field for 10 years and I've been following the scaling laws for most of that period. I think the thing we're seeing is in many ways pretty ordinary and has happened many times during the history of the field. It's just that, because the field is a bigger deal with more economic consequences, more people are paying attention to it [now]. And very much over-interpreting very ambiguous data. If we go back to the history, the scaling laws don't say that anytime you train a larger model, it does better. The scaling laws say that if you scale up models with the model size in proportion to the data, if all the engineering processes work well in training the models, if the quality of the data remains constant, as you scale it up, [then] . . . the models will continue to get better and better. MM: And this, as you say, isn't a mathematical constant, right? DA: It's an observed phenomenon and nothing I've seen gives any evidence whatsoever against this phenomenon. We've seen nothing to refute the pattern that we've seen over the last few years. What I have seen [is] cases where, because something wasn't scaled up in quite the right way the first time, it would appear as though things were levelling off. There were four or five other times at which this happened. MM: So in the current moment, when you're looking at your training runs of your current models, are there any limitations? DA: I've talked many times about synthetic data. As we run out of natural data, we start to increase the amount of synthetic data. So, for example, AlphaGo Zero [a version of Google DeepMind's Go-playing software] was trained with synthetic data. Then there are also reasoning methods, where you teach the model to self-reflect. So there are a number of ways to get around the data wall. MM: When we talk about scaling, the big requirement is cost. Costs seem to be rising steeply. How does a company like Anthropic survive when the costs are going up like that? Where is this money coming from over the next year or so? DA: I think people continue to understand the value and the potential of this technology. So I'm quite confident that some of the large players that have funded us and others, as well as the investment ecosystem, will support this. And revenue is growing very fast. I think the math for this works. I'm pretty confident the level of say, $10bn -- in terms of the cost of the models -- is something that an Anthropic will be able to afford. In terms of profitability, this is one thing that a number of folks have gotten wrong. People often look at: how much did you spend and how much did something cost, in a given year. But it's actually more enlightening to look at a particular model. Let's just take a hypothetical company. Let's say you train a model in 2023. The model costs $100mn dollars. And, then, in 2024, that model generates, say, $300mn of revenue. Then, in 2024, you train the next model, which costs $1bn. And that model isn't done yet, or it gets released near the end of 2024. Then, of course, it doesn't generate revenue until 2025. So, if you ask "is the company profitable in 2024", well, you made $300mn and you spent $1bn, so it doesn't look profitable. If you ask, was each model profitable? Well, the 2023 model cost $100mn and generated several hundred million in revenue. So, the 2023 model is a profitable proposition. These numbers are not Anthropic numbers. But what I'm saying here is: the cost of the models is going up, but the revenue of each model is going up and there's a mismatch in time because models are deployed substantially later than they're trained. MM: Do you think it's possible for a company like Anthropic to do this without a hyperscaler [like Amazon or Google]? And do you worry about their concentrating power, since start-ups building LLMs can't actually work without their funding, without their infrastructure? DA: I think the deals with hyperscalers have made a lot of sense for both sides [as] investment is a way to bring the future into the present. What we mainly need to buy with that money is chips. And both the company and the hyperscaler are going to deploy the products on clouds, which are also run by hyperscalers. So it makes economic sense. I'm certainly worried about the influence of the hyperscalers, but we're very careful in how we do our deals. The things that are important to Anthropic are, for example, our responsible scaling policy, which is basically: when your models' capabilities get to a certain level, you have to measure those capabilities and put safeguards in place if they are going to be used. In every deal we've ever made with a hyperscaler, it has to bind the hyperscaler, when they deploy our technology, to the rules of our scaling policy. It doesn't matter what surface we're deploying the model on. They have to go through the testing and monitoring that our responsible scaling calls for. Another thing is our long-term benefit trust. It's a body that ultimately has oversight over Anthropic. It has the hard power to appoint many of Anthropic's board seats. Meanwhile, hyperscalers are not represented on Anthropic's board. So the ultimate control over the company remains in the hands of the long-term benefit trust, which is financially disinterested actors that have ultimate authority over Anthropic. MM: Do you think it's viable for an LLM-building company today to continue to hold the power in terms of the products you produce and the impact it has on people, without an Amazon or Google or a Microsoft? DA: I think it's economically viable to do it while maintaining control over the company. And while maintaining your values. I think doing it requires a large amount of resources to come from somewhere. That can be from a hyperscaler. That could, in theory, be from the venture capital system. That could even be from a government. We've seen some cases, for better or worse, [in which] individuals like Elon Musk are taking their large private wealth and using that. I do think [that] to build these very large foundation models requires some very large source of capital, but there are many different possible sources of capital. And I think it's possible to do it while staying in line with your values. MM: You recently signed a deal with the US Department of Defense. Was that partly a funding decision? DA: No, it absolutely was not a funding decision. Deploying things with governments, at the procurement stage? Anyone who's starting up a company will tell you that, if you want to get revenue quickly, that's just about the worst way to do it. We're actually doing that because it's a decision in line with our values. I think it's very important that democracies maintain the lead in this technology and that they're properly equipped with resources to make sure that they can't be dominated or pushed around by autocracies. One worry I have is, while the US and its allies may be ahead of other countries in the fundamental development of this technology, our adversaries -- like China or Russia -- may be better at deploying what they have to their own governments. I wouldn't do this if it were just a matter of revenue. It's something I actually believe . . . is central to our mission. MM: You wrote about this "entente strategy", with a coalition of democracies building AI. Is it part of your responsibility as an AI company to play a role in advancing those values as part of the [military] ecosystem? DA: Yes, I think so. Now, it's important to do it carefully. I don't want a world in which AIs are used indiscriminately in military and intelligence settings. As with any other deployment of the technology -- maybe even more so -- there need to be strict guardrails on how the technology is deployed. Our view as always is we're not dogmatically against or for something. The position that we should never use AI in defence and intelligence settings doesn't make sense to me. The position that we should go gangbusters and use it to make anything we want -- up to and including doomsday weapons -- that's obviously just as crazy. We're trying to seek the middle ground, to do things responsibly. MM: Looking ahead to artificial general intelligence, or super intelligent AI, how do you envision those systems? Do we need new ideas to make the next breakthroughs? Or will it be iterative? DA: I think innovation is going to coexist with this industrial scaling up. Getting to very powerful AI, I don't think there's one point. We're going to get more and more capable systems over time. My view is that we're basically on the right track and unlikely to be more than a few years away. And, yeah, it's going to be continuous, but fast.