Nine people at Anthropic are tasked with preventing AI from causing harm as company soars to $183B

The nine people trying to stop AI from ruining the world

One night in May 2020, during the height of lockdown, Deep Ganguli was worried. Ganguli, then research director at the Stanford Institute for Human-Centered AI, had just been alerted to OpenAI's new paper on GPT-3, its latest large language model. This new AI model was potentially 10 times more advanced than any other of its kind -- and it was doing things he had never thought possible for AI. The scaling data revealed in the research suggested there was no sign of it slowing down. Ganguli fast-forwarded five years in his head, running through the kinds of societal implications he spent his time at Stanford anticipating, and the changes he envisioned seemed immeasurable. He knew he couldn't sit on the sidelines while the tech rolled out. He wanted to help guide its advancement. His friend Jack Clark had joined a new startup called Anthropic, founded by former OpenAI employees concerned that the AI giant wasn't taking safety seriously enough. Clark had previously been OpenAI's policy director, and he wanted to hire Ganguli at Anthropic for a sweeping mission: ensure AI "interacts positively with people," in everything from interpersonal interactions to the geopolitical stage. Over the past four years, Ganguli has built what's known as Anthropic's societal impacts team, a small group that's looking to answer the thorniest questions posed by AI. They've written research papers on everything from AI's economic impact to its persuasiveness, as well as explorations of how to mitigate elections-related risks and discrimination. Their work has, perhaps more than any other team, contributed to Anthropic's carefully tended reputation as the "safe" AI giant dedicated to putting humans first. But with just nine people among Anthropic's total staff of more than 2,000, in an industry where mind-boggling profits could await whoever's willing to move quickest and most recklessly, the team's current level of freedom may not last forever. What happens when just a handful of employees at one of the world's leading AI companies -- one that nearly tripled its valuation to $183 billion in less than a year, and is now valued in the range of $350 billion -- are given the blanket task of figuring out how the ultra-disruptive technology is going to impact society? And how sure are they that executives, who are at the end of the day still looking to eventually turn a profit, will listen? Nearly every major AI company has some kind of safety team that's responsible for mitigating direct, obvious harms like AI systems being used for scams or bioweapons. The goal of the societal impacts team -- which does not have a direct analog at OpenAI, Meta, or Anthropic's other big competitors -- is broader. Ganguli sees his job as finding "inconvenient truths" about AI that tech companies have incentives not to publicize, then sharing them with not only Anthropic leadership, but the rest of the world. "We are going to tell the truth," Ganguli said. "Because, one, it's important. It's the right thing to do. Two, the stakes are high. These are people. The public deserves to know. And three, this is what builds us trust with the public, with policymakers. We're not trying to pull the wool over anyone's eyes. We're just trying to say what we're seeing in the data." The team meets in the office five days a week, spending a good amount of time in Anthropic's eighth-floor cafeteria, where Saffron Huang, one of the research scientists, usually grabs a flat white before a working breakfast with Ganguli and others. ("That's the Kiwi in me," says Huang, a New Zealander who founded a nonprofit in London before joining Anthropic in 2024.) Team members work out together at the gym and have late nights at the office and day trips to the beach. They've met each other's mothers and ridden in each other's cars while picking up their kids from school. They see so much of each other that Ganguli sometimes forgoes after-work hangouts -- "I see you all more than my family!" a team member recalls him saying. The result is a level of comfort voicing opinions and disagreements. The group is big on the "cone of uncertainty," a phrase they use when, in true scientist fashion, they're not sure about aspects of the data they're discussing. It's also the name of a literal traffic cone that research engineer Miles McCain and Anthropic's facility team found, cleaned up, and fixed with googly eyes before installing it in the office. The societal impacts team launched as Ganguli's one-man operation when Anthropic was solely a research lab. Research scientist Esin Durmus joined him in February 2023, as Anthropic was gearing up to launch Claude the following month. Their work involved considering how a real future product might affect humanity -- everything from how it could impact elections to "which human values" it should hold. Durmus' first research paper focused on how chatbots like Claude could offer biased opinions that "may not equitably represent diverse global perspectives on societal issues." Around Claude's launch, the team relied on testing models before deployment, attempting to anticipate how people would engage with them. Then, suddenly, thousands -- later millions -- of people were using a real product in ways the team had no way to gauge. AI systems, they knew, were unpredictable. For a team designed to measure the impact of a powerful new technology, they knew frustratingly little about how society was using it. This was an unprecedented cone of uncertainty, spurring what eventually became one of the team's biggest contributions to Anthropic so far: Claude's tracking system, Clio. Anthropic needed to know what people were doing with Claude, the team decided, but they didn't want to feel like they were violating people's trust. "If we're talking about insight versus privacy, you can have a ton of insight by having no privacy," Ganguli said, adding, "You could also have a ton of privacy with zero insight." They struck a balance after consulting with Anthropic engineers and external civil society organizations, resulting in, essentially, a chatbot version of Google Trends. Clio resembles a word cloud with clusters of topics describing how people are using Claude at any given time, like writing video scripts, solving diverse math problems, or developing web and mobile applications. The smaller clusters run the gamut from dream interpretation and Dungeons & Dragons to disaster preparedness and crossword puzzle hints. Today, Clio is used by teams across Anthropic, offering insight that helps the company see how well safeguards and reinforcement learning are working. (There's a Slack channel called Clio Alerts that shares automated flags on what each team is doing with the tool; Ganguli says he often stares at it.) It's also the basis of much of the societal impacts team's own work. One of the most "inconvenient truths" the team has released came from using Clio to analyze Anthropic's safety monitoring systems. Together with the safeguards team, Miles McCain and Alex Tamkin looked for harmful or inappropriate ways people were using the platform. They flagged uses like the creation of "explicit pornographic stories with graphic sexual content," as well as a network of bots that were trying to use Claude's free version to create SEO-optimized spam, which Anthropic's own safety classifiers hadn't picked up -- and they published the research in hopes that it'd help other companies flag their own weaknesses. The research led to Anthropic stepping up its detection of "coordinated misuse" at the individual conversation level, plus figuring out how to monitor for issues they may not be able to even name yet. "I was pretty surprised that we were able to just be quite transparent about areas where our existing systems were falling short," said McCain, who built the Clio tool and also focuses on how people use Claude for emotional support and companionship, as well as limiting sycophancy. He mentioned that after the team published that paper, Anthropic made Clio an "important part of our safety monitoring stack." As team leader, Ganguli talks the most with executives, according to members -- although the team presents some of their research results every so often on an ad hoc basis, he's the one with the most direct line to leadership. But he doesn't talk to Anthropic CEO Dario Amodei regularly, and the direct line doesn't always translate to open communication. Though the team works cross-functionally, the projects are rarely assigned from the top and the data they analyze often informs their next moves, so not everyone always knows what they're up to. Ganguli recalled Amodei once reaching out to him on Slack to say that they should study the economic impacts of AI and Anthropic's systems, not realizing the societal impacts team had already been discussing ways to do just that. That research ended up becoming Anthropic's Economic Index, a global tracker for how Claude is being used across each state and the world -- and how that could impact the world economy. When pressed on whether executives are fully behind the team's work, even if it were not to reflect well on the company's own technology, team members seem unfazed -- mostly because they say they haven't had any tangible reasons to worry so far. "I've never felt not supported by our executive or leadership team, not once in my whole four years," Ganguli said. The team also spends a good bit of time collaborating with other internal teams on their level. To Durmus, who worked on a paper charting the types of value judgments Claude makes, the societal impacts team is "one of the most collaborative teams" at the company. She said they especially work with the safeguards, alignment, and policy teams. McCain said the team has an "open culture." Late last year, he said, the group worked closely with Anthropic's safety team to understand how Claude could be used for nefarious election-related tasks. The societal impacts team built the infrastructure to run the tests and ran periodic analyses for the safety team -- then the safety team would use those results to decide what they'd prioritize in their election safety work. And since McCain and his colleagues only sit a couple of rows of desks away from the trust and safety employees, they also have a good working relationship, he said, including a Slack channel where they can send concerns their way. But there's a lot we don't know about the way they work. There's a tungsten cube on Saffron Huang's desk, apparently. I have to take her word on that, as well as any other details about the team's working environment, because most of Anthropic's San Francisco headquarters is strictly off-limits to visitors. I'm escorted past a chipper security desk with peel-and-stick nametags and an artful bookshelf, and then it's into the elevator and immediately to the office barista, who's surrounded by mid-century modern furniture. (I'm proudly told by members of Anthropic's public relations team, who never leave my side, that the office is Slack's old headquarters.) I'm swiftly escorted straight into a conference room that tries to mask its sterile nature with one warm overhead light and a painting of a warped bicycle on the wall. I ask if I can see Huang and the rest of the team's workspace. No, I'm told, that won't be possible. Even a photo? What about a photo with redacted computer screens, or getting rid of everything on the desks that could in any way be sensitive? I'm given a very apologetic no. I move on. Huang's tungsten cube probably looks just like any other. But the fact I can't confirm that is a reminder that, though the team is committed to transparency on a broad scale, their work is subject to approval from Anthropic. It's a stark contrast with the academic and nonprofit settings most of the staff came from. Huang's first brush with Anthropic came in 2023. She'd started a nonprofit called the Collective Intelligence Project, which sought to make emerging technologies more democratic, with public input into AI governance decisions. In March 2023, Huang and her cofounder approached Anthropic about working together on a project. The resulting brainstorming session led to their joint "collective constitutional AI" project, an exercise in which about 1,000 randomly chosen Americans could deliberate and set rules on chatbot behavior. Anthropic compared what the public thought to its own internal constitution and made some changes. At the time of the collaboration, Huang recalls, Anthropic's societal impacts team was only made up of three people: Ganguli, Durmus, and Tamkin. Huang was considering going to grad school. Ganguli talked her out of it, convincing her to join the societal impacts team. The AI industry is a small world. Researchers work together in one place and follow the people they connect with elsewhere. Money, obviously, could be a major incentive to pick the private sector over academia or nonprofit work -- annual salaries are often hundreds of thousands of dollars, plus potentially millions in stock options. But within the industry, many employees are "post-money" -- in that AI engineers and researchers often have such eye-popping salaries that the only reason to stay at one job, or take another, is alignment with a company's overall mission. "To me, being in a healthy culture, having these team dynamics, working together toward a good purpose, building safe AI that can benefit everyone -- that comes before anything, including a lot of money," Durmus said. "I care about this more than that." Michael Stern, an Anthropic researcher focused on AI's economic impact, called the societal impacts team a "lovely mix of misfits in this very positive way." He'd always had trouble fitting into just one role, and this team at Anthropic allowed him to combine his interests in safety, society, and security with engineering and policy work. Durmus, the team's first hire after Ganguli himself, had always been interested in both computer science and linguistics, as well as how people interact and try to sway each other's opinions online. Kunal Handa, who now works on economic impact research and how students use Claude, joined after cold-emailing Tamkin while Handa was a graduate student studying how babies learn concepts. Tamkin, he had noticed, was trying to answer similar questions at Anthropic, but for computers instead. (Since time of writing, Tamkin has moved to Anthropic's alignment team, to focus on new ways to understand the company's AI systems and making them safer for end users.) In recent years, many of those post-money people concerned with the advancement (and potential fallout) of AI have left the leading labs to go to policy firms or nonprofits, or even start their own organizations. Many have felt they could have more impact in an external capacity. But the societal impacts team's broad scope and expansive job descriptions still prove more attractive for several team members. "I am not an academic flight risk ... I find Deep's pitch so compelling that I never even really considered that path," McCain said. For Ganguli himself, it's a bit different. He speaks a lot about his belief in "team science" -- people with different backgrounds, training, and perspectives all working on the same problem. "When I think about academia, it can be kind of the opposite -- everyone with the same training working on a variety of different problems," Ganguli said, adding that at Stanford, he sometimes had trouble getting people to emulate team science work, since the university model is set up differently. At Anthropic, he also values having access to usage data and privileged information, which he wouldn't be able to study otherwise. Ganguli said that when he was recruiting Handa and Huang, they were both deciding between offers for graduate school at MIT or joining his team at Anthropic. "I asked them, 'What is it that you actually want to accomplish during your PhD?' And they said all the things that my team was working on. And I said, 'Wait, but you could just actually do that here in a supportive team environment where you'll have engineers, and you'll have designers, and you'll have product managers -- all this great crew -- or you could go to academia where you'll kind of be lone wolf-ing it.'" He said their main concerns involved academia potentially having more freedom to publish inconvenient truths and research that may make AI labs look less than optimal. He told them that at Anthropic, his experience so far has been that they can publish such truths -- even if they reveal things that the company needs to fix. Of course, plenty of tech companies love transparency until it's bad for business. And right now, Anthropic in particular is walking a high-stakes line with the Trump administration, which regularly castigates businesses for caring about social or environmental problems. Anthropic recently detailed its efforts to make Claude more politically middle-of-the-road, months after President Donald Trump issued a federal procurement ban on "woke AI." It was the only AI company to publicly voice its stance against the controversial state AI law moratorium, but after its opposition earned it the ire of Trump's AI czar David Sacks, Amodei had to publish a public statement boosting Anthropic's alignment with aspects of Trump administration policy. It's a delicate balancing act that a particularly unwelcome report could upset. But Ganguli is confident the company will keep its promise to his team, whatever's happening on the outside. "We've always had the full buy-in from leadership, no matter what," he said. Ask each member of Anthropic's societal impacts team about their struggles and what they wish they could do more of, and you can tell their positions weigh heavily on them. They clearly feel that an enormous responsibility rests upon their shoulders: to shine a light on how their company's own technology will impact the general public. People's jobs, their brains, their democratic election process, their ability to connect with others emotionally -- all of it could be changed by the chatbots that are filling every corner of the internet. Many team members believe they'll do a better job guiding how that tech is developed from the inside rather than externally. But as the exodus of engineers and researchers elsewhere shows, that idealism doesn't always pan out for the broader AI industry. A struggle that the majority of team members brought up was time and resource constraints -- they have many more ideas than they have bandwidth for. The scope of what the team does is broad, and they sometimes bite off more than they can chew. "There are more coordination costs when you're 10 times the size as you were two years ago," Tamkin said. That pairs, sometimes, with the late nights -- i.e., "How am I going to talk to 12 different people and debug 20 different errors and get enough sleep at night in order to release a report that feels polished?" The team, for the most part, would also like to see their research used more internally: to directly improve not only Anthropic's AI models, but also specific end products like Claude's consumer chatbot or Claude Code. Ganguli has one-on-one meetings monthly with chief science officer Jared Kaplan, and they often brainstorm ways to allow the societal impacts team to better impact Anthropic's end product. Ganguli also wants to expand the team soon, and many team members hope that type of resource expansion means they'll be able to better document how users are interacting with Claude -- and the most surprising, and potentially concerning, ways in which they're doing so. Many team members also brought up the fact that looking at data in a vacuum or lab setting is very different from the effect AI models have in the real world. Clio's analysis of how people are using Claude can only go so far. Simply observing use cases and analyzing aggregated transcripts doesn't mean you know what your customers are doing with the outputs, whether they're individual consumers, developers, or enterprises. And that means "you're left to sort of guess what the actual impact on society will be," McCain said, adding that it's a "really important limitation, and [it] makes it hard to study some of the most important problems." As the team wrote in a paper on the subject, "Clio only analyzes patterns within conversations, not how these conversations translate into real-world actions or impacts. This means we cannot directly observe the full societal effects of AI system use." It's also true that until recently, the team could only really analyze and publish consumer usage of Claude via Clio -- in September, for the first time, the team published an analysis of how businesses are using Claude via Anthropic's API. "Models and AI systems don't exist in isolation -- they exist in the context of their deployments, and so over the past year, we've really emphasized studying those deployments -- the ways that people are interacting with Claude," McCain said. "That research is going to have to also evolve in the future as the impacts of AI affect more and more people, including people who may not be interfacing with the AI system directly ... Concentric circles outward." That's why one of the team's next big research areas is how people use Claude not just for its IQ, but also for its EQ, or emotional intelligence. Ganguli says that a lot of the team's research to date has been focused on cut-and-dried answers and measurable impacts on the economy or labor market, and that its EQ research is relatively new -- but the team will prioritize it in the next six months. "Once people leave the chatbot, we're not entirely sure exactly how they were affected or impacted, and so we're trying to develop new methods and new techniques that allow us to understand," he said, referring to taking a more "human-centered approach" and doing more "social science research" akin to coupling data analysis with surveys and interviews. Since people are emotionally influenced by their social networks, it stands to reason they can be influenced greatly by AI agents and assistants. "People are going to Claude ... looking for advice, looking for friendship, looking for career coaching, thinking through political issues -- 'How should I vote?' 'How should I think about the current conflicts in the world?'" Ganguli said. "That's new ... This could have really big societal implications of people making decisions on these subjective things that are gray, maybe more matters of opinion, when they're influenced by Claude, or Grok, or ChatGPT, or Gemini, or any of these things." By far the most pressing EQ-related issue of the day is widely known as "AI psychosis." The phenomenon references a range of conditions where AI leads a user down a delusional spiral and causes them, on some level, to lose touch with reality. The user typically forms an emotional bond with a chatbot, made more intense by the chatbot's memory of previous conversations and its potential to drift away from safety guardrails over time. Sometimes this can lead to the user believing they've unearthed a romantic partner "trapped" inside the chatbot who longs to be free; other times it can lead to them believing they've discovered new secrets to the universe or scientific discoveries; still other times it can lead to widespread paranoia and fear. AI psychosis or delusion has been a main driver behind some teen suicides, as well as ensuing lawsuits, Senate hearings, newly passed laws, and parental controls. The issue, experts say, is not going anywhere. "What does it mean for our world, in which you have a machine with endless empathy you can basically just dump on, and it'll always kind of tell you what it thinks?" Ganguli said. "So the question is: What are the kinds of tasks people are using Claude for in this way? What kind of advice is it giving? We've only just started to uncover that mystery."

Anthropic is all in on 'AI safety' -- and that's helping the $183 billion startup win over big business | Fortune

Dario Amodei is, in his telling, the accidental CEO of an accidental business -- one that just happens to be among the fastest-growing on the planet. "When we first started Anthropic, we didn't have any idea about how we would make money, or when, or under what conditions," he says. Anthropic is the San Francisco-based AI company that Amodei cofounded and leads. And it hasn't taken long for it to start pulling in lots of money, under lots of conditions. The startup has emerged as one of the leading rivals to OpenAI and Google in the race to build ever-more-capable artificial intelligence. And while Anthropic and its Claude family of AI models don't have quite the same brand recognition as crosstown rival OpenAI and its ChatGPT products, over the past year Claude has quietly emerged as the model that businesses seem to like best. Anthropic, currently valued at $183 billion, has by some metrics pulled ahead of its larger rivals, OpenAI and Google, in enterprise usage. The company is on track to hit an annualized run rate of close to $10 billion by year-end -- more than 10 times what it generated in 2024. It also told investors in August that it could bring in as much as $26 billion in 2026, and a staggering $70 billion in 2028. Even more remarkably, Anthropic is generating such growth without spending nearly as much as some rivals -- at a time when massive capital expenditures across the industry are stoking anxiety about an AI bubble. (OpenAI alone has signed AI infrastructure deals worth more than $1 trillion.) That's in part because Anthropic says it has found ways to train and run its AI models more efficiently. To be sure, Anthropic is nowhere near profitable today: It was pacing to end 2025 having consumed $2.8 billion more cash than it took in, according to recent news accounts citing forecasts provided to investors. But the company is also on track to break even in 2028, according to those projections -- two years ahead of OpenAI. On the AI infrastructure spending race, Amodei can be sardonic. "These announcements are kind of frothy," he says. "Business should care about bringing in cash, not setting cash on fire, right?" Of his rivals, he quips: "Can you buy so many data centers that you over-leverage yourself? All I'll say is, some people are trying." Anthropic's commercial traction is in some ways profoundly ironic. The company was founded in 2020 by Amodei, his sister Daniela, and five other former OpenAI employees who broke away from that company, in part, because they were concerned it was putting too much emphasis on commercial products over "AI safety," the effort to ensure AI doesn't pose significant risks to humanity. At Anthropic, safety was going to be the sine qua non. "AI safety continues to be the highest-level focus," Amodei says of Anthropic as we sit in his office -- in a building adjacent to Salesforce Tower that once housed the offices of Slack, and whose 10 stories are now completely occupied by Anthropic. But the company soon found what Amodei calls "synergies" between its work on safety and building models that would appeal to enterprises. "Businesses value trust and reliability," he says. In another twist, the emphasis on trust and caution that has helped Anthropic gain traction with big business has entangled the company in conflicts with influential figures in politics and business. Key Trump administration officials range from skeptical to downright hostile to Anthropic's positions on AI safety and its advocacy for regulation. The company has clashed with Nvidia CEO Jensen Huang -- over Anthropic's support for limiting exports of AI chips to China -- and with Salesforce CEO Marc Benioff over Amodei's warnings about AI-induced job losses. The opprobrium of these influential figures is just one obstacle Anthropic is navigating. It has also faced lawsuits over its use of copyrighted books and music to train Claude. It agreed to settle one class action lawsuit with authors over its use of pirated libraries of books for $1.5 billion in September. That's cash the company would rather spend on growth, but had it lost in court, Anthropic might have been bankrupted. It's a lot for a young company to manage, especially one undergoing hypergrowth. Anthropic had fewer than 200 employees in late 2023. Today it has approximately 2,300. It's hiring an army of salespeople, customer support engineers, and marketing professionals, even as it staffs up on researchers to push the frontier of AI development. It's also expanding internationally at a rapid clip. Just since September, it has opened Paris and Tokyo offices, and announced ones in Munich, Seoul, and Bengaluru, adding to its existing global footprint of Dublin, Zurich, and London. Having established itself as "the AI company for business," Anthropic's challenge is to keep that title in an industry where performance leaderboards can shift overnight and market gains can quickly disappear. As its rocket ship burns through the stratosphere, the question is, Can Anthropic achieve escape velocity? Or will powerful forces -- the gravitational pull of the immense costs associated with cutting-edge AI models, the buffeting winds from political turbulence and intense competition, and the internal pressures inherent in managing an organization growing at supersonic rates -- send it spinning back down to earth? Safety leads to sales Dario Amodei has a head of curly-brown hair, and as he speaks, he absent-mindedly twirls a lock of it around his finger, as if reeling back in the thoughts unspooling from his lips as he muses about AI security and trust issues like "prompt injection" and hallucinations. Many of the mysteries Anthropic is most interested in unlocking through its research -- how to make sure models adhere to human intentions and instructions (what's known in the AI field as "alignment") and how to peer inside the brains of large language models to figure out why they generate certain outputs (or "interpretability") -- are things businesses care about too. The 42-year-old Amodei has a penchant for dressing in what might best be described as academic chic. (The day we meet, he's wearing a shawl-necked navy sweater over a white T-shirt, with blue trousers, and dark Brooks running shoes rounding off the look.) It's perhaps a sartorial vestige of his former life: Prior to his current role, he had always been a scientist, first in physics, then computational neuroscience, and finally AI research. "When I started this company, I'd never run a company before, and I certainly didn't know anything about business," Amodei says. "But the best way to learn, especially something practical like that, is just doing it and iterating fast." While Dario focuses on vision, strategy, research, and policy, sister Daniela, who is nearly four years younger, serves as Anthropic's president, handles day-to-day operations, and oversees the commercial side of the business. "We're like yin and yang" in roles and responsibilities, Daniela says of herself and Dario, but "extremely aligned" on values and direction. She allows that one of the benefits of working with your sibling is that there's always someone around who can call you on your bulls-t. "You have sibling privileges," she says. "Sometimes I'm like, 'Hey, I know this is what you meant, but people didn't hear it that way.' Or he'll just be, like, 'You're coming across, um, you're cranky.'" Under the Amodeis, Anthropic's emphasis on business has helped it to differentiate itself from OpenAI, which has 800 million weekly users and has increasingly catered to them by rolling out consumer products -- from viral video-creation tool Sora to an Instant Checkout feature for e-commerce. Claude has tens of millions of individual users, according to news accounts (Anthropic hasn't disclosed those numbers), but Anthropic says most of these people are using Claude for work and productivity, not for entertainment or companionship. Amodei says focusing on enterprises better aligns Anthropic's incentives around safety with those of its customers. Consumer-focused businesses, he says, tend to wind up trying to monetize users' attention through advertising, which gives them an incentive to make products addictive. That, he says, leads to apps that serve up "AI slop" or chatbots designed to serve as "AI girlfriends." (Amodei doesn't mention OpenAI by name, but that company has made controversial moves toward doing both those things.) "For each of these things, it's not that I object in principle," he says. "There could be some good way to do them. But I'm not sure the incentives point toward the good way." More important, for enterprise customers, safety is a persuasive selling point. Many feel that, as a result of Anthropic's innovations, it is harder for users to push Claude to jump its guardrails and produce problematic outputs, whether that's giving someone instructions for making a bioweapon, revealing company secrets, or spewing hate speech. Whatever their motives, business customers are eagerly signing up. The company says it has more than 300,000 enterprise customers, and that the number of those on pace to spend more than $100,000 annually with the company has risen sevenfold in the past year. Menlo Ventures, an Anthropic investor, has released survey data showing it with about a third of the enterprise market, compared with 25% for OpenAI and about 20% for Google Gemini. OpenAI disputes the reliability of these numbers, noting that it has more than 1 million business customers. But data that the companies shared with investors this summer showed that Anthropic had pulled ahead of its much larger rival in revenue derived from their respective APIs -- the interfaces through which enterprises access their models when building AI-enabled products and services. Anthropic reported $3.1 billion from its API compared with $2.9 billion for OpenAI. Nick Johnston, who leads the strategic technology partnerships team at Salesforce, says Salesforce's own customers, especially in finance and health care, pushed his company to forge a closer relationship with Anthropic because they felt the model was more secure than competitors. (Public safety benchmarks run by independent organizations bear this out.) Some of Claude's better performance is down to a technique Anthropic pioneered called "constitutional AI." This involves giving Claude a written constitution -- a set of principles -- that is used to train the model. Anthropic drew the principles for Claude's current constitution from sources as varied as the UN Universal Declaration of Human Rights, Apple's terms of service, and rules that Anthropic competitor Google DeepMind developed and published for Swallow, a chatbot it created in 2022. Dave Orr, Anthropic's head of safeguards, says there's much more that goes into making Claude secure. The company screens out certain information -- such as scientific papers on potentially dangerous viruses -- from Claude's initial training data. It also applies what it calls "constitutional classifiers," other AI models that screen users' prompts for jailbreaking attempts and monitor Claude's outputs to ensure they comply with the constitution. Anthropic employs "red-teamers" to probe for vulnerabilities that Orr's teams then try to fix. It also has a "threat intelligence" group that investigates users whose prompts raise red flags. That team has uncovered Chinese hackers using Claude to penetrate critical infrastructure networks in Vietnam, and North Korean fraudsters using Claude to land IT jobs at U.S. companies. Anthropic executives stress that Claude's reliability as a business tool is essentially inextricable from its emphasis on safety. Kate Jensen, who heads Anthropic's Americas operation and was until recently head of sales and partnerships, says that a lot of customers prefer Claude because they trust it to just work. Does the model rarely hallucinate? Can it follow instructions reliably? "Does the model do what you asked it to do? Yes or no?" she asks, rhetorically. "That shouldn't really be a massive enterprise differentiator, but right now in AI, it is. And for us, it's always been table stakes." Winning at coding Indeed, Claude has been winning enterprise customers largely because it performs better than rivals at tasks businesses care about. This has been particularly true for coding, where Claude has, until recently, dominated almost all the public performance benchmarks. Claude drafts about 90% of Anthropic's own code, although human software developers check it and edit it. "Claude Code" -- a tool specifically for software developers that debuted in February -- supercharged Claude's adoption. David Kossnick, head of AI products at design software company Figma, says his company built many of its early generative AI features using OpenAI's models. But when Figma decided to create Figma Make, a product that lets users design and build functional prototypes and working apps from typed instructions, it chose Claude to power it. "Anthropic's code generation was consistently impressive," he says. (Figma still uses OpenAI and Google models for other features.) Figma is one of many companies whose relationship with Claude was boosted by Anthropic's close partnership with Amazon</a> and its cloud-computing arm, AWS. Amazon has committed to invest $8 billion in Anthropic, and it has integrated Anthropic's models deeply into AWS, making it easy for customers to use Claude with their data. Given that AWS is the world's largest cloud provider, that has helped drive business to Anthropic. Anthropic has relationships with Google Cloud and Microsoft Azure too. And recently IBM, whose AI strategy had been built around open-source models, made an exception and struck a strategic partnership with Anthropic to integrate Claude into select products, even though Claude isn't open-source. Rob Thomas, IBM's chief commercial officer, says IBM was excited about Claude's ability to work with its proprietary libraries of coding data, particularly in older languages such as Java and COBOL. The Latin of programming languages, COBOL powers Big Blue's mainframes, which are still used in banking, insurance, health care, and the U.S. government. But skilled COBOL coders have largely retired. IBM has used Claude, in conjunction with other AI models, to create Project Bob, an agentic tool it plans to release in 2026 that can carry out various software tasks, including modernizing COBOL-written programs. If coding is the gateway drug for many Anthropic customers, a growing number are discovering Claude's uncanny abilities at other tasks. Novo Nordisk, the pharmaceutical giant best known these days for its blockbuster diabetes and weight-loss drug Ozempic, evaluated a host of AI models in an effort to reduce the time it takes to prepare the reams of paperwork involved in clinical trials. Waheed Jowiya, the company's digitalization strategy director, says Novo Nordisk built a system around Claude that has taken the time required to compile clinical trial reports down from 12 to 15 weeks to just 10 to 15 minutes. Microsoft, a major investor in OpenAI, had been using OpenAI's models exclusively to power its Copilot in office productivity software -- but it found Claude was better at handling Excel spreadsheets and PowerPoint presentations, and switched accordingly. Both Deloitte and Cognizant have adopted Claude companywide and are helping Anthropic co-sell Claude to their own clients -- another revenue-scaling opportunity, since big companies rely on such firms' consulting work to get value from generative AI. Anthropic has begun rolling out tailored versions of Claude for specific professions. But it's cautious about launching too many "verticals": Mike Krieger, the Instagram cofounder who is now Anthropic's chief product officer, says it will create tailored products only if they will help either solve some confounding aspect of general-purpose intelligence or create what he calls "a flywheel effect" that accelerates progress toward superhuman AI. Krieger says Claude Code checked the second box (offering the prospect of AI models writing code for future models). Claude for Financial Services, which launched in July, checked the first one, since building accurate financial models requires lots of reasoning steps. The company has a "frontier prototyping team" that builds internal products designed to push the envelope of what Claude can do, with an eye toward commercializing them if they succeed. For all its abilities, plenty remains beyond Claude's grasp. When Anthropic teamed up with Andon Labs, an AI safety testing outfit, to see if Claude Sonnet 3.7 could run the vending machines in Anthropic's San Francisco headquarters, it fared disastrously. The model failed to raise prices on in-demand items, told employees to remit payments through an account that didn't exist, offered all Anthropic staff a 25% discount (not realizing the impact that would have on profits in an office in which pretty much everyone worked for Anthropic), and decided to stock tungsten cubes, an expensive but useless novelty item. (Tungsten cubes briefly became an Anthropic office meme.) While Anthropic works to up Claude's vending machine game, its rivals aren't standing still. OpenAI is reportedly working on a product to directly challenge Claude for Financial Services. Its newest coding product, GPT-5 Codex, has narrowly bested Anthropic on some software development benchmarks. Google's new Gemini 2.5 Pro model also has decent coding skills and performs competitively with Claude on many reasoning tasks. Each of those models is considerably cheaper than Claude, and a number of Chinese AI companies have produced powerful coding models that they've released for free. Right now, most enterprises are willing to pay more for AI models to gain even a slight advantage in accuracy on essential tasks. But that could change as the gap between the performance of different AI models narrows. That means price could become Anthropic's Achilles' heel. IBM's Thomas says, "I don't think Bob would hit the mark for users if Anthropic wasn't there, but if we'd only built on Claude we'd probably miss the mark on price." In June, Anysphere, the startup behind the AI-powered software development platform Cursor, angered many users when it jacked up prices. Anysphere blamed the increase partly on Anthropic, because Cursor relies heavily on Claude under the hood. Around the same time, Anthropic reduced the number of requests its own paid subscribers could make for a given subscription tier -- in essence, a stealth price hike. Daniela Amodei acknowledges that Anthropic's price changes were not communicated well. But she adds that "pricing in the AI industry is like an acid trip," and that "everyone in the industry, including us, is still doing some form of price discovery, because it's just evolving so quickly." She also says that Anthropic has created smaller, less-expensive models, such as its Claude Haiku series, which perform certain tasks just as well as its massive Claude 4.1 Opus, at a fraction of the price. "Depending on the use case you might not need the Ferrari," she says. Left unsaid: If you do need the Ferrari, don't expect Chevy prices. Tense relationships If Anthropic's safety emphasis has won it customers, it's also alienated policymakers in Trump's Washington. The week Amodei and I meet, the company is scrambling to respond to a series of highly critical social media posts from White House AI and crypto czar David Sacks, who is also a prominent venture investor and podcaster. Sacks, who has repeatedly attacked the company for being "Trump haters" and a cog in the AI "doomer industrial complex," was exercised about remarks that Anthropic's cofounder and head of policy Jack Clark gave at an AI conference, where Clark likened AI models to mysterious, unpredictable, and at times scary creatures. Sacks accused Clark and Anthropic of engaging in a cynical attempt at "regulatory capture," playing up threats in order to drum up public support for rules with which Anthropic was best-positioned to comply. Other top White House figures with interest in tech, including Vice President JD Vance, have voiced skepticism of AI safety efforts, worrying that they will hobble U.S. efforts to compete with China. White House policymakers were also displeased that Anthropic endorsed California's new AI law, which requires labs building powerful AI models to disclose the actions they are taking to avert potentially catastrophic risks. The administration has advocated for a 10-year moratorium on state-level AI regulation. Dario Amodei was notably absent from a White House dinner in September attended by leaders of top U.S. AI and tech companies, nor was he among the tech CEOs accompanying the president on his state visit to the U.K. later that month. It's true that Amodei is not a fan of Trump. He once likened the president to a "feudal warlord" in a now deleted preelection Facebook post urging friends to vote for Kamala Harris. He also decided Anthropic would cut ties with two law firms that struck settlements with Trump. But Amodei insists the company has "lots of friends in the Trump administration" and is more aligned with the White House than Sacks and others give it credit for. He points, for example, to a shared belief that the U.S. must rapidly expand energy generation capacity to power new data centers. Amodei notes that he traveled to Pennsylvania to attend an energy and innovation summit where he met Trump. He also attended a dinner during Trump's recent state visit to Japan, where he again met the president. In a blog post widely interpreted as a response to Sacks' criticisms, Amodei went out of his way to say Anthropic concurred with Vance's recent remarks that AI will have both benefits and harms, and that U.S. policy should try to maximize the benefits and minimize the harms. These tensions haven't prevented Anthropic from winning multiple key government contracts. Most recently, in July, the Pentagon handed the company a $200 million, two-year contract to prototype "frontier AI capabilities" that would advance U.S. national security. But Amodei says he won't kowtow to the president. "The flip side of that is when we disagree, we're gonna say so," he says. "If we agreed with everything that some government official wanted us to, I'm sure that could benefit us in business in some way. But that's not what the company is about." As for the California AI law, Clark, the policy director, says Anthropic would prefer federal regulation, but that "the technology isn't sitting around waiting for a federal bill to get written." He says the California bill "was developed carefully and in a very consultative manner with industry and other actors." Clark also tells me that Anthropic has been testing Claude to weed out any political bias in the responses it gives to questions that involve ideological framing or policy positions more closely aligned with either major party. One area where Anthropic sees mostly eye to eye with the Trump administration is on restricting China's access to AI technology. But Amodei's advocacy for export controls has put it on a collision course with Nvidia's Huang. Huang has said he "disagrees with pretty much everything [Amodei] says"; he has also said that Anthropic's position is that AI is so dangerous, only Anthropic should build it. (Amodei has called Huang's comments "an outrageous lie.") Amodei tells me he has great respect for Huang and admires him for coming to America as an immigrant and pulling himself up by his bootstraps to create the world's most valuable company. "We always want to work with them; we always want to partner with them," he says of Nvidia. Comparing the race to create superpowerful AI to the Manhattan Project, Amodei says, "Just like we worry when an authoritarian government gets nuclear weapons, I think we should worry when they get powerful AI, and we should worry about them being ahead in powerful AI." The infrastructure race AI has increasingly become an infrastructure race, with companies like OpenAI, Meta, Elon Musk's xAI, Microsoft, Google, and Amazon announcing billions of dollars in spending on vast AI data centers that consume as much electricity as sizable American cities. Overall, the hyperscalers are expected to spend as much as $400 billion on AI infrastructure in 2025, with that figure ramping up to close to $800 billion by 2029, according to data from IDC. In many ways, Amodei himself helped create this race. In 2020, when he was still a senior researcher at OpenAI, he helped formulate what are known as the "AI scaling laws" -- an empirical observation that increasing an AI model's size, feeding it more data, and training it on more computing power produces a predictable gain in performance. Belief in these scaling laws has driven AI companies to build ever larger models and bigger data center clusters. Today, there's debate among AI researchers about the extent to which this premise still holds. But Amodei says he doesn't think scaling is ending. "We see things continuing to get better," he says. "Every three to four months, we release a new model, and it's a significant step up every time." Still, Amodei says observers shouldn't expect Anthropic to announce infrastructure deals of quite the same magnitude as OpenAI or Meta. A $50 billion deal Anthropic announced with cloud company Fluidstack in mid-November to build customized data centers for the company in Texas and New York is its largest to date. And it is unlikely to be its last. But, by comparison, OpenAI has announced multiple deals in the hundreds of billions. Daniela Amodei says that Anthropic has discovered ways to optimize model training and inference that wring more out of fewer AI chips. "Anthropic is a minor player, comparatively, in terms of our actual compute," she says. "How have we arguably been able to train the most powerful models? We are just much more efficient at how we use those resources." Leaked internal financial forecasts from Anthropic and OpenAI bear this out. Anthropic projects that between now and 2028 it will make 2.1 times more in revenue per dollar of computing cost than what OpenAI forecasts, according to a story in The Information that cited figures the companies shared with investors. And while the $78 billion Anthropic told investors it forecast spending on compute through 2028 under an optimistic scenario is a massive figure, it's only a third of the $235 billion OpenAI was budgeting over that time frame, according to information it had given its own investors. Like its competitors, Anthropic is turning to different partners for computing power. AWS has built Project Rainier, a network of data centers, including a gigantic $11 billion facility in rural Indiana that houses some 500,000 Trainium 2s -- Amazon's own AI chips -- for Anthropic to use to train and run its models. By year-end, Anthropic will have more than 1 million Trainium 2s at its disposal. Google, meanwhile, has invested $3 billion into Anthropic, and in October Anthropic said it would begin using 1 million of Google's specialized AI chips, called TPUs, in addition to Amazon's. Amodei acknowledges that the relationship with Google "is a little different" than Anthropic's with AWS, since Google's frontier AI model, Gemini, competes directly against Claude. "'Coopetition' is very, very common in this industry," Amodei adds, "so we've been able to make it work." Even as Anthropic keeps spending relatively restrained, the company has had to continually raise money. Press reports have circulated that it may be in the process of raising its third venture capital round in 18 months, even though it just completed a $13 billion fundraise in August. If it does raise again, the company would likely seek a valuation between $300 billion and $400 billion. This summer, Wired published a Slack message from Amodei in which he explained to employees why he was reluctantly seeking financing from Persian Gulf states. "'No bad person should ever profit from our success' is a pretty difficult principle to run a business on," Amodei wrote. Holding on to the culture Amodei's pained message points to one of the most pressing challenges facing Anthropic -- how to hold on to its "AI for the good of humanity" culture as its growth skyrockets. "I have probably been the leader who's been the most skeptical and scared of the rate at which we're growing," Daniela Amodei tells me. But she says she's been "continually, pleasantly surprised" that the company hasn't come apart at the seams, culturally or operationally. She says the fact that all seven cofounders still work at Anthropic helps, because it seeds cultural hearth-tenders across different parts of the company. She also says that the company's AI safety mission tends to draw a certain type of person. "We're like umami," she says. "We have a very distinct flavor. People who love our umami flavor are very attracted to Anthropic, and Anthropic is very attracted to those people." Anthropic's mission has also made it easier to retain talent at a time when Meta has reportedly been offering experienced AI researchers pay packages worth hundreds of millions of dollars. Dario reinforces the company's values at regular companywide addresses called DVQs -- short for "Dario Vision Quests." He uses the sessions to explain strategy and policy decisions but also Anthropic's mission. "When the company was small, we all had a common understanding of the potential of AI technology," he says. "And now a lot of people are coming in, so we have to impart that understanding." Both Dario and Daniela say they've had to stretch into their roles as senior executives as Anthropic has grown. Dario says he's had to remind himself to stop feeling bad when he doesn't recognize employees in the elevators or, as happened recently, when he discovers that Anthropic employs an entire five-person team that he didn't realize existed. "It's an inevitable part of growth," he concedes. When the company was smaller, Dario was directly involved in training Anthropic's models alongside head of research Jared Kaplan. "Now he's injecting high-level ideas, right?" Daniela says. "'We should be thinking more about x.' That's such a different way of leading." Daniela says she, too, has had to learn to be more hands-off. Before, when someone came to her with a problem, she would leap in, saying, "'I am going to help you figure it out.' Now I'm like, 'What is the one thing I want them to take back to their teams?'" The two siblings have also had to be intentional about separating work from family life. Daniela says Dario comes to her house most Sundays to hang out with her family. They'll play video games and play with her kids, but work talk is verboten. "This is a separate time that's just for us, because we were siblings before we were cofounders," she says. Dario Amodei tells me he remains convinced that AGI -- humanlike artificial general intelligence -- and then AI superintelligence loom on the horizon. And he denies being a "doomer." Yes, he's worried about potential dangers, from models that will make it easier for someone to engineer a bioweapon to large-scale job displacement. But he thinks AGI will help cure many diseases -- and wants those cures to arrive as soon as possible. And he firmly believes AI could supercharge the economy. "The GDP growth is going to inflect upwards quite a lot, if we get this right," he says. Another thing he's optimistic about: Anthropic's continued revenue acceleration. He's a scientist. He gets the law of big numbers. Companies don't keep growing at 10x for long. "I'm an AI optimist, but I'm not that crazy," he says. Still, he thinks Anthropic could surpass OpenAI as the world's largest AI company by revenue. "I would argue it's maybe even the most likely world in which our revenue passes theirs a year from now," he says. Then he pauses before adding, "I think I'd rather have the largest revenue than the largest data center, because one is black [on an income statement], and the other is red. Again, things I've had to learn about business: It's better to make money than just to lose money." This article appears in the December 2025/January 2026 issue of Fortune with the headline "Anthropic is still hung up on 'AI safety.' Turns out big business loves that." Anthropic by the numbers The AI startup is growing rapidly, and its path to profit looks shorter than that of some rivals