Trump administration reverses course on AI oversight after Anthropic's Claude Mythos concerns

Everything that could go wrong with Trump's AI safety tests, according to experts

This week, the Trump administration backpedaled and signed agreements with Google DeepMind, Microsoft, and xAI to run government safety checks on the firms' frontier AI models before and after their release. Previously, Donald Trump had stubbornly cast aside the Biden-era policy, dismissing the need for voluntary safety checks as overregulation blocking unbridled innovation. Soon after taking office, he took the extra step of rebranding the US AI Safety Institute to the Center for AI Standards and Innovation (CAISI), removing "safety" from the name in a pointed jab at Joe Biden. But after Anthropic announced that it would be too risky to release its latest Claude Mythos model -- fearing that bad actors might exploit its advanced cybersecurity capabilities -- Trump's suddenly concerned about AI safety. According to White House National Economic Council Director Kevin Hassett, Trump may soon issue an executive order mandating government testing of advanced AI systems prior to release, Fortune reported. In CAISI's press release, the center acknowledges that the voluntary agreements signed by Google, Microsoft, and xAI "build on" Biden's policy. Celebrating the new partnerships, CAISI Director, Chris Fall, did not mention Mythos but promised that the "expanded industry collaborations" would help CAISI scale its work "in the public interest at a critical moment." "Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications," Fall said. To date, CAISI said it has completed about 40 evaluations, including those of frontier models that have yet to be released. When conducting tests, CAISI frequently gains access to models with "reduced or removed safeguards," which CAISI said allowed them to more "thoroughly evaluate national security-related capabilities and risks." Through the evaluations, the government will also gain a better understanding of model capabilities, CAISI claimed. And to ensure that evaluators understand top national security concerns as they emerge across government, a "group of interagency experts" has formed a task force "focused on AI national security concerns," CAISI said. Some firms that have signed agreements have signaled confidence in CAISI's testing plans. On LinkedIn, Tom Lue, Google DeepMind's vice president of frontier AI global affairs, said he was "pleased" with CAISI's testing plans. In a blog, Microsoft said that "testing for national security and large-scale public safety risks necessarily must be a collaborative endeavor with governments, while crediting the expertise "uniquely held by institutions like CAISI" to conduct such testing. xAI, which is currently fighting against OpenAI in a trial over which firm's leaders care more about AI safety, did not immediately respond to Ars' request to comment. However, critics aren't sold on the government's plan to vet models and are increasingly dubious of firms whose AI model designs are largely kept secret. Critics suggested that CAISI may lack the funding or expertise to evaluate frontier AI models. And as Trump seemingly suspects, seeking voluntary commitments from AI firms may not create the kind of day-to-day transparency the public needs about frontier AI risks, critics have warned. Further, any politicization of the evaluation process -- like opposing the release of models whose outputs disfavor a certain administration's political views -- could decrease trust in AI. Unchecked, that could ultimately dissuade firms from signing agreements, since increasing trust is supposedly a key motivator driving the latest attempt at government collaboration. Nobody knows what "safe" means In its rush to announce its partners, CAISI did not specify the testing standards that will be used for evaluations. That could be a problem, according to a LinkedIn post from Devin Lynch, a former director for cyber policy and strategy implementation at the White House Office of the National Cyber Director: "Pre-deployment evaluations with frontier labs are exactly the kind of public-private collaboration needed to build trust, safety, and security into AI. The harder question is what 'evaluation' actually means at the frontier. Capability assessments are only as good as the threat models behind them. Our research on the AI tech stack finds that the Governance layer -- standards, audits, liability frameworks -- remains the least mature but most essential. CAISI will need to define, and publish, what it's testing for, not just who it's testing with." In a statement provided to Ars, Sarah Kreps, director of the Tech Policy Institute at Cornell University, said that AI firms should be developing closer ties with the government as AI advances. However, "the definition of 'safe' is contested" and "once you build a government vetting process for technology, you get the good with the bad," she said. Without defining standards, "the process can be politicized," Kreps said. That risks creating a system where "whoever holds power gets to shape how the vetting works." So far, neither the Biden nor the Trump administrations has figured out how to avoid that, Kreps said. Fears of government controlling AI outputs Microsoft's blog said that "CAISI, Microsoft and NIST will collaborate on improving methodologies for adversarial assessments," which suggests that the plan is to develop these standards on the fly. According to Microsoft, "testing AI systems in ways that probe unexpected behaviors, misuse pathways, and failure modes" is "much like stress-testing whether airbags, seatbelts, and braking systems work effectively and reliably in safety-critical driving scenarios." But Gregory Falco, a Cornell University assistant professor of mechanical and aerospace engineering and expert in tracking governance of AI, insists that there's a better way. "Government oversight of AI cannot simply mean political review of model outputs, nor should it become a mechanism for deciding whether a model says favorable or unfavorable things about a president or administration," Falco said. Rather than relying on a politicized government leveraging evaluations to control the AI systems that the public uses, the US could build "some form of independent audit," Falco said. Imagine, Falco suggests, if AI firms understood that their models could be audited at any point, how much more accountability and discipline might such a system create? Operating similarly to the Internal Revenue Service (IRS), a rigorous AI audit system could create "real consequences for reckless deployments," Falco said. For AI firms facing such consequences, the pressure would be on to ramp up internal AI safety testing, Falco suggested. That seems like the "only viable path," Falco said, since "the federal government does not currently have the in-house technical expertise, infrastructure, or day-to-day insight needed to directly evaluate these systems on its own." Rumman Chowdhury, an AI governance consultant and founder of Humane Intelligence, similarly criticized CAISI's preparedness. Chowdhury told Fortune that "current White House efforts to offer 'sensible oversight' over frontier AI models may sound good, but the devil is in the details." "It depends on their interpretation of these words," Chowdhury said. "Evaluations are a policy tool, they are not actually data-driven. My concern is that this is another political tool that the administration wants to own and wield." CAISI may lack funding As for funding, Congress in January approved up to $10 million to expand CAISI, Fortune reported. However, conservative think tank America First Policy Institute conducted a recent analysis finding that "CAISI remains underfunded compared with peer institutes internationally and lacks 'appropriate funding.'" To critics, the CAISI testing plan may not go far enough to protect the public from the most unforeseeable AI risks. Falco maintains that only independent audits can spare the public from the worst outcomes. "The danger is that government oversight becomes political, performative, or captured by the companies it is supposed to evaluate," Falco said. "The opportunity is to build a practical audit framework that lets the US remain the global leader in AI while creating credible accountability around the most consequential risks." To Lynch, the bigger test may be whether Trump's testing plan succeeds in its mission to evade risks and stoke more trust in AI systems, while keeping a light touch to avoid overregulating firms. CAISI "is building something important here," Lynch said. "The test will be whether these collaborations ignite innovation, protect national security, and produce AI that is both trusted and trustworthy."

The death of AI idealism

Why it matters: OpenAI and Anthropic were founded on the idea that AI would be deployed in ways that prioritized safety and the public good. Now those principles are giving way to an arms race for market share, as those companies and others release ever more powerful models. The big picture: The men behind today's biggest AI labs often pitched themselves as a safer, less-greedy alternative to earlier tech leaders. * Acknowledging the breathtaking power of AI, they first rejected Silicon Valley's "move fast and break things" ethos. * Now, AI behemoths are locked in an escalating competition for enterprise, consumer and government business. * When the Pentagon blacklisted Anthropic because it wanted to restrict how its AI could be used -- including for mass surveillance and fully autonomous weapons -- rivals swooped in and agreed to the "all lawful use" terms Anthropic had rejected. * Meanwhile, just last week the Pentagon reached an agreement allowing Google's Gemini models to be used for "any lawful government purpose," Axios' Maria Curi confirmed. Flashback: Altman and Musk co-founded OpenAI in large part out of a desire to develop artificial general intelligence before Google and its AI chief Demis Hassabis. * Musk was obsessed with the idea of Hassabis and his corporate bosses dominating the world's most powerful technology. * Hassabis, for his part, was focused more on AI's potential to cure diseases and power new scientific discoveries. Zoom in: Musk's court case centers on his argument that Altman and OpenAI president Greg Brockman should not be trusted with a for-profit AI company. * One big problem: Musk runs xAI, his own for-profit OpenAI rival. His argument asks jurors to distrust OpenAI's profit motive while overlooking his own. * "I suspect that there are a number of people who do not want to put the future of humanity in Mr. Musk's hands," U.S. District Judge Yvonne Gonzalez Rogers told the trial's lawyers. The case also hinges on the belief that AI is, in fact, a danger to humanity. * Musk used his first two days of testimony in Oakland to repeat his fears that AI could kill us all. * On his third day, Judge Gonzalez Rogers cut off that line of argument, warning that AI catastrophe and extinction were outside the scope of the case. Context: Anthropic CEO Dario Amodei straddles both visions of AI, touting his startup as a safer version of what came before while also warning AI could wipe out half of all entry-level white-collar jobs. He called AI a "serious civilizational challenge" that will "test who we are as a species." * Nvidia CEO Jensen Huang recently argued that these apocalyptic warnings are themselves dangerous, saying the AI CEOs who use them (presumably Amodei) have "a god complex." Driving the news: In testimony Monday, OpenAI president Greg Brockman acknowledged that he helped launch OpenAI as a nonprofit AI lab and agreed with its original promise to advance AI "to benefit humanity as a whole," free from the need to generate financial returns. * He also acknowledged that his stake in OpenAI's for-profit arm may now be worth more than $20 billion, perhaps closer to $30 billion. The latest: The New York Times reported Monday that the Trump administration -- which has taken a laissez-faire approach to regulating AI -- is considering new oversight. * Per the report, the White House is considering creating a working group of tech execs and government officials to vet the safety of new AI models before they're publicly released. * Axios reported other details of the emerging plan. What we're watching: Testimony in the Musk trial continues this week to determine if OpenAI's change in structure comprised its original mission or preserved it. Bottom line: It's all a far cry from the do-good idealism AI's founders once prided themselves on.

Trump administration suddenly embraces AI oversight ideas it once rejected | Fortune

When it comes to AI, the Trump Administration has largely positioned itself as the opposite of the Biden White House -- criticizing what Trump's tech policy advisors saw as overly burdensome AI safety efforts and licensing regimes, and embracing an anti-regulation approach. Former Trump "AI and crypto czar" David Sacks best embodied this policy ethos. But the Trump Administration, according to multiple news reports, is now about to engage in a head-spinning policy pirouette. Driven by concerns about the national security implications of Anthropic's new "Mythos" AI model, with its ability to identify and exploit cyber security vulnerabilities -- as well as broader fears around cyber capabilities and dangerous misuse -- the administration is now reportedly considering oversight for advanced AI models. The policies under discussion, according to news reports, include an executive order that would create a government-industry working group to examine how frontier AI systems should be evaluated before release. At the same time, the Center for AI Standards and Innovation (CAISI) -- the Trump administration's renamed version of the Biden-era United States AI Safety Institute -- announced partnerships with Google, Microsoft, and xAI to evaluate some AI models before deployment. According to an agency press release, CAISI's agreements with frontier AI developers "enable government evaluation of AI models before they are publicly available, as well as post-deployment assessment and other research." The agency said it has completed more than 40 such evaluations, including on state-of-the-art models that remain unreleased. In an interview on Fox Business this morning, White House National Economic Council Director Kevin Hassett said the administration is studying a possible executive order that would create "a clear road map" for how advanced AI systems should be evaluated before release. "We're studying possibly an executive order to give a clear road map to everybody about how this is going to go and how future AIs that also could potentially create vulnerabilities should go through a process so that they're released to the wild after they've been proven safe -- just like an FDA drug," Hassett said. "Mythos is the first, but it's incumbent on us to build a system so U.S. AI can be the leader in AI and be safe at the same time. That's really pretty much what we're working on almost full-time right now." The current debate carries with it a strong sense of déjà vu. The original U.S. AI Safety Institute was created by Joe Biden through his November 2023 AI Executive Order, with the goal of helping the federal government evaluate and better understand frontier AI systems from companies like OpenAI, Anthropic, and Google. The order also invoked the Defense Production Act to require companies training the largest AI models to share certain safety testing results with the government. In other words, the administration that once criticized Biden's AI oversight efforts is now considering adopting broadly similar policies, even though the original U.S. AI Safety Institute was systematically rebranded and restructured (the word "safety" was notably removed) and its inaugural director, Elizabeth Kelly, stepped down shortly after Trump's inauguration in January 2025. (She subsequently joined Anthropic as head of "beneficial deployments," one of several hires of former Biden officials that may have contributed to the acrimonious relationship between Trump's tech policy team and Anthropic.) At the end of April, Chris Fall, who served as an Energy Department official in the first Trump administration, was tapped to lead the rebranded CAISI, with a Commerce Department spokesperson saying "Dr. Fall brings the scientific leadership needed to ensure America leads the world in evaluating frontier AI models and advancing the technical standards that protect our national and economic security." Fall replaced Collin Burns, a former member of Anthropic's technical staff, who was dismissed from his position after just days on the job, with unnamed Trump administration officials telling reporters that they had not been informed of Burns' appointment. Fall spent nearly four years as vice president for applied sciences at technology research nonprofit MITRE. "The is a 180 for the Trump administration, that has very explicitly been anti-any sort of regulation and also has explicitly tried to block states from enacting any kind of regulation," said Rumman Chowdhury, an CEO of Humane Intelligence and former US Science Envoy for AI. Still, the renewed push for evaluations is being framed less around AI ethics concerns and worry about existential dangers, which was a strong focus of the Biden Administration, and more around immediate national security risks. That backdrop includes the uproar over Anthropic's Mythos model and a broader shift in Washington toward viewing frontier AI systems through the lens of cyberwarfare, infrastructure security, and geopolitical competition. Anthropic itself was labeled a national security threat by the administration after refusing to grant the Pentagon unrestricted use of its technology -- a designation the company is now challenging in court. Trump recently struck a more conciliatory tone, telling CNBC that Anthropic was "shaping up" and that "I think we will get along with them just fine." Chowdhury said the current White House efforts to offer "sensible oversight" over frontier AI models may sound good, but the devil is in the details. "It depends on their interpretation of these words," she said. "Evaluations are a policy tool, they are not actually data-driven. My concern is that this is another political tool that the administration wants to own and wield." But it remains unclear whether CAISI has the funding and authority needed to fulfill its mission. In 2024, The Washington Post published an investigation into National Institute of Standards and Technology (NIST), the agency that houses CAISI, finding that budget constraints had left the 123-year-old institution understaffed in key technology areas and many facilities at its Gaithersburg, Maryland, and Boulder, Colorado campuses below acceptable building standards. At the time, now Senate minority leader Chuck Schumer had announced that an appropriations bill included up to $10 million for the establishment of the USAISI at NIST. In January 2026, Congress approved funding increases for NIST's AI work including $55 million for NIST AI research and measurement efforts and up to $10 million specifically to expand the agency, rebranded as CAISI. But one policy analysis this year, from conservative think tank America First Policy Institute, said CAISI remains underfunded compared with peer institutes internationally and lacks "appropriate funding." The challenge is compounded by the fact that much of the government's evaluation effort depends on cooperation from the same companies building the models. "In 2024, BIML identified 23 LLM security risks that are located inside the black box of the frontier models (and thus managed by the vendors themselves)," Gary McGraw, CEO of the AI security nonprofit Berryville Institute of Machine Learning (BIML), said in an email to Fortune. "In our view, any regulatory guidance should systematically address these risks by opening the black box to scrutiny." McGraw added that BIML is "deeply concerned that the foxes might be asked to guard the chicken house even though they already designed and constructed it in secret." In addition, while AI model vetting is useful, it should not be mistaken for AI system security, said Rob van der Veer, founder of the the OWASP (Open Worldwide Application Security Project) AI Exchange and chief AI officer at global technology consultancy Software Improvement Group. "AI model vetting can motivate model makers to invest more in resilience, and it can help expose obvious weaknesses," he said by email. "But AI models will remain fragile, no matter how much we test them...so yes, test the models. Vet them. Improve them. But design the system as if the model can still fail. Because it can."

The Elon Musk-OpenAI trial is producing more heat than light in the debate over who should control AI | Fortune

Hello and welcome to Eye on AI...In this edition: Sparks fly as Musk and Brockman testify in battle over OpenAI's restructuring...the White House does a 180 degree U-turn on AI regulation and may begin reviewing AI models prior to release...OpenAI and Anthropic both target PE-backed companies with new joint ventures...a breakthrough in a foundation model for robotics...AI scientists may still be a ways off. People in Silicon Valley and far beyond have been enthralled by the drama playing out in a courtroom in Oakland, California, where a jury is currently hearing testimony in Elon Musk's lawsuit against OpenAI cofounders Sam Altman and Greg Brockman. The judge and jurors in the case (the jury's verdict is merely advisory) will need to decide whether Altman's and Brockman's communications with Musk around the formation of OpenAI established a formal "charitable trust" and whether Altman and Brockman subsequently violated that trust when they restructured OpenAI so that its non-profit board no longer had sole control over its for-profit arm. They will also have to decide on Musk's allegations that Altman and Brockman unjustly enriched themselves as OpenAI re-oriented from a research-oriented lab to being primarily a commercial entity. Most legal analysts say Musk's case is weak and that he's likely to lose. In fact, I'm surprised the case has even come to trial. I thought that Musk would opt to settle at the last minute. I had long-assumed that this was one of those legal cases where the lawsuit itself was the whole point, not whether Musk ultimately prevailed. I thought his intention was two-fold: 1) to sow enough investor doubt about the viability of OpenAI's new for-profit company structure to make it harder for OpenAI to raise further investment and possibly go for an IPO and 2) to use the discovery process to surface lots of embarrassing emails, internal documents, and details about Altman, Brockman, and the constant drama at OpenAI that would tarnish the reputation of his former cofounders. So far, it's not clear the litigation has had much impact on OpenAI's ability to continue to raise money. It has held several successful funding rounds since Musk filed his suit, including an additional $122 billion fundraise at a $852 billion valuation that closed in March. An IPO still appears to be on the cards -- and to the extent that it is looking shaky, it has nothing to do with Musk's lawsuit. But plenty of documents have emerged that paint Altman and Brockman in a less than flattering light and those documents have helped feed lots of media coverage about internal strife at OpenAI. So you might think Musk would say: blows landed, mission accomplished, time to cut bait. Yet Musk apparently thought there was more potential to damage that could be done by going to trial. We know this because Musk said so explicitly in an email to Brockman on the eve of the trial -- an email that OpenAI's lawyers made public on Sunday and tried, unsuccessfully, to have admitted into evidence. According to OpenAI's lawyers, Musk reached out to Brockman about discussing a settlement of the case in the week before the trial. Brockman suggested that both sides drop their respective claims (OpenAI has counter-sued Musk claiming harassment.) Musk wrote back that "By the end of this week, you and Sam will be the most hated men in America. If you insist, so it will be." The email was a spectacular moment in a trial that has, so far, resulted in few bombshell revelations on the witness stand. That's because much of the sensational stuff has already been disclosed in the documents that surfaced through the pre-trial discovery process. Hearing those details repeated on the stand doesn't change the public narrative much. There have been a couple of wowzer moments though: One was Musk's admission that his AI company, xAI, had trained its Grok model in part by 'distilling' OpenAI's GPT models. Distillation is the process of training a model on the answers from another model. This tactic violates OpenAI's terms of service, so it is likely that this was done using fake or fraudulent OpenAI accounts, and Musk's admission to this conduct was something of a bombshell. Musk's excuse was essentially "everyone does it." The other startling moments so far came in Monday's testimony from Brockman, which included a number of potentially damaging moments. Brockman acknowledged he never followed through on his own initial pledge to donate $100,000 to OpenAI's non-profit when it was set up, but now has a stake in the for-profit company worth $30 billion. Musk's lawyers also questioned Brockman about his own journal entries from November 2017 in which he wrote about being "warm to steal the nonprofit from [Musk] to convert to b corp without him." He also wrote, "[Musk's] story will correctly be that we weren't honest with him in the end about still wanting to do for profit just without him." Brockman's words may prove damning, since they seem to confirm some of the key allegations Musk makes in his suit. So too may be Brockman's admission that he was an investor in the AI chip startup Cerebras at the time OpenAI was discussing a potential acquisition of the company and that he never disclosed his investment to Musk. Altman was also a Cerebras investor. That may help Musk's attorneys make the case for unjust enrichment although the merger proposal did not go ahead. (OpenAI did later sign a major partnership with Cerebras that significantly boosted the chip startup's valuation.) Still, it's far from certain Musk will prevail, either legally, or in shifting public opinion against his one-time-cofounders-turned-bitter-rivals, Brockman and Altman. In many ways, the trial is a distraction, generating much more heat than it is shedding light on the bigger concerns about who controls AI and the risks the technology presents. While the Musk-OpenAI courtroom showdown has been billed as the first great technology trial of the AI era, a legal showdown that matters far more will take place two weeks from now in a courtroom in Washington, D.C. That's when a federal appeals court panel will hear arguments in Anthropic's challenge to the 'supply chain risk' designation the Trump Administration slapped on it for refusing to agree to its specified contract terms for providing its AI models to the U.S. military. That's a case with huge implications not just for Anthropic and the fate of the AI industry, but also for the balance of power between the state and industry more generally. Even as that case moves forward, the ground is shifting in D.C. Anthropic's Mythos model, with its powerful cyber capabilities, combined with growing public fears about AI technology, seem to have convinced the Trump administration to perform a head-spinning U-turn: moving from a highly-laissez faire approach to AI to a mandate that the government receive early access to AI models and essentially license their release to the wider public. (More on that in the news section below.) This policy reversal may not have the drama of a trial, but it matters far more for the shape of AI development. But before we get to the news: Do you want to learn more about how AI is likely to reshape your industry? Do you want to hear insights from some of tech's savviest executives and mingle with some of the best investors, thinkers, and builders in Silicon Valley and beyond? Do you like fly fishing or hiking? Well, then come join me and my fellow Fortune Tech co-chairs in Aspen, Colo., for Fortune Brainstorm Tech, the year's best technology conference. And this year will be even more special because we are celebrating the 25th anniversary of the conference's founding. We will hear from CEOs such as Carol Tomé from UPS, Snowflake CEO Sridhar Ramaswamy, Anduril CEO Brian Schimpf, Yahoo! CEO Jim Lanzone, and many more. There are AI aces like Boris Cherny, who heads Claude Code at Anthropic, and Sara Hooker, who is cofounder and CEO of Adaption Labs. And there are tech luminaries such as Steve Case and Meg Whitman. And you, of course! Apply to attend here. UK-based Google DeepMind workers vote to unionize over military AI contracts amid internal backlash over its Pentagon deal -- by Beatrice Nolan Employee revolt once forced Google to back off on military contracts. But, in the wake of a new Pentagon AI contract, their leverage appears limited -- by Beatrice Nolan A decade after the 'Godfather of AI' said radiologists were obsolete, their salaries are up to $571K and demand is growing fast -- by Marco Quiroz-Gutierrez White House looks to control access to advanced AI models. The Trump administration -- which spent the past year tearing up the Biden-era AI rulebook -- is now weighing an executive order to convene a working group of tech executives and officials to design frontier-model oversight, with a formal pre-release review process reportedly among the options on the table, the New York Times reports citing sources familiar with the deliberations. White House officials briefed Anthropic, Google and OpenAI on the plans last week, and some inside the administration are pushing for a system that would give the government first access to new models but without the ability to block their release. The abrupt policy shift has been driven in part by Anthropic's Mythos model, whose cyber-vulnerability discovery capabilities prompted the company to withhold a public release, and by mounting bipartisan public concern about AI's impact on jobs, energy, education and mental health. It also tracks a leadership change at the West Wing: AI czar David Sacks departed in March, and Chief of Staff Susie Wiles and Treasury Secretary Scott Bessent -- who recently held a "productive" meeting with Dario Amodei aimed at thawing the Pentagon-Anthropic standoff -- have stepped in to shape policy. Meanwhile, the Wall Street Journal reports that Google, Microsoft, and xAI have already agreed to give early access to their advanced models to the U.S. government. It also reported previously that the White House has opposed Anthropic sharing Mythos with more companies to help them safeguard their systems -- although it is unclear if this is because it fears that sharing the model more widely will increase the chance it will wind up in the hands of bad actors or because it wants to hoard Mythos' potential offensive cyber capabilities for itself and doesn't want more companies using it to harden their defenses. OpenAI and Anthropic both set up companies to push AI into private equity-backed companies. The two AI rivals unveiled competing joint ventures within minutes of each other on Monday, both designed to push their AI tools deep into the operations of private equity-backed companies. OpenAI's "Deployment Company" drew more than $4 billion from 19 investors -- led by TPG, Brookfield Asset Management, Advent and Bain Capital, with Dragoneer and SoftBank also participating -- at a $10 billion valuation, with OpenAI itself contributing capital and retaining majority control. The PE backers were, according to press reports citing leaked documents, offered a 17.5% guaranteed annual return floor over five years. Anthropic's $1.5 billion vehicle, by contrast, is anchored by Blackstone, Hellman & Friedman and Goldman Sachs -- with General Atlantic, Leonard Green, Apollo, GIC and Sequoia also backing it. It is targeting mid-sized businesses, and will see "forward-deployed engineers" sent to implement Anthropic's AI models inside those companies. You can read more from the Wall Street Journal here and Bloomberg here. Anthropic announces new financial services agents. The company debuted 10 new AI agents built for banks and financial services firms -- handling tasks like building pitchbooks, closing the books, and drafting credit memos -- as it deepens its push into a sector that's central to its enterprise strategy ahead of an anticipated IPO this year. Anthropic's arch rival OpenAI has also been targeting financial services use cases, but the new roll out also puts Anthropic in more direct competition with vendors like Microsoft and Salesforce, as well as specialist financial data providers such as Bloomberg and Alpha Sense. Read more from the Wall Street Journal here. SAP moves to stop OpenClaw and other third-party agents from using its software. SAP last month told customers it could throttle, suspend or terminate access for those using unauthorized external AI agents to pull data from its apps -- an escalation in the brewing data wars between incumbent enterprise software vendors and vendors of AI tools, the Information reports. SAP has its own AI agent called Joule, but many customers prefer the functionality that third-party agents have to handle workflows across many different software applications. SAP CEO Christian Klein framed the move as protection against "mass data requests" that strain performance and as a defense of SAP's proprietary semantic models, but the policy lands amid clear signs of pressure: SAP shares are down roughly 28% this year and longtime customer Mercedes-Benz has cut its SAP instances by 40% in recent months while leaning on its own and frontier-lab AI models to clean and analyze data. SAP says it already permits agents from some other companies, including Microsoft, Google, Amazon and IBM, and hinted at "agentic integration architectures" with Anthropic -- suggesting Claude Code or Cowork access may be close -- while singling out open-source harnesses like OpenClaw as a security risk. SAP's stance mirrors that of Workday, Salesforce and ServiceNow, which have all made moves to erect some form of tollgates around their data. OpenAI changes privacy policy to share user data with advertisers. OpenAI updated its U.S. privacy policy on April 30 to allow the use of cookies and limited identifiers (like email addresses or cookie IDs) to promote its products on third-party websites and measure ad effectiveness, Wired reported. The company has said, however, that ChatGPT conversations remain private and aren't shared with marketing partners. Wired found that this marketing tracking was enabled by default for free accounts but off by default for Plus and Enterprise subscribers, with users able to opt out by changing a toggle in account settings. The change comes as OpenAI expands its own in-product advertising (rolling out ads beneath ChatGPT outputs in February) and prepares for a potential IPO later this year, with the off-platform ads aimed largely at converting free users into paying subscribers. Foundation models for robotics makes a big leap forward. Physical Intelligence, a San Francisco-based company with some pedigreed cofounders (ex-Google DeepMind and both Stanford and UC Berkeley robotics profs) that builds foundation models for robotics, achieved a breakthrough with a new foundation model called π0.7. The model can recombine learned skills to handle new situations, something large language models can do, but which has proved elusive in physical AI. A single π0.7 model can fold laundry, operate an espresso machine, peel vegetables, and take out the trash without any task-specific fine-tuning, matching the performance of specialized models trained for each individual task. More striking, π0.7 showed that it could transfer those skills between different brands and types of robots without additional training -- although here the performance only matched that of a human operator who had never done the task before operating the robot by remote control. The team also showed it can be "coached" through entirely new multi-stage tasks, such as loading a sweet potato into an air fryer, using only verbal step-by-step instructions. All of this is a pretty big deal that will make it far easier for more companies to begin to deploy robots in more settings far faster than before. One of the big breakthroughs that Physical Intelligence made was in what they call "diverse context conditioning" -- training the model not just on what to do but on rich metadata describing how each demonstration went, including quality scores, speed, mistakes, and AI-generated images of intermediate subgoals. The meta data labels seem to be key, helping the model learn which intermediate actions were most likely to result in success. You can read the research paper here on arxiv.org and see the company's blog on π0.7 here. July 6-11: International Conference on Machine Learning (ICML), Seoul, South Korea. Maybe AI scientists aren't so close after all. There's been a lot of hype recently about how fast AI scientists are coming along and that AI models will soon be able to automate scientific research. AI research itself certainly seems on the cusp of automation with AI, and there have been promising experiments in other fields, such as drug discovery and material discovery. But researchers from Germany's Friedrich Schiller University Jena and the Indian Institute of Technology Delhi found that large language models (they tested OpenAI's GPT-4o and GPT-OSS, as well as Anthropic's Claude Sonnet 4.5) that have not been specifically trained to act as AI scientists, can produce scientific results that seem superficially valid but actually lack key evidence and reasoning steps. The results are actually pretty abysmal. Hypotheses were stated but left untested by experiments in 63% of cases. In 68% of cases, the models failed to incorporate available experimental evidence into their process. In 71% of reasoning traces, the models' hypotheses are not updated in the face of counter-evidence. Only 26% of reasoning traces showed any belief revision based on new evidence from experiments. Using multiple experiments and independent lines of evidence to bear on a single hypothesis occurred in less than 10% of cases. Results like these make it seem like scientists' jobs will be safe for quite a while longer than some AI boosters claim. You can read the research here. AI is becoming an even more useful -- and dangerous -- tool as it gets smarter. Fortune AI Editor Jeremy Kahn breaks down best practices for deploying AI agents, how to protect your data from AI-powered cyberattacks, and just how smart AI can really get. Watch the playbook.

After India, Is the Trump Regime Taking a Relook at Its Hands-off Approach Towards AI?

India could take a leaf off President Trump's purported move to set up an oversight process for all new AI models Ever since artificial intelligence (AI) appeared in its current avatar, there has been a vertical division on building guardrails around it, with little or no consensus between the warring opinions. However, in the wake of the recent Claude Mythos circus orchestrated by Anthropic, policymakers are discussing a hands-on policy approach to this fast-growing ecosystem India recently announced that they are actively considering a prescriptive approach towards future AI regulations and now the world's biggest free market protagonist Donald Trump seems to be having second thoughts on allowing a free rein to this technology. A report in The New York Times said the White House was studying a proposal of actively vetting AI models before their release. A day after this report, the US Department of Commerce announced new agreements with Google DeepMind, Microsoft and xAI to conduct pre-deployment evaluations and targeted research to better assess frontier AI capabilities and advance the state of AI security. These agreements are based on tie-ups announced earlier and renegotiated following the concerns around Claude Mythos. "Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications," says Chris Fall, Director of the Centre for AI Standards and Innovation (CAISI), housed within the Department of Commerce's National Institute of Standards. "These expanded industry collaborations help us scale our work in the public interest at a critical moment," he says. Per a Department of Commerce statement Secretary Howard Lutnick would oversee the functioning of CAISI as the industry's primary point of contact within the US government to facilitate testing, collaborative research and best practice development related to commercial AI systems. CAISI's agreements with frontier AI developers enable government evaluation of AI models before they are publicly available, as well as post-deployment assessment and other research. To date, they have completed over 40 evaluations, including on state-of-the-art models that remain unreleased, says the official statement. The report in the NYT also suggests that the Trump administration is debating an executive order to create an AI working group that would bring together tech executives and government officials to examine potential oversight procedures. India already has a 10-member inter-ministerial AI governance and economic group (AIGEG) and a six-member tech experts panel doing this job. For the most part of his administration, Trump had given Silicon Valley a free rein around AI which eventually resulted in states passing their own legislation and creating a direct face-off with the White House. However, in recent times, the growing number of lawsuits over improper functioning of AI and its resultant tragic impact on human lives seems to have prompted the change of heart. Or, it could also simply be Trump's ego battle with Anthropic's Dario Amodei whom he had ejected from all government business following the latter's refusal to permit The Pentagon to use their AI models for largescale domestic surveillance and in autonomous weapons of war. Out went Anthropic and in came arch rival OpenAI which claimed it swung a better deal and was richer by $200 million. A few months later. when Anthropic aborted the release of its Claude Mythos model due to cybersecurity considerations last month, it was felt that the company had smoked the peace pipe within the White House. Top officials took the offer from Anthropic to run the Mythos model within their systems and check for cybersecurity flaws. Now, the administration seems to be taking a more rigid stance and some believe that have already discussed their plans with executives of Anthropic, Google and OpenAI. Of course, there is still no clarity around how the working group proposes to handle the oversight process itself. The NYT report claimed it could mirror the one being developed in Britain were government bodies are assigned to ensure that the AI models meet pre-defined safety standards. It is quite obvious that the move is a reversal of what the Trump administration stood for when it came to conversations about regulations muzzling the growth of AI. In fact, Trump wanted an AI edge over China and even rolled back Biden era regulations that required AI developers to run safety evaluations and report potential military uses. Back in July last year, Trump had exclaimed that the beautiful baby (AI) has been born and "we have to grow that baby and let that baby thrive. We can't stop it... not with politics. We can't stop it with foolish rules and even stupid rules," he had thundered during an AI event back then. However, it looks like it's again TACO Tuesday time and Trump Has Chickened Out. Of course, in this instant many experts are happy that the non-interventionist nonsense has been put to bed. Quite clearly Trump's White House does not want any blame for any possible AI-enabled cyberattack on the United States. At the same time, he also wants to lay his hands on any technology that enhances the cyber-capabilities of the US Intelligence agencies. So, there's a line of thought that in order for Washington to be able to use models like Mythos, they must have a review system in place that gives first access to the administration to all future AI models. However, the caveat would be that the government would never block their release. Heard of wine tasting? Well, this is a form of AI tasting!! Whatever be the case, it looks like those that sought to keep all AI development free are either having a change of heart or have since been replaced by those that believe in building guardrails in parallel to the AI development and not after. Now all that the Trump regime needs is a clear idea on who or which national agency holds point in this exercise. And this is precisely where the Narendra Modi government has an edge. For they have already created the AIGEG. Now, all that the policymakers, the experts and the bureaucrats have to do is frame up the guidelines, get parliamentary approval and get cracking. It is time, India paces up its Atmanirbhar tune and expands the chorus of sovereign AI.