Trump Administration Expands AI Safety Framework to Include Open AI Models Amid China Competition

Trump Administration May Include Open Models in Secretive AI Framework

The Trump administration is expanding its secretive AI safety framework to inspect some open models before they are made public. Earlier this month, Axios reported that the Trump administration was putting together an AI safety framework to assess the cyber capabilities of models, but would keep the whole thing secret, except from the model providers. Now, WIRED is reporting that although the framework currently only applies to closed models from top providers like OpenAI, it will soon include open models that are deemed advanced enough. The administration is reportedly worried that only giving official approval to closed models would harm the reputation of open models, and "paradoxically disincentivize" American companies from releasing them. Open models are receiving renewed interest in the American AI world after several new releases from Chinese AI labs sparked chatter that Chinese open models are catching up to the capabilities of some of the latest closed offerings from industry-leading American AI companies. The freak-out started last month when Moonshot released Kimi K3, an open-source model that was cheaper yet on par with or better than some of the latest offerings from the likes of Anthropic, OpenAI, and Google. Experts theorized that the Trump administration would respond to this threat with blanket bans, and that did seem to be their main strategy for some time, until AI industry leaders decided to make open models a cause they wanted to fight for. Numerous AI companies, including Meta, Microsoft and Palantir, signed an open letter titled "Open Weights and American AI Leadership," urging the Trump administration to avoid restricting open models, arguing that it would only cause the U.S. to fall behind as China races ahead, and claiming that closed models are "not inherently safe." Nvidia CEO Jensen Huang, who has long championed the idea that open models will be key to winning the global AI race and made this case to the Trump administration before, even decided to make his first-ever X post about this. "Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty," Huang said in the post. "The world needs both frontier closed models and frontier open models." Nvidia's passion for this extended to the formation of the Open Secure AI Alliance, a coalition meant to aid the proliferation of open-weight AI models. OpenAI hadn't initially joined the letter, but CEO Sam Altman later gave his co-sign. Anthropic refrained from backing the letter, which has caused the pro-open model camp to accuse CEO Dario Amodei of trying to push the Administration towards adopting a stricter stance in favor of closed models. Anthropic, which had a very public falling out with the Trump administration not too long ago, is becoming quite the leader in closed models, so critics claim anything that discourages the proliferation of open models would only help their business. Amodei has rejected the accusation.

Open-source AI faces more government scrutiny

Why it matters: Open-source advocates say the models are key for democratizing access to AI and keeping it out of the control of a few powerful players. * But as capabilities improve and introduce new risks, policymakers are not ready to walk away from regulating open source entirely. State of play: The Trump administration has long viewed anything at or above the capabilities of Anthropic and OpenAI's most advanced models as needing some form of government collaboration, regardless of whether they're open- or closed-source, a source familiar with the administration's thinking said. * Open-source players proposed ways to treat their models once they reach frontier capabilities in their recommendations to the White House for the AI framework. * Some companies that were consulted said they wanted to be subject to the pre-release government review, a source familiar with the discussions said. Context: The AI framework defines a covered frontier model as a closed-source model with state-of-the-art capabilities and national security risks, Axios previously reported. * Open models are excluded, and the framework explicitly says nothing in it should be interpreted as restricting open models once they've been released. * Treasury Secretary Scott Bessent and White House Office of Science and Technology Policy Director have recently posted in support of open-source AI. What we're watching: A national security presidential memorandum issued in June could separately set standards for the government's use of open models. * By early September, officials are due to issue a new policy governing AI use in national security systems. * By early October, officials are tasked to develop standardized methods for testing and evaluating AI systems, as well as baseline AI security and risk-management practices. The October implementation deadlines could clarify how open-weight models will be evaluated and procured for national security use, said Eric Syphard, Booz Allen's head of AI. * Booz Allen is building in‑house AI evaluation capabilities so it can independently test open‑weight models before putting them into customer systems. The bottom line: Major U.S. companies are embracing open-source models and AI developers are stepping up efforts to compete with China in the space, even as the government weighs greater oversight.

White House AI Testing Shift Could Put Open Models Back in the Risk File | PYMNTS.com

The reported change, detailed by WIRED, would mean open models could be included in the framework once they reach frontier-level capabilities. That would reverse, or at least soften, the position reportedly shared with AI companies earlier this month, when administration officials said the voluntary review process would not apply to U.S. open-weight models. The distinction matters because open-weight models are becoming more attractive to enterprises. They can be downloaded, customized and hosted privately, making them appealing to banks, retailers and software companies that want more control over cost, data location and model behavior. But that same control creates a different risk profile. Once a powerful open model is released, it is difficult to recall, and its safeguards can be modified or removed by downstream users. The White House's June executive order set up a voluntary framework under which developers of covered frontier models could provide the federal government with early access for cybersecurity evaluation before broader release. The White House fact sheet framed the effort as a way to strengthen cybersecurity and secure innovation without creating mandatory licensing or preclearance. That balance has become more complicated as open models grow more capable. For financial institutions, the question is practical. A bank may prefer an open model because it can run the system in its own environment, keep customer data internal and avoid dependence on a closed provider. Yet if that model can also assist with cyber exploitation, automate tool use or be modified in ways that weaken safeguards, the bank inherits more responsibility for containment. Recent research shows why that matters. A June paper on autonomous penetration capabilities in LLM-powered systems found that tested open-weight and proprietary models could perform penetration tasks at varying success rates, depending on capability and agent scaffolding. The point for banks is not that every model is dangerous. It is that model capability, tools, permissions and network access combine to create operational risk. That has direct implications for payments. An AI agent connected to a bank's internal systems, fraud tools or payment APIs needs more than a model approval memo. It needs restricted credentials, network limits, transaction thresholds, tool allowlists and independent logging. A model that is safe in a sandbox may behave very differently when connected to live systems that can move money, change account status or block a transaction. The reported White House shift could help buyers by giving at least some open models a comparable government testing path. But it will not eliminate the need for institution-specific validation. Federal review may assess broad cyber capability. It cannot determine whether a model is safe inside a particular bank's payment stack, merchant-risk system, collections workflow or customer service agent. The likely result is a more layered vendor-risk process. Banks and merchants will ask whether a model was eligible for government review, whether it was submitted, what findings can be shared and what changed after testing. They will also need to document where the model runs, who can modify it, which safeguards are active and how quickly it can be replaced. Open-weight AI may still offer major advantages: lower cost, customization, local deployment and less dependence on a few dominant providers. But the compliance file will need to become more rigorous as capability rises. For commerce, payments and financial services, the lesson is straightforward. The choice between open and closed AI is no longer just a technology decision. It is a risk, resilience and audit decision. If open models enter the White House testing framework, that may close one assurance gap. It will not close the control gap inside the enterprise.