OpenAI and Anthropic Partner with US AI Safety Institute for Safer AI Development

US AI Safety Institute will have access to OpenAI and Anthropic for a safer AI-led future - SiliconANGLE

US AI Safety Institute will have access to OpenAI and Anthropic for a safer AI-led future OpenAI and Anthropic PBC today announced they have agreed to share AI models before and after release with the U.S. government's AI Safety Institute. The institute, housed at the U.S. Department of Commerce's National Institute of Standards and Technology, NIST, was set up through an executive order by President Biden in 2023. Working with a consortium of companies and experts, the onus is on establishing safety guidelines and good practices while evaluating potentially dangerous AI systems. The companies today explained that the institute will have early access to anything they create as well as access once the product is on the market. The institute will also provide feedback to its counterpart in the U.K. "Safety is essential to fueling breakthrough technological innovation," said Elizabeth Kelly, director of the U.S. AI Safety Institute. "With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety. These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI." The existential threat AI could pose to humanity was a hot-button topic long before this recent surge in the use of generative AI. The rationale has mostly been to proceed with care, but at the same time not get bogged down by AI panics. In an open letter in June, signed by a group of current and former researchers from OpenAI, Alphabet Inc.'s Google DeepMind research group, and Anthropic, the signatories asked that there be more transparency and oversight to protect the public from potentially harmful AI products. The signatories warned that "strong financial incentives" could mean a lack of "effective oversight." They added the leading AI companies in the U.S. "have only weak obligations to share some of this information with governments, and none with civil society," and it's likely they won't "share it voluntarily." The announcement today might placate the concerned group. OpenAI's Sam Altman wrote on X that it's "important that this happens at the national level," adding that the "U.S. needs to continue to lead!" "This strengthens our ability to identify and mitigate risks, advancing responsible AI development," said Anthropic's co-founder and Head of Policy Jack Clark, in a statement to the media. "We're proud to contribute to this vital work, setting new benchmarks for safe and trustworthy AI."

OpenAI and Anthropic Open Up to US AI Safety Institute - Decrypt

The U.S. Department of Commerce's National Institute of Standards and Technology (NIST) has made a deal with top AI developers OpenAI and Anthropic to establish formal collaboration with the U.S. AI Safety Institute (AISI), the agency said on Thursday. The institute would "receive access to major new models from each company prior to and following their public release," according to the announcement. Anthropic is developing Claude, while OpenAI offers ChatGPT. The arrangement will allow the Institute to evaluate the capabilities and safety risks of the respective AI models. "Safety is essential to fueling breakthrough technological innovation. With these agreements in place, we look forward to beginning our technical collaborations with Anthropic and OpenAI to advance the science of AI safety," agency director Elizabeth Kelly said in a statement. "These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI." Anthropic and OpenAI did not immediately respond to requests for comment by Decrypt. "Looking forward to doing a pre-deployment test on our next model with the U.S. AISI," Anthropic co-founder Jack Clark wrote on Twitter. "Third-party testing is a really important part of the AI ecosystem and it's been amazing to see governments stand up safety institutes to facilitate this." "We are happy to have reached an agreement with the U.S. AI Safety Institute for pre-release testing of our future models," OpenAI co-founder and CEO Sam Altman wrote on Twitter. "For many reasons, we think it's important that this happens at the national level. [The] U.S. needs to continue to lead!" The issue of AI safety has permeated every level of the industry, with many leading experts and executives leaving OpenAI over concerns about its practices -- in some cases forming rival companies centered around cautious development. Governments are also concerned. Launched in October 2023 by the Biden Administration the AISI formed after President Biden issued a sweeping Executive Order aimed at reigning in artificial intelligence development. In February, the Biden Administration announced the first members of what it called the AISI Consortium (AISIC). The AISIC included several high-profile AI firms, including OpenAI, Anthropic, Google, Apple, NVIDIA, Microsoft, and Amazon. The U.S. AISI said it will share its OpenAI and Anthropic findings with its European counterparts at the U.K. AI Safety Institute.

OpenAI, Anthropic Agree to Work With US Institute on Safety Testing

The US government announced agreements with leading artificial intelligence startups OpenAI and Anthropic to help test and evaluate their upcoming technologies for safety. Under the agreements, announced on Thursday, the US AI Safety Institute will receive early access to major new AI models from the companies to evaluate capabilities and risks as well as collaborate on methods to mitigate potential issues. The AI Safety Institute is part of the Commerce Department's National Institute of Standards and Technology, or NIST. The agreements come at a time when there's been an increasing push to mitigate potentially catastrophic risks of AI through regulation, such as the controversial California AI safety bill SB 1047, which recently passed the state Assembly.

OpenAI and Anthropic agree to share models with U.S. government safety institute

Driving the news: Anthropic and OpenAI have each signed a memorandum of understanding to allow formal collaboration with the U.S. Artificial Intelligence Safety Institute, a part of the Commerce Department's National Institute of Standards and Technology. Between the lines: The U.S. AI Safety Institute was set up as part of President Biden's AI executive order. What they're saying: "Safety is essential to fueling breakthrough technological innovation," U.S. AI Safety Institute director Elizabeth Kelly said in a statement. "These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI."

OpenAI, Anthropic sign deals with US govt for AI research and testing

(Reuters) - AI startups OpenAI and Anthropic have signed deals with the United States government for research, testing and evaluation of their artificial intelligence models, the U.S. Artificial Intelligence Safety Institute said on Thursday. The first-of-their-kind agreements come at a time when the companies are facing regulatory scrutiny over safe and ethical use of AI technologies. California legislators are set to vote on a bill as soon as this week to broadly regulate how AI is developed and deployed in the state. Under the deals, the U.S. AI Safety Institute will have access to major new models from both OpenAI and Anthropic prior to and following their public release. The agreements will also enable collaborative research to evaluate capabilities of the AI models and risks associated with them. "We believe the institute has a critical role to play in defining U.S. leadership in responsibly developing artificial intelligence and hope that our work together offers a framework that the rest of the world can build on," said Jason Kwon, chief strategy officer at ChatGPT maker OpenAI. Anthropic, which is backed by Amazon and Alphabet, did not immediately respond to a Reuters request for comment. "These agreements are just the start, but they are an important milestone as we work to help responsibly steward the future of AI," said Elizabeth Kelly, director of the U.S. AI Safety Institute. The institute, a part of the U.S. commerce department's National Institute of Standards and Technology (NIST), will also collaborate with the U.K. AI Safety Institute and provide feedback to the companies on potential safety improvements. The U.S. AI Safety Institute was launched last year as part of an executive order by President Joe Biden's administration to evaluate known and emerging risks of artificial intelligence models. (Reporting by Harshita Mary Varghese in Bengaluru; Editing by Shinjini Ganguli)