OpenAI Partners with U.S. AI Safety Institute, Pledges Early Access to Future Models

ChapGPT-maker OpenAI partners with AI Safety Institute

OpenAI has partnered with the U.S. AI Safety Institute, a federal government body, to give them early access to the company's next foundational model for safety testing. In a post on X, CEO Sam Altman noted that their goal was to "work together to push forward to science of AI evaluations." Earlier this year in May, the ChatGPT-maker had disbanded their Superalignment team which was started with the need to ensure that their AI products align with human intentions and prevent them from going "rogue." The move led to resignations from the team's leads Jan Leike who went on to join rival Anthropic AI's safety research team and co-founder Ilya Sutskever, who then started his own safety-focused AI startup called Safe Superintelligence Inc. After his departure, Leike had indicated in a post that his team hadn't been given the compute promised as the focus was entirely on launching new products. In the same post Altman made announcing the partnership with the U.S. AI Safety Institute, he emphasised that they would stick to the portion of 20% compute allocated for safety efforts. (For top technology news of the day, subscribe to our tech newsletter Today's Cache) In June, retired U.S. army general Paul M. Nakasone joined the AI firm's board of directors. In the blog post making the announcement, it was said that, "Nakasone's appointment reflects OpenAI's commitment to safety and security, and underscores the growing significance of cybersecurity as the impact of AI technology continues to grow." Altman also confirmed that they will be removing the indefinite non-disparagement clause from employee contracts that had received criticism for its severity. "We want current and former employees to be able to raise concerns and feel comfortable doing so," he noted. Read Comments

OpenAI pledges to give U.S. AI Safety Institute early access to its next model | TechCrunch

OpenAI CEO Sam Altman says that OpenAI is working with the U.S. AI Safety Institute, a federal government body that aims to assess and address risks in AI platforms, on an agreement to provide early access to its next major generative AI model for safety testing. The announcement, which Altman made in a post on X late Thursday evening, was light on details. But it -- along with a similar deal with the U.K.'s AI safety body struck in June -- appears to be intended to counter the narrative that OpenAI has deprioritized work on AI safety in the pursuit of more capable and powerful AI technologies. In May, OpenAI effectively disbanded a unit working on the problem of developing controls to prevent "superintelligent" AI systems from going rogue. Reporting -- including ours -- suggested that OpenAI cast aside the team's safety research in favor of launching new products, ultimately leading to the resignation of the team's two co-leads, Jan Leike (who now leads safety research at AI startup Anthropic) and OpenAI co-founder Ilya Sutskever (who started his own safety-focused AI company, Safe Superintelligence Inc.). In response to a growing chorus of critics, OpenAI said it would eliminate its restrictive non-disparagement clauses that implicitly discouraged whistleblowing and create a safety commission, as well as dedicate 20% of its compute to safety research. (The disbanded safety team had been promised 20% of OpenAI's compute for its work, but ultimately never received this.) Altman re-committed to the 20% pledge and re-affirmed that OpenAI voided the non-disparagement terms for new and existing staff in May. The moves did little to placate some observers, however -- particularly after OpenAI staffed the safety commission will all company insiders including Altman and, more recently, reassigned a top AI safety executive to another org. Five senators including Brian Schatz, a Democrat from Hawaii, raised questions about OpenAI's policies in a recent letter addressed to Altman. OpenAI chief strategy officer Jason Kwon responded to the letter today, writing the OpenAI "[is] dedicated to implementing rigorous safety protocols at every stage of our process." The timing of OpenAI's agreement with the U.S. AI Safety Institute seems a tad suspect in light of the company's endorsement earlier this week of the Future of Innovation Act, a proposed Senate bill that would authorize the Safety Institute as an executive body that sets standards and guidelines for AI models. The moves together could be perceived as an attempt at regulatory capture -- or at the very least an exertion of influence from OpenAI over AI policymaking at the federal level. Not for nothing, Altman is among the U.S. Department of Homeland Security's Artificial Intelligence Safety and Security Board, which provides recommendations for the "safe and secure development and deployment of AI" throughout the U.S.' critical infrastructures. And OpenAI has dramatically increased its expenditures on federal lobbying this year, spending $800,000 in the first six months of 2024 versus $260,000 in all of 2023. The U.S. AI Safety Institute, housed within the Commerce Department's National Institute of Standards and Technology, consults with a consortium of companies that includes Anthropic as well as big tech firms like Google, Microsoft, Meta, Apple, Amazon and Nvidia. The industry group is tasked with working on actions outlined in President Joe Biden's October AI executive order, including developing guidelines for AI red-teaming, capability evaluations, risk management, safety and security and watermarking synthetic content.

OpenAI Says 'Dedicated' to Safety in Letter to US Lawmakers

OpenAI, responding to questions from US lawmakers, said it's dedicated to making sure its powerful AI tools don't cause harm, and that employees have ways to raise concerns about safety practices. The startup sought to reassure lawmakers of its commitment to safety after five senators including Senator Brian Schatz, a Democrat from Hawaii, raised questions about OpenAI's policies in a letter addressed to Chief Executive Officer Sam Altman.