AI Safety Index Reveals Alarming Gaps in Leading Companies' Practices

Meta gets an F in first AI safety scorecard -- and the others barely pass

As artificial intelligence evolves, it's clear there's some need for oversight. Most AI labs openly support regulation and provide access to frontier models for independent evaluation before release -- but they could be doing more. The world is turning to AI to solve all manner of problems, but without proper oversight, it could always create more. Future of Life has developed a report card for the different AI labs including OpenAI, Meta, Anthropic and Elon Musk's xAI. The AI Safety Index is an independent review looking at 42 indicators of "responsible conduct". The report gives a letter grade to each company based on these indicators and Meta, focused on open-source AI models through its Llama family, gets an F The panel includes a series of luminaries in education and think tanks to check on how AI companies are operating, and the initial results are alarming. Looking at Anthropic, Google DeepMind, Meta, OpenAI, x.AI and Zhipu AI, the report has found "significant gaps in safety measures and a serious need for improved accountability." According to the first report card, Meta scores lowest (x.AI isn't far behind), while Anthropic comes out on top -- but still only gets a C. All flagship models were found to be "vulnerable to adversarial attacks", while also having the potential to be unsafe and break away from human control. Perhaps most damning, the report says "Reviewers consistently highlighted how companies were unable to resist profit-driven incentives to cut corners on safety in the absence of independent oversight." "While Anthropic's current and OpenAI's initial governance structures were highlighted as promising, experts called for third-party validation of risk assessment and safety framework compliance across all companies." In short, this is the kind of oversight and accountability we need to see in the burgeoning AI industry before it's too late, as the more powerful the models get, the more real the harms become.

Leading AI Companies Get Lousy Grades on Safety

The just-released AI Safety Index graded six leading AI companies on their risk assessment efforts and safety procedures... and the top of class was Anthropic, with an overall score of C. The other five companies -- Google DeepMind, Meta, OpenAI, xAI, and Zhipu AI -- received grades of D+ or lower, with Meta flat out failing. "The purpose of this is not to shame anybody," says Max Tegmark, an MIT physics professor and president of the Future of Life Institute, which put out the report. "It's to provide incentives for companies to improve." He hopes that company executives will view the index like universities view the U.S. News and World Reports rankings: They may not enjoy being graded, but if the grades are out there and getting attention, they'll feel driven to do better next year. He also hopes to help researchers working in those companies' safety teams. If a company isn't feeling external pressure to meet safety standards, Tegmark says,"then other people in the company will just view you as a nuisance, someone who's trying to slow things down and throw gravel in the machinery." But if those safety researchers are suddenly responsible for improving the company's reputation, they'll get resources, respect, and influence. The Future of Life Institute is a nonprofit dedicated to helping humanity ward off truly bad outcomes from powerful technologies, and in recent years it has focused on AI. In 2023, the group put out what came to be known as "the pause letter," which called on AI labs to pause development of advanced models for six months, and to use that time to develop safety standards. Big names like Elon Musk and Steve Wozniak signed the letter (and to date, a total of 33,707 have signed), but the companies did not pause. This new report may also be ignored by the companies in question. IEEE Spectrum reached out to all the companies for comment, but only Google DeepMind responded, providing the following statement: "While the index incorporates some of Google DeepMind's AI safety efforts, and reflects industry-adopted benchmarks, our comprehensive approach to AI safety extends beyond what's captured. We remain committed to continuously evolving our safety measures alongside our technological advancements." The Index graded the companies on how well they're doing in six categories: risk assessment, current harms, safety frameworks, existential safety strategy, governance and accountability, and transparency and communication. It drew on publicly available information, including related research papers, policy documents, news articles, and industry reports. The reviewers also sent a questionnaire to each company, but only xAI and the Chinese company Zhipu AI (which currently has the most capable Chinese-language LLM) filled theirs out, boosting those two companies' scores for transparency. The grades were given by seven independent reviewers, including big names like UC Berkeley professor Stuart Russell and Turing Award winner Yoshua Bengio, who have said that superintelligent AI could pose an existential risk to humanity. The reviewers also included AI leaders who have focused on near-term harms of AI like algorithmic bias and toxic language, such as Carnegie Mellon University's Atoosa Kasirzadeh and Sneha Revanur, the founder of Encode Justice. And overall, the reviewers were not impressed. "The findings of the AI Safety Index project suggest that although there is a lot of activity at AI companies that goes under the heading of 'safety,' it is not yet very effective," says Russell."In particular, none of the current activity provides any kind of quantitative guarantee of safety; nor does it seem possible to provide such guarantees given the current approach to AI via giant black boxes trained on unimaginably vast quantities of data. And it's only going to get harder as these AI systems get bigger. In other words, it's possible that the current technology direction can never support the necessary safety guarantees, in which case it's really a dead end." Anthropic got the best scores overall and the best specific score, getting the only B- for its work on current harms. The report notes that Anthropic's models have received the highest scores on leading safety benchmarks. The company also has a "responsible scaling policy" mandating that the company will assess its models for their potential to cause catastrophic harms, and will not deploy models that the company judges too risky. All six companies scaled particularly badly on their existential safety strategies. The reviewers noted that all of the companies have declared their intention to build artificial general intelligence (AGI), but only Anthropic, Google DeepMind, and OpenAI have articulated any kind of strategy for ensuring that the AGI remains aligned with human values. "The truth is, nobody knows how to control a new species that's much smarter than us," Tegmark says. "The review panel felt that even the [companies] that had some sort of early-stage strategies, they were not adequate." While the report does not issue any recommendations for either AI companies or policymakers, Tegmark feels strongly that its findings show a clear need for regulatory oversight -- a government entity equivalent to the U.S. Food and Drug Administration that would approve AI products before they reach the market. "I feel that the leaders of these companies are trapped in a race to the bottom that none of them can get out of, no matter how kind-hearted they are," Tegmark says. Today, he says, companies are unwilling to slow down for safety tests because they don't want competitors to beat them to the market. "Whereas if there are safety standards, then instead there's commercial pressure to see who can meet the safety standards first, because then they get to sell first and make money first."

Which AI Companies Are the Safest -- and Least Safe?

As companies race to build more powerful AI, safety measures are being left behind. A report published Wednesday takes a closer look at how companies including OpenAI and Google DeepMind are grappling with the potential harms of their technology. It paints a worrying picture: flagship models from all the developers in the report were found to have vulnerabilities, and some companies have taken steps to enhance safety, others lag dangerously behind. The report was published by the Future of Life Institute, a nonprofit that aims to reduce global catastrophic risks. The organization's 2023 open letter calling for a pause on large-scale AI model training drew unprecedented support from 30,000 signatories, including some of technology's most prominent voices. For the report, the Future of Life Institute brought together a panel of seven independent experts -- including Turing Award winner Yoshua Bengio and Sneha Revanur from Encode Justice -- who evaluated technology companies across six key areas: risk assessment, current harms, safety frameworks, existential safety strategy, governance & accountability, and transparency & communication. Their review considered a range of potential harms, from carbon emissions to the risk of an AI system going rogue. "The findings of the AI Safety Index project suggest that although there is a lot of activity at AI companies that goes under the heading of 'safety,' it is not yet very effective," said Stuart Russell, a professor of computer science at University of California, Berkeley and one of the panelists, in a statement. Read more: No One Truly Knows How AI Systems Work. A New Discovery Could Change That Despite touting its "responsible" approach to AI development, Meta, Facebook's parent company, and developer of the popular Llama series of AI models, was rated the lowest, scoring a F-grade overall. X.AI, Elon Musk's AI company, also fared poorly, receiving a D- grade overall. Neither Meta nor x.AI responded to a request for comment. The company behind ChatGPT, OpenAI -- which early in the year was accused of prioritizing "shiny products" over safety by the former leader of one of its safety teams -- received a D+, as did Google DeepMind. Neither company responded to a request for comment. Zhipu AI, the only Chinese AI developer to sign a commitment to AI safety during the Seoul AI Summit in May, was rated D overall. Zhipu could not be reached for comment. Anthropic, the company behind the popular chatbot Claude, which has made safety a core part of its ethos, ranked the highest. Even still, the company received a C grade, highlighting that there is room for improvement among even the industry's safest players. Anthropic did not respond to a request for comment. In particular, the report found that all of the flagship models evaluated were found to be vulnerable to "jailbreaks," or techniques that override the system guardrails. Moreover, the review panel deemed the current strategies of all companies inadequate for ensuring that hypothetical future AI systems which rival human intelligence remain safe and under human control. Read more: Inside Anthropic, the AI Company Betting That Safety Can Be a Winning Strategy "I think it's very easy to be misled by having good intentions if nobody's holding you accountable," says Tegan Maharaj, assistant professor in the department of decision sciences at HEC Montréal, who served on the panel. Maharaj adds that she believes there is a need for "independent oversight," as opposed to relying solely on companies to conduct in-house evaluations. There are some examples of "low-hanging fruit," says Maharaj, or relatively simple actions by some developers to marginally improve their technology's safety. "Some companies are not even doing the basics," she adds. For example, Zhipu AI, x.AI, and Meta, which each rated poorly on risk assessments, could adopt existing guidelines, she argues. However, other risks are more fundamental to the way AI models are currently produced, and overcoming them will require technical breakthroughs. "None of the current activity provides any kind of quantitative guarantee of safety; nor does it seem possible to provide such guarantees given the current approach to AI via giant black boxes trained on unimaginably vast quantities of data," Russell said. "And it's only going to get harder as these AI systems get bigger." Researchers are studying techniques to peer inside the black box of machine learning models. In a statement, Bengio, who is the founder and scientific director for Montreal Institute for Learning Algorithms, underscored the importance of initiatives like the AI Safety Index. "They are an essential step in holding firms accountable for their safety commitments and can help highlight emerging best practices and encourage competitors to adopt more responsible approaches," he said.