NIST Unveils AI Model Risk Assessment Tool Amid Growing Concerns

AI News: US Commerce Dept Launches Tool Test Model Risks

The industry's growth has called for more proactive policies recently The increasing demand and popularity of Artificial Intelligence (AI) models have forced authorities in several regions to lay down standards. Introducing an AI tool into such regions would require compliance with such standards. As a result, the United States Commerce Department in collaboration with a few entities have launched Dioptra. Dioptra is a modular, open source web-based tool that the US Commerce Dept re-released in collaboration with the National Institute of Standards and Technology (NIST). I just wanted to let you know that the relaunch of this initiative is coming at a critical time as the AI landscape continues to evolve quickly. The testbed is designed in such a way that it measures the capacity of malicious attacks to impact the performance of an AI system negatively. It is particular about attacks that "poison" AI model training data. Noteworthy, Dioptra's initial release took place two years ago with the focus on helping companies that train AI models and those entities who utilize the innovative technology. Its functionality at the time encompasses assessing, analyzing and tracking AI risks. According to NIST, the tool can be used to benchmark and research models. Also, NIST said "Testing the effects of adversarial attacks on machine learning models is one of the goals of Dioptra." Like several other tools and AI regulation, Dioptra is an approach towards attaining AI model safety and protection of users. A few days ago, tech behemoths including Google, Microsoft, Nvidia, and OpenAI, took the matter of AI safety more intentionally and launched the Coalition for Secure AI (CoSAI). Precisely, CoSAI is focused on establishing robust security frameworks and standards for AI development and deployment. It is worth noting that these companies have some of the most sophisticated AI models. OpenAI have a suite of AI tools including its newly launched search engine, SearchGPT. Google has Gemini 1.5 Flash as well as other tools while Nvidia remains a thriving chip maker. The consistent launch of these AI models to the public threatens users safety. Governments in different regions are putting effort and resources to checkmate the burgeoning niche. Therefore, the re-release of Dioptra is a reflection of these profound commitment to the industry.

NIST releases a tool for testing AI model risk

The National Institute of Standards and Technology (NIST), the U.S. Commerce Department agency that develops and tests tech for the U.S. government, companies and the broader public, has re-released a testbed designed to measure how malicious attacks -- particularly attacks that "poison" AI model training data -- might degrade the performance of an AI system. Called Dioptra (after the classical astronomical and surveying instrument), the modular, open source web-based tool, first released in 2022, seeks to help companies training AI models -- and the people using these models -- assess, analyze and track AI risks. Dioptra can be used to benchmark and research models, NIST says, as well as to provide a common platform for exposing models to simulated threats in a "red-teaming" environment. "Testing the effects of adversarial attacks on machine learning models is one of the goals of Dioptra," NIST wrote in a press release. "The open source software, like generating child available for free download, could help the community, including government agencies and small to medium-sized businesses, conduct evaluations to assess AI developers' claims about their systems' performance." Dioptra debuted alongside documents from NIST and NIST's recently created AI Safety Institute that lay out ways to mitigate some of the dangers of AI, like how it can be abused to generate nonconsensual pornography. It follows the launch of the U.K. AI Safety Institute's Inspect, a toolset similarly aimed at assessing the capabilities of models and overall model safety. The U.S. and U.K. have an ongoing partnership to jointly develop advanced AI model testing, announced at the U.K.'s AI Safety Summit in Bletchley Park in November of last year. Dioptra is also the product of President Joe Biden's executive order (EO) on AI, which mandates (among other things) that NIST help with AI system testing. The EO, relatedly, also establishes standards for AI safety and security, including requirements for companies developing models (e.g. Apple) to notify the federal government and share results of all safety tests before they're deployed to the public. As we've written about before, AI benchmarks are hard -- not least of which because the most sophisticated AI models today are black boxes whose infrastructure, training data and other key details are kept under wraps by the companies creating them. A report out this month from the Ada Lovelace Institute, a U.K.-based nonprofit research institute that studies AI, found that evaluations alone aren't sufficient to determine the real-world safety of an AI model in part because current policies allow AI vendors to selectively choose which evaluations to conduct. NIST doesn't assert that Dioptra can completely de-risk models. But the agency does propose that Dioptra can shed light on which sorts of attacks might make an AI system perform less effectively and quantify this impact to performance. In a major limitation, however, Dioptra only works out-of-the-box on models that can be downloaded and used locally, like Meta's expanding Llama family. Models gated behind an API, such as OpenAI's GPT-4o, are a no-go -- at least for the time being.