OpenAI Introduces Crucial Safety Feature for LLMs in Response to Developer Demand

OpenAI responds to 'popular demand' with launch of key LLM safety feature for developers - SiliconANGLE

OpenAI responds to 'popular demand' with launch of key LLM safety feature for developers OpenAI has announced a key update that finally delivers the top-requested feature developers have been asking for. With the launch of its Structured Outputs in the API feature, it's providing developers with a reliable way to ensure that the outputs of their generative artificial intelligence models match the data stored within their JavaScript Object Notation Schema files. JSON Schema files are essential for AI development, because they're used to describe the content, structure and type of data within the JSON documents that developers feed into their large language models. They also describe the constraints on the outputs based on that data. JSON documents are files that use the JSON format for storing and exchanging text-based data. They're extremely popular because the form is simple to use, very flexible and compatible with multiple programming languages. In addition, it's easily readable by both humans and machines, and for that reason it has become an industry standard. Sadly, LLMs don't seem to quite be able to match the ability of humans and regular machines when it comes to reading JSON files. They struggle to do so, and so LLMs that use JSON data will often "hallucinate" and make up responses or output toxic content. That's because they're only able to partially follow the instructions within the JSON files. To get around this, developers have been forced to use an assortment of clunky open-source files and test their models with numerous prompts and repeated requests to ensure their outputs fall within safe boundaries. This is why Structured Outputs in the API has become the most in-demand feature from developers that work with OpenAI's LLMs. By ensuring that LLM outputs match the contents of their JSON files, developers can build more consistent and safer generative AI applications. In a tweet, OpenAI Chief Executive Sam Altman said the feature was being released due to "popular demand", and in a blog post, the company said that evaluations of Structured Outputs on its most powerful LLM, GPT-4o, achieved a perfect 100% score. OpenAI first added support for JSON documents with the debut of its JSON Mode last year, but the feature received mixed reviews because of the problem of ensuring reliable outputs. With today's update, OpenAI said developers will now find it much easier to constrain its LLMs and ensure they match their JSON Schemas. As a side benefit, Structured Outputs in the API also helps LLMs to better understand more complex JSON Schemas, the company said. "While both (JSON Mode and Structured Outputs) ensure valid JSON is produced, only Structured Outputs ensures schema adherence," the company wrote in its blog post. According to the company, this means developers will no longer have to worry that the LLMs they're using might omit a required key or hallucinate on invalid enumeration values. With Structured Outputs, developers will be able to request a "step-by-step" answer that guides them through to the intended output. "Safety is a top priority for OpenAI," the company insisted. "The new Structured Outputs functionality will abide by our existing safety practices and allow models to refuse an unsafe request." Structured Outputs in the API is available now with GPT-4o, GPT-40-mini and multiple fine-tuned versions of those two LLMs. It can be used with various application programming interfaces, including the Assistants API, Batch API and Chat Completions API, and it is compatible with vision inputs as well as text. Today's update comes at yet another tumultuous time for OpenAI, which has been hit by the departures of three key executives in the last week. On Tuesday, it was reported that Jon Schulman, who played a key role in its product development efforts, was leaving the company to join rival AI firm Anthropic PBC. Schulman announced his departure on X, becoming the 8th of OpenAI's 11 co-founders to quit the company. Within hours, it followed that another co-founder, Greg Brockman - who serves as the company's president - was stepping back for four months in order to take a sabbatical and "relax for the first time" since helping to launch the company nine years ago. Brockman, Altman and chief researcher Wojciech Zaremba are the only members of the original founding team still working at the company. Schulman's and Brockman's announcements came on the heels of last week's news that OpenAI's product manager Peter Deng had left the company, having only joined it last year. Also this week, another of OpenAI's co-founders, tech entrepreneur Elon Musk, revealed that he has filed another lawsuit against the company. This time around, he is accusing the company of misleading him about the goals of its AI development efforts. In the lawsuit, Musk says that OpenAI positioned itself as a nonprofit entity that was "committed to making its AI technology available under an open-source license." He argues that the launch of its for-profit arm later and its subsequent partnership with Microsoft Corp. breached this agreement. Musk added that OpenAI was guilty of committing a "Shakespearian" betrayal when it abandoned its open-source mission.

OpenAI has finally released the No. 1 feature developers have been desperate for

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More The JavaScript Object Notation (JSON) file and data interchange format is an industry-standard because it is both easily readable by humans and parsable by machines. However, large language models (LLMs) notoriously struggle with JSON -- they might hallucinate, create wonky responses that only partially adhere to instructions or fail to parse completely. This often requires developers to use workarounds such as open-source tooling, many different prompts or repeated requests to ensure output interoperability. Now, OpenAI is helping ease these frustrations with the release of its Structured Outputs in the API. Released today, the functionality helps ensure that model-generated outputs match JSON Schemas. These schemas are critical because they describe content, structure, types of data and expected constraints in a given JSON document. OpenAI says it is the No. 1 feature developers have been asking for because it allows for consistency across various applications. OpenAI CEO Sam Altman posted on X today that the release is by "very popular demand." The company said that its evaluations with Structured Outputs on its new GPT-4o scores a "perfect 100%." The new feature announcement comes on the heels of quite a bit of excitement at OpenAI this week: Three key executives -- John Schulman, Greg Brockman and Peter Deng -- suddenly each announced their departure, and Elon Musk is yet again suing the company, calling the betrayal of their AI mission "Shakespearian." Easily ensuring schema adherence JSON is a text-based format for storing and exchanging data. It has become one of the most popular data formats among developers because it is simple, flexible and compatible with various programming languages. OpenAI quickly met demand from developers when it released its JSON mode on its models at last year's DevDay. With Structured Outputs in the API, developers can constrain OpenAI models to match schemas. OpenAI says the feature also allows its models to better understand more complicated schemas. "Structured Outputs is the evolution of JSON mode," the company writes on its blog. "While both ensure valid JSON is produced, only Structured Outputs ensure schema adherence." This means that developers "don't need to worry about the model omitting a required key, or hallucinating an invalid enum value." (Enumeration value is a process that names constants in language, making code easier to read and maintain). Developers can ask Structured Outputs to generate an answer in a step-by-step way to guide through to the intended output. According to OpenAI, developers don't need to validate or retry incorrectly formatted responses, and the feature allows for simpler prompting while providing explicit refusals. "Safety is a top priority for OpenAI -- the new Structured Outputs functionality will abide by our existing safety policies and will still allow the model to refuse an unsafe request," the company writes. Structured Outputs is available on GPT-4o-mini, GPT-4o and fine-tuned versions of these models, and can be used on the Chat Completions API, Assistants API and Batch API, and it is also compatible with vision inputs. OpenAI emphasizes that the new functionality "takes inspiration from excellent work from the open source community: namely, the outlines, jsonformer, instructor, guidance and lark libraries."