Gentrace Raises $8M to Revolutionize AI App Testing and Collaboration

Generative AI app testing platform Gentrace raises $8M to make LLM development more accessible - SiliconANGLE

Generative AI app testing platform Gentrace raises $8M to make LLM development more accessible Gentrace, a developer platform for testing and monitoring artificial intelligence applications, said today it has raised $8 million in an early-stage funding round led by Matrix Partners to expand large language model testing product capabilities beyond engineering teams. Today's Series A funding round attracted participation from Headline and K9 Ventures and brings the company's total raised to date to more than $14 million. Founded in 2023, Gentrace offers a testing and monitoring product that allows non-technical users to participate in the evaluation, testing and monitoring of AI applications. According to the company, many industries are rushing to add generative AI to their offerings, development teams are facing the challenge of ensuring it remains reliable and safe. Similarly, the ability to evaluate and test large language models remains largely the domain of development and engineering teams, making it difficult to collaborate with product managers, subject matter experts, designers and quality assurance teams. "Generative AI represents a paradigm shift in software development, but the reality is there's way too much noise and not enough signal on how to test and build them easily or correctly," said Doug Safreno, co-founder and chief executive of Gentrace. "We're not just creating another dev tool -- we're reimagining how entire organizations can collaborate and build better LLM products." To help tackle this challenge, Gentrace announced Experiments, a tool that allows cross-functional teams to collaborate in purpose-built testing environments to assess AI model performance. They can test AI outputs directly, preview test outcomes, anticipate errors and explore scenarios while exchanging data and information freely between technical and non-technical members. The company's platform and Experiments interfaces with many existing tools and model providers including OpenAI, vector database Pinecone Systems Inc. and visual LLM programming environment Rivet. Early adopters, including companies such as Webflow and Quizlet using the platform helped predict AI-related issues before they affected users. According to Quizlet, by implementing Gentrace's platform, the company increased testing frequency from two times per month to more than 20 times per week, significantly improving the speed of iteration with testing. "Gentrace was the right product for us because it allowed us to implement our custom evaluations, which was crucial for our unique use cases," said Madeline Gilbert, a staff machine learning engineer at Quizlet. As an education technology company providing study tools for students and teachers, Quizlet uses generative AI on unstructured notes and materials to create study tools. According to Gilbert, even minor changes, such as a comma in a prompt, could significantly change the predictability of models. Gentrace's solution allowed quality assurance teams and subject matter experts to evaluate and test quickly after any modification. "It's dramatically improved our ability to predict the impact of even small changes in our LLM implementations," said Gilbert.

Gentrace makes it easier for businesses to test AI-powered software

As businesses continue to integrate generative AI into their products, many find it challenging to actually test whether the AI is behaving correctly and giving useful answers. To help address this problem, a startup called Gentrace offers an integrated platform for testing software built around large language models. Whereas traditional software that can be subjected to automated tests to verify that, say, data submitted to a web form ended up properly formatted in a database, AI-powered software often can't be expected to behave exactly in a specified way in response to input, says Gentrace cofounder and CEO Doug Safreno. Customers can end up defining a set of test data for the AI after any changes to the AI model, the databases it interacts with, or other parameters. But without a testing platform, running those tests can mean maintaining spreadsheets of AI test prompts and manually logging that they give satisfactory results. And while automation is possible, verifying that an AI response contains certain keywords or even asking another AI system to confirm that an AI looks satisfactory, complex testing often requires engineers to be heavily involved, even if other team members like product managers might know better what good output looks like, Safreno says. "The problem becomes, nobody can look at it and collaborate on these tests and on these evaluation methods," he says. "As new product requirements come in, they're not being captured in the testing."