OpenAI and Anthropic Push for Control Over What Independent Reviewers Can Publish About AI Safety

OpenAI and Anthropic Rethink Independence Promises

OpenAI and Anthropic have sought significant control over what outside evaluators publish about their AI safety evaluations, despite recent public commitments to independence. According to Reuters, both companies worry that embedded reviewers working inside their facilities could expose trade secrets to competitors1

1

2

2

. As recently as October 5, both labs were seeking strict editorial control over what independent reviewers could publish. By October 8, Anthropic's talks with some evaluators appeared to have evolved, though the nature of these negotiations remains fluid2

2

.

Last month, Anthropic CEO Dario Amodei stated that external evaluators should have the right to publish key findings without editorial control. OpenAI echoed this commitment days later, promising assessors editorial independence1

1

3

3

. The AI industry's call for independent reviews now faces scrutiny as these negotiations reveal the tension between transparency and trade secrets.

Measures to Prevent Intellectual Property Leaks

Both companies are exploring measures to protect proprietary information while allowing AI model reviews to proceed. Ideas under discussion include having lab employees observe embedded evaluation teams as they work and preventing evaluators embedded at one lab from working at another simultaneously1

1

2

2

. The concern centers on embedded evaluators who work inside the labs' walls, where access to sensitive systems and data is unavoidable.

What labs consider crucial intellectual property can overlap with information relevant to judging a model's risk, which can lead to arguments and redactions, according to sources familiar with past dealings with the labs2

2

. Language AI labs already use in certain contracts with firms that evaluate their models can bar those firms from publishing confidential information in their reports without the companies' consent2

2

.

Diverging Approaches to Publication Rights

Anthropic told Reuters that seeking significant control over what outside evaluators publish did not accurately characterize all of its discussions with independent reviewers. For evaluators it has worked with before, the company said those reviewers control their methods, evaluation criteria, conclusions and what to publish. However, for new evaluator firms, Anthropic reserves the right to comment on and approve reports as the reviewers learn how its models operate1

1

2

2

.

OpenAI stated it was not seeking approval rights over the full text of reports produced by evaluators. The company said its standard practice is to redact confidential information, and evaluators can disclose whether major redactions were made2

2

3

3

. OpenAI has already cut some details from a report written by safety groups METR and Redwood Research about the time its agents hacked Hugging Face, though the groups said nothing key to their findings was removed1

1

.

Stakes for AI Governance and Industry Trust

In September, Anthropic named Faculty, Accenture's specialist AI business, as its first embedded evaluator, while OpenAI plans to let outside groups test models during training1

1

2

2

. Faculty CEO Marc Warner emphasized that "evaluators are only useful if people can trust what they report, so independence and transparency are critical"1

1

2

2

.

Rumman Chowdhury, who was US science envoy for AI under Joe Biden, warned that labs could use access as leverage while there are no rules. "Evaluators have this very privileged access, but whether explicitly or implicitly, they know that it comes at a cost," she told Reuters1

1

. Without agreed standards and legislation for independent reviews, some experts said the companies building the most powerful AI models could dictate how their systems are examined, leaving evaluators at the mercy of the labs themselves2

2

.

Growing Pressure for Government Oversight

Independent evaluation was one of four safety measures in a voluntary oversight agreement signed last month by President Trump and leading AI executives, but the agreement did not specify how safety reviews should be conducted3

3

. In an open letter signed last month by 200 researchers, including Geoffrey Hinton, potential evaluators called for independent control of their findings, broad access to AI labs and protection from retaliation3

3

.

Several bills have recently been introduced in Congress aimed at requiring greater transparency by AI labs. Democratic senators introduced the AI Risk Management and Security Act, which would put the Commerce Department in charge of devising technical standards for evaluating AI models' risk3

3

. A separate bill by Republican Sen. Josh Hawley would put the Department of Energy in charge of stress-testing the most powerful AI systems before they reach the public, imposing severe financial penalties on companies that release new frontier models without prior review3

3

.

Microsoft President Brad Smith told the BBC that "the moment has arrived" for government action, noting that "right now, your washing machine is subject to more safety rules than AI." He called for global AI safety rules and requirements for companies to disclose when rogue AI agents breach systems3

3

. For OpenAI and Anthropic, however, calls for greater transparency come as they seek trillion-dollar valuations ahead of planned IPOs, making the disclosure of valuable trade secrets potentially damaging to investor confidence3

3

.