Skip to content
D-CSIL™

AI News · 2026-11-16 · 7:02 PM CT

OpenAI lets outside assessors inside — with limits

Paul Derrington, Jr.

By Paul Derrington, Jr.

CEO | Chief Engineer | Research Engineer | Editor, D-CSIL

3 min read

Follow @PaulDerrington on X

Key Takeaways

OpenAI signed four outside groups — Apollo Research, METR, Transluce and Goodfire — to assess its models during training and internal use, not just before release. The access is real but scoped, and Apollo says it does not yet meet its own bar for a full embedded evaluation. OpenAI pays Apollo and Goodfire regardless of results; METR and Transluce accept no financial compensation.

What OpenAI signed on Monday

OpenAI said on November 16 that it has signed agreements with four independent assessors — Apollo Research, METR, Transluce and Goodfire — for a first phase of what it calls embedded assessments. Instead of testing a finished model shortly before release, the assessors get deep access to internal models and systems over an extended period, to assess alignment and the effectiveness of safeguards during training and internal use.

Depending on what an evaluation needs, OpenAI says assessors can receive some of the same internal resources its own researchers use: model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation, and discussions with the researchers and engineers responsible for the work. OpenAI says the work supports its commitments under the White House Accord on Super Intelligence, where it agreed to independent assessments of its safety controls.

Four assessors, four different jobs

Apollo Research has the most specific brief: whether models could covertly work against their developers' intentions, and whether OpenAI's safeguards would detect it. Apollo will test four claims in two stages — training did not incentivize subversion; strong evaluations and red teaming do not elicit it; models have not attempted it during internal use; and reasoning behind subversion would be detectable. Apollo says six of its evaluators will work within OpenAI for four months and that it will publish two public reports.

METR will assess misaligned behavior in what OpenAI calls its largest reinforcement learning run to date, including failures existing tests and monitors may have missed; a second, broader monitor assessment is preliminarily agreed but still pending agreement on scope. Goodfire will embed interpretability researchers to analyze model internals, mainly to inform OpenAI's own researchers, and will publish learnings as the work progresses. Transluce will examine the quality and coverage of OpenAI's post-training alignment evaluations, and the two sides intend a follow-on assessment of about eight weeks in early 2027.

On payment: Apollo and Goodfire accept financial compensation that is not contingent on their assessment results. METR and Transluce do not accept financial compensation, though they may accept API credits or compute for use in their assessments.

The limits are part of the story

Apollo is explicit that this pilot does not yet meet its own criteria. It says a credible embedded evaluation needs access to all data relevant to the assessment, the ability to disclose findings to an outside audience, and the right to explain the constraints on the work — and that this pilot fulfills only part of that, though it meets Apollo's bar for explaining its limits publicly, which Apollo calls meta-transparency. Apollo calls the pilot "an ambitious first step" towards deeper access and disclosure rights.

In practice, Apollo's access covers only the agreed claims, and only for the specific models and time periods agreed for the assessment. OpenAI may withhold some in-scope material where it deems it reasonably necessary because of technical infeasibility, security risks or IP protection — but it has agreed not to withhold information, or request redactions, simply because findings are unfavorable. OpenAI can redact confidential evidence from public reports, but Apollo says OpenAI cannot redact Apollo's verdicts, and OpenAI's Safety and Security Committee will receive the unredacted reports. Redwood Research CEO Buck Shlegeris was blunter in a post quoted by TechCrunch on Monday, saying the agreements "obviously don't meet that bar" for employee-like access.

There are no findings yet. This is an announcement about who gets to look, and under what rules — not a result. What to watch is whether Apollo's two reports can show their evidence, whether the METR monitor assessment gets an agreed scope, and whether later phases widen access toward the employee-like standard Apollo set out.

Disclaimer: Derrington Collaborative AI is my personal business, operated independently of and outside my official U.S. Government position and duties. The views and opinions expressed on this site and on any and all of my social media accounts are my own. They do not reflect the official policy or position of the Department of the Army, the Department of Defense, or the U.S. Government. Mention of any company, product, or service does not constitute an official endorsement.