Skip to content
D-CSIL™

AI News · 2026-11-22 · 7:02 AM CT

OpenAI locks down its models' reasoning

Paul Derrington, Jr.

By Paul Derrington, Jr.

CEO | Chief Engineer | Research Engineer | Editor, D-CSIL

3 min read

Follow @PaulDerrington on X

Key Takeaways

OpenAI published its playbook against adversarial distillation on November 20: attackers using a frontier model's outputs to train a copy without its safeguards. On GPT-6.2 Astra, its updated defenses cut one key extraction attack from a 54.0% success rate to 0.2%. All of the headline figures are OpenAI's own internal results, with no independent verification yet.

A person in a hoodie coding on dual monitors at night, depicting cybersecurity work
Photo: Julio Lopez / Pexels

What OpenAI changed

Distillation is a normal technique — one model learns from another model's answers, reasoning, or judgments. OpenAI's November 20 post, “Combatting adversarial distillation,” is about the unauthorized version: an actor trains a rival model on a frontier model's outputs and deploys it without comparable protections, so it may help with attacks the original would have refused.

The defenses work at two levels. At the request level, adversarial training teaches the model to refuse extraction attempts while still answering legitimate questions, input classifiers flag attempts to pull out protected reasoning, and output classifiers catch leaks in assistant messages and in tool-call arguments — a leak path OpenAI stresses, because reasoning can slip out in a tool's arguments without appearing in the final answer. At the API level, GPT-6.2 Astra and Sol now tie protected reasoning to the conversation history that produced it: add messages and the model builds on its earlier reasoning, but rewrite an earlier message and the API drops the reasoning that followed it. Older model families with weaker safeguards are also blocked from reading GPT-6.2's protected reasoning.

Why the reasoning traces are the prize

OpenAI puts the weight on protected reasoning — the intermediate chain-of-thought a model generates before answering, which can contain information the final answer leaves out. In an internal cyber study, the company trained two student models from the same starting point on coding and defensive cybersecurity work: one learned from the content users normally see (final responses, reasoning summaries, tool interactions), the other also got the teacher's exact reasoning traces. After supervised fine-tuning, the exact-trace student generally scored higher; after a light round of reinforcement learning, it still led at the largest inference budget tested on both benchmarks shown, while the summary-trained student led at lower budgets.

The measured result: in adaptive black-box red-teaming aggregated across six attacker models, attempts to recover reasoning from an earlier request succeeded 54.0% of the time against GPT-6 Astra with its original safeguards and 0.2% against GPT-6.2 Astra with the updated ones. For reasoning from the current request, the rate fell from 22.9% to 0.9%. OpenAI says it is now applying selected improvements to GPT-6 Astra as well. These are the company's internal numbers — the six attacker models are not named and no independent group has reproduced them.

What happens next, and what builders should watch

Beyond single requests, OpenAI says it hunts coordinated campaigns: it reports multiple unauthorized distillation campaigns this year, weekly enforcement ranging from thousands to tens of thousands of accounts — rising sharply over the summer — and sharing indicators of compromise with other providers. The post also carries a policy ask: OpenAI told Congress it supports the Chip Security Act, the AI OVERWATCH Act, and the MATCH Act, and it recounts briefing Congress in February on activity it attributed to DeepSeek, plus a September campaign it attributed in core part to individuals associated with Moonshot AI, the developer of Kimi.

For anyone building on the API, the practical change is the history rule: agent harnesses that edit, prune, or replay earlier messages — a common way to save tokens — can now silently lose the reasoning state attached to those messages. OpenAI's advice to other developers is plain: check tool-call arguments for leaks, keep successful extraction attacks as regression tests, and pair request-level controls with campaign-level investigation. Whether rivals adopt the same history-bound approach is the open question this post puts on the table.

Disclaimer: Derrington Collaborative AI is my personal business, operated independently of and outside my official U.S. Government position and duties. The views and opinions expressed on this site and on any and all of my social media accounts are my own. They do not reflect the official policy or position of the Department of the Army, the Department of Defense, or the U.S. Government. Mention of any company, product, or service does not constitute an official endorsement.