TL;DR
Knowledge distillation is widely used in machine learning. House hearings, a White House memo, and a joint NSA/FBI/CISA advisory address allegations of unauthorized extraction, distinct from the legitimate technique. Here is what the public record supports, what vendor benchmarks show, and our view of the policy stakes.
What distillation actually is
Knowledge distillation trains a student model using a teacher model’s outputs. Some methods use probability distributions; others use generated answers. It can produce smaller, cheaper models, with capability depending on the task and training.
Geoffrey Hinton and colleagues described the technique in their 2015 paper, Distilling the Knowledge in a Neural Network. Knowledge distillation is widely used in machine learning.
What’s on the record
On April 16, 2026, the House Select Committee on the CCP held a hearing on China’s campaign to extract American AI capabilities, with testimony describing unlawful model distillation techniques. A week later, on April 23, the White House Office of Science and Technology Policy issued memo NSTM-4 on adversarial distillation of American AI models.
According to contemporaneous April 30, 2026 courtroom reporting by WIRED and TechCrunch, Musk answered ‘Partly’ when pressed on whether xAI had distilled OpenAI models. This account relies on contemporaneous courtroom reporting.
The NSA announced a joint NSA/FBI/CISA advisory on September 8, 2026, alleging industrial-scale extraction of restricted capabilities from U.S. frontier models. WaterISAC’s September 10 summary of advisory AA26-251A names DeepSeek, Moonshot AI, Alibaba, MiniMax, StepFun, and Z.AI. These are key parts of the public record.
These are allegations, not court findings. The House committee explicitly distinguishes legitimate distillation from alleged fraudulent accounts, proxy networks, access evasion, and terms violations. The White House’s April 23 memo likewise distinguishes legitimate use from unauthorized industrial extraction.
The technical reality
OpenAI’s 2023 research found that strong pretrained models fine-tuned on weaker models’ labels could outperform those weak supervisors. That differs from compressing a large teacher into a smaller student. DeepSeek-R1-Zero demonstrated reasoning emerging from reinforcement learning without supervised fine-tuning; the released DeepSeek-R1 added supervised stages.
DeepSeek-V3’s contributions included FP8 mixed-precision training; its multi-head latent attention came from V2. Alibaba Cloud reports 92.6 on GPQA Diamond for Qwen3.8-2.4T-A95B. Z.ai’s GLM-5.2 announcement reports 62.1 on SWE-bench Pro, with GPT-5.5 at 58.6 in its comparison table. These are vendor-published figures, not independently verified for this briefing, and they do not establish general model superiority or resolve provenance allegations.
What it means for you
Distillation can help reduce inference costs by transferring useful behavior into smaller models. Open-weight distilled models can provide an alternative to relying entirely on hosted model services, although capability and hardware requirements vary.
Our view: broadly treating distillation itself as theft could deepen incumbents’ moats and raise the cost of access to AI. Enforcement against unauthorized extraction is different from banning a legitimate engineering technique. The House committee, White House memo, and NSA announcement acknowledge that distinction. Watch how policy affects legitimate distillation and access to open weights; those choices could shape which smaller models remain available.