Anthropic Claims Alibaba Ran a Model Distillation Attack on Claude
In a June 10 letter to Senators Tim Scott and Elizabeth Warren, Anthropic accused Alibaba of a large-scale effort to clone Claude using model distillation, a technique that copies an AI by asking it millions of questions. Here's what the evidence shows, what remains unverified, and what it means for
On June 10, 2025, one day before a Senate hearing titled "AI and the American Dream," Anthropic sent a confidential letter to Senators Tim Scott and Elizabeth Warren accusing Alibaba of a large-scale attempt to clone its Claude model. Ars Technica obtained and reported on the letter. The alleged method is model distillation: bombarding a target AI with millions of queries, recording its answers, and training a copycat model on those responses. Important caveat: the specific claims rest on a confidential letter, Anthropic has not publicly released its underlying query-log evidence, and Alibaba had not issued a detailed rebuttal at the time of reporting. Treat the accusation as one company's allegation, not a proven fact.
The June 10 Letter: What We Actually Know
Anthropic sent its letter to Senators Tim Scott and Elizabeth Warren on June 10, 2025, one day before a Senate hearing titled "AI and the American Dream." Ars Technica obtained the letter and published its contents. In it, Anthropic accused Alibaba of running what it described as a large-scale effort to clone Claude and copy its capabilities.
Here's what is documented versus what is asserted:
- Documented: The letter exists, was addressed to two named senators, and was timed against a specific hearing. Ars Technica reviewed it. - Asserted by Anthropic: That Alibaba specifically was behind industrial-scale automated querying, and that this matched a distillation attack pattern. - Not yet public: The raw query logs, IP attribution details, or any independent forensic confirmation. Anthropic classified the letter as confidential and has not released technical proof. - Missing at time of reporting: A detailed public rebuttal from Alibaba, which had not confirmed or denied the specific claim.
Why senators and not a lawsuit? Anthropic framed the incident as a national security matter, not merely an IP dispute. That framing is itself a signal: it shifts the conversation toward export policy rather than a courtroom where evidence would have to be disclosed. Read the accusation carefully and note where proof is offered versus where it is described.
What makes distillation different from classic espionage: nobody needs to breach a data center or steal source code. Claude is a public product. Anyone with an account can send it prompts. The alleged attack weaponized that openness at scale.
How Model Distillation Works, Step by Step
Model distillation is a legitimate, well-documented machine-learning technique. It becomes an attack when used to copy a competitor's model against its terms of service. Here's the process:
1. Point automated software at the target AI (Claude) through its normal API or app. 2. Fire large volumes of carefully chosen prompts covering coding, reasoning, writing, and math. 3. Record every response. These become the training "answer key." 4. Fine-tune a smaller student model to imitate those answers as closely as possible. 5. The student model inherits much of the teacher's behavior, without anyone ever seeing how the teacher was built.
Analogy: it's like copying a chef's entire menu by ordering every dish thousands of times and reverse-engineering the recipes from taste. You never enter the kitchen.
Scale is the whole game. Distillation improves with more query-response pairs, so a serious effort typically involves millions to tens of millions of prompts. Public research on distillation (including DeepSeek-style work and academic "stealing" papers) has shown distilled students can recover a large share of a teacher's benchmark performance at a fraction of the training cost, sometimes cutting a multi-month capability gap dramatically.
Be honest about limits: a distilled model is never a perfect clone. It inherits behavior, not internal weights, and it can also inherit the teacher's mistakes. Measuring exactly how much capability was copied is genuinely hard, and Anthropic's letter does not publicly quantify it with a hard percentage.
Why Export Bans and Chip Restrictions Don't Stop It
A common assumption is that US export bans and chip restrictions keep American AI safe. For distillation, that assumption is largely wrong.
Distillation sidesteps hardware controls. You don't need the target's servers or the fastest Nvidia chips to copy behavior. You need two things: access to the finished model (a public API provides this) and enough compute to fine-tune a smaller student (cloud credits or modest local clusters provide this). Neither requires the restricted top-tier hardware the bans target.
There's a counterintuitive lesson for regular users. The more helpful an AI is, the more valuable its answers are to a copier. Claude's detailed step-by-step reasoning, the exact thing that makes it useful, is also the richest training signal for a distillation attack. Helpfulness is the leak.
How AI companies defend against this today (concrete technical mechanisms):
- Rate limiting and volume anomaly detection to flag industrial-scale automated querying. - Per-account and per-key monitoring for query patterns that don't match human use. - Output watermarking and canary responses to detect when outputs surface in a competitor's model. - Terms-of-service clauses that explicitly ban using outputs to train competing models. - Response-shaping, such as limiting exposure of full chain-of-thought reasoning by default.
These are the same signals Anthropic says it used to detect the alleged campaign.
What This Means for You, and What to Do This Week
You're not Anthropic, so why care? Because distilled clones are cheaper, less controlled, and often stripped of safety guardrails. A model trained on a frontier AI's answers can be repackaged into scam tooling, disinformation engines, or fraud automation with none of the refusal behavior the original had.
If you're a regular user, do this:
- Verify identity through a second channel before acting on any AI-influenced financial request. Assume AI-generated text, voice, or code could come from an unrestricted clone. - Never paste passwords, secrets, or proprietary code into an AI you cannot verify. A cloned or fly-by-night model may log everything you send. - Prefer vendors that publish data-retention, safety, and provenance policies. Drop tools that don't.
If you're a developer or run an AI-powered product, do this to defend against distillation:
- Set strict rate limits and per-key quotas, and alert on abnormal query spikes. - Log and cluster query patterns; industrial-scale scraping looks statistically different from human use. - Watermark or fingerprint your outputs so you can later detect them in a competitor's model. - Add terms-of-service language explicitly prohibiting output-based training, and enforce it. - Limit unnecessary exposure of full reasoning traces where product needs allow.
Treating every chatbot as equally trustworthy is a real risk. A brand-name model with visible guardrails is not the same as a cheaper, unverified knockoff.
Key Takeaways
FAQ
Q: Can someone really copy an AI just by asking it questions?
A: Yes, through model distillation, where you record millions of an AI's answers and train a smaller model to imitate them. The copy won't be identical and can inherit the original's errors, but it can absorb much of the original's coding and reasoning ability without ever seeing its underlying code or weights.
Q: How solid is Anthropic's evidence against Alibaba?
A: As of the reporting, the public evidence is a single confidential letter to two senators, obtained by Ars Technica. Anthropic has not publicly released query logs, IP attribution, or independent forensic confirmation, and Alibaba had not issued a detailed public rebuttal. It should be treated as a serious allegation, not a proven fact.
Q: Why did Anthropic go to senators instead of filing a lawsuit?
A: Anthropic framed the incident as a national security issue rather than only an intellectual property dispute. That routes the conversation toward export and AI policy. It also avoids, at least initially, a courtroom where Anthropic would have to disclose its underlying technical evidence.
Q: If Claude is a public product, isn't distilling it just fair use?
A: It's legally murky, which is partly why Anthropic went to senators rather than only to court. Most AI terms of service explicitly ban using outputs to train competing models, so mass distillation likely breaches those contracts even where copyright law is unsettled.
Q: What can developers actually do to stop distillation attacks?
A: Set aggressive rate limits and per-key quotas, monitor for abnormal query volumes and non-human patterns, watermark or fingerprint outputs so they can be detected later in a rival model, add and enforce terms-of-service clauses banning output-based training, and limit unnecessary exposure of full reasoning traces.
Q: How do I protect my own data from a possibly cloned AI?
A: Never enter passwords, financial details, or proprietary code into any AI you can't verify as a legitimate vendor. Start today by checking whether your AI tool publishes a data-retention and safety policy, and stop using it if it doesn't.
Conclusion
Anthropic's June 10 letter is a serious allegation, but as of now it rests on confidential evidence rather than public proof, and Alibaba's side is largely absent. What is not in doubt is the technique: model distillation lets someone copy an AI's behavior by interrogating it, and export bans don't stop it. Today, developers should harden their APIs with rate limits, monitoring, and output watermarking, while everyone else should audit which AI tools they trust and stop pasting anything sensitive into the ones they can't verify.
Related Posts
- How Does AI Help Cybersecurity Teams β And How Do Attackers Abuse the Same Tools?
The same AI tools that help security teams detect threats in milliseconds are being weaponized to clone voices, generate perfect phishing emails, and impersonate executives on live video calls. This isn't a future risk β it already cost one company $25 million in a single afternoon. Here's exactly w - How Are AI Deepfakes Used in Romance Scams?
Romance scammers are now using real-time AI deepfake video and cloned voices to impersonate attractive strangers β and sometimes even your own family members. The technology costs less than $20/month and is shockingly convincing. Here's what the attack looks like and how to protect yourself today. - How Are AI Deepfakes Targeting Your College?
AI voice clones and deepfake video calls now impersonate students and university leaders to steal financial aid, redirect tuition, and crack into campus accounts. A 15-second voicemail of you is enough. Here's how the scam runs and how to shut it down.
Also on AI Future Lab
- β οΈ Verification: lanthanum decahydride β Paper vs Simulation [2026-06-30]
We tested lanthanum decahydride: paper claims 250 K, our simulation predicts 188K. Here's what the gap tells us. - β οΈ Verification: LaHββ β Paper vs Simulation [2026-06-26]
We tested LaHββ: paper claims 260K, our simulation predicts 250K. Here's what the gap tells us.