Anthropic CEO's Stark Warning About China

Anthropic co-founder and CEO Dario Amodei has called for sanctions on China, calling it the "most capable threat." In an official blogpost posted on Monday, Amodei further fuelled speculation that China may be distilling powerful American frontier AI models such as Anthropic's Mythos to power open-weight models such as the Kimi K3 which has stunned the world with its capabilities. Simply put, frontier models are the most advanced AI systems currently available, while open-weight models give developers access to the model's underlying weights, making it possible to customise and run them independently rather than through the company's own service.

"My primary concern is the risk that authoritarian governments-not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat-build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people," Amodei wrote.

Last week, the US government's Office of Science and Technology Policy (OSTP) Director Michael Kratsios alleged that China's Moonshot AI used Anthropic's powerful Fable model to develop its K3 model. 

"We have information that Moonshot AI distilled Anthropic's Fable for the development of its K3 model," Kratsios wrote on X.

He explained: "To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection."

AI distillation, also known as knowledge distillation, is a technique in which a large, complex and computationally expensive model, known as the "teacher", trains a smaller and faster model.

Also read: US Alleges China's Moonshot Used Anthropic To Build Powerful AI

Amodei in his blog urged a crack down on industrial-scale distillation operations. "Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans."

US President Donald Trump's tech advisor highlighted that legitimate AI distillation used to create smaller, more efficient models is fine and plays a vital role in this open innovation ecosystem. "However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable."

Amodei opines that distillation does not allow China to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to "within a few months of the US frontier."

"It is true that many of the companies carrying out these operations release open-weights models-but the open weights are far less relevant than the fact that the operations are backed by an authoritarian state seeking to overtake the US at the frontier," he wrote.

Choke China With No Chips

Amodei strongly urged the US to not sell "powerful chips or chipmaking equipment to China." 
"We should crack down on the rampant smuggling and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips."

The Bio/ Pandemic Threat

In the wake of the US threatening to shut down Chinese open-weight models, around 200 startups from Silicon Valley including the likes of YCombinator wrote an open letter urging the Trump administration to not ban access as it could risk crippling the next generation of US startups. The recently-formed Little Tech Association sent letters to President Donald Trump, Commerce Secretary Howard Lutnick and others in the US administration with its appeal.

Amodei in principle agrees with the group's demand and is himself not in favour of a ban.

"I agree with much of it (the open letter): open weights expand access to the AI economy, they strengthen competition at least for some use cases, and they give customers greater control."

However, Amodei doesn't agree with all of the claims in the letter.

"But I don't agree with the letter's assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true."

He warned of the possibility of these models sparking another pandemic through the easy weaponisation of biology through these models.

"I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed). Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance."

Launched by the US government in May 2020, the public-private partnership dramatically accelerated the development, manufacturing and distribution of COVID-19 vaccines, therapeutics and diagnostics. Despite unprecedented funding, regulatory support and collaboration between government agencies and pharmaceutical companies, it still took months to produce and roll out vaccines at scale. Amodei argues that while AI could dramatically shorten the time it takes a malicious actor to engineer a dangerous pathogen, mounting a defence against such a threat would remain a complex, resource-intensive effort that cannot be compressed to the same degree.

China Fires Back

In the backdrop of the US threats China has warned it would take "all necessary measures" if the US moves to sanction Chinese artificial intelligence companies over allegations they improperly used American models to train their own systems.
In fact, China has hit back with its own allegation: "It is understood that many American AI companies have distilled Chinese models in their research and training," China's Ministry of Commerce alleged.