Chinese labs use distillation to offset compute deficits against US rivals

Nathan argues that model progress is driven by three forms of capital: compute, data, and talent. While US frontier labs like OpenAI and Anthropic possess the most of all three, Chinese labs are technically excellent and possess a talent pool comparable to US peers, but are severely compute-starved. An average OpenAI researcher has access to 5,000 H100 equivalents, a figure that shocks Chinese pre-training researchers. To bridge this gap, Chinese labs utilize distillation, which Nathan describes as a way to spend money to acquire compute-equivalent outputs. He notes that the lag between open-source and frontier models has shortened to two to three months.

Although public literature makes it difficult to map the exact role of distillation in current reinforcement learning-heavy post-training, Nathan observes that Chinese companies use Claude and OpenAI model outputs at vast scales via proxies and intermediaries. He argues that while distillation is not a "huge effect" on current post-training based on available scientific literature, it provides a directional advantage. He also highlights that Chinese labs are building models to capture the US market, using open-weight releases as a practical commercialization strategy rather than altruism, and that they have a more serious approach to security due to government dynamics.