Data and RL environments become critical bottlenecks for model progress
Nathan argues that high-quality data and reinforcement learning environments are the primary levers for capability gains in non-verifiable domains. He predicts a multi-year transition as labs move to in-house data creation to address current supply chain flaws.
Nathan identifies data and reinforcement learning environments as the primary levers for capability gains in non-verifiable domains such as knowledge work, biology, and legal. He argues that frontier labs are spending billions on high-quality data and synthetic environments, but the current supply chain is flawed, characterized by low hit rates and issues like reward hacking caused by models exploiting test file errors. Labs are increasingly taking data creation in-house to prevent signal leakage, yet external purchases remain massive. The data industry is described as having 'low hanging fruit' in quality and operational security, with contractors often unaware they are selling data. Nathan predicts a multi-year transition where winners in the data space will improve quality and return on investment. He also discusses the economic dynamics of inference, noting that model prices are dropping two times every eight months, which could lead to either higher margins via Jevin's paradox or decreased revenues as the market fills. He emphasizes that the open-model economy takes longer to build due to fragmented information flow between model builders, inference providers, and post-training companies, whereas closed labs have integrated advantages in serving efficiency and revenue capture.