Rao predicts post-Transformer architectures will drive efficiency gains
Naveen Rao argues that physical efficiency gains from Moore's Law have stalled, necessitating a shift to co-designed models and hardware. He forecasts a $10 trillion inference market within a decade, driven by reinforcement learning.
Naveen Rao asserts that physical efficiency gains from Moore's Law and Dennard scaling have largely stopped, with recent improvements coming from smaller data types and architectural changes. He advocates for a "post-Transformer" world where models and hardware are co-designed for energy efficiency rather than speed, arguing that current Transformer architectures are inefficient for reasoning. Rao predicts a shift from pre-training to reinforcement learning as the primary method for improving models, noting that while reinforcement learning is more compute-inefficient, it is necessary for solving tasks where data is scarce.
Rao believes the inference market will reach $10 trillion within 10 years, with 80% of token traffic flowing through open models in three years. He expects frontier models to capture 80% of the margin, similar to the iPhone market share dynamic. This outlook suggests that the next order of magnitude in efficiency will come from specialized hardware and architectural innovation rather than incremental scaling of existing models.