May 01, 2026 · Y Combinator Startup Podcast · 5 min read

Recursive AI Models Outperform Scaling Laws

An analysis of Hierarchical and Tiny Recursive Models demonstrating that inference-time recursion and latent memory outperform parameter scaling. These architectures achieve state-of-the-art results on complex reasoning tasks with a fraction of the compute, signaling a shift in AI development strategy.

Technology Artificial Intelligence Software Engineering

The takeaways

Recursion Beats Parameter Scaling. Tiny Recursive Models achieve 87% accuracy on ArcPrize 1 with only 7 million parameters, outperforming massive LLMs trained on the entire internet. This proves that architectural depth via recursion is more efficient than brute-force parameter expansion.
Latent Memory Solves Incompressible Tasks. Standard LLMs fail at incompressible problems like sorting due to lack of external memory. Recursive models utilize continuous latent states as a memory tape, enabling complex algorithmic reasoning that one-shot feed-forward networks cannot perform.
Outer Refinement Loops Drive Performance. The primary performance gain in hierarchical reasoning comes from outer refinement loops rather than deep internal layers. Implementing iterative self-correction at inference time yields significant accuracy improvements without increasing model size.
Truncated Backprop Enables Stability. Using truncated backpropagation through time with a depth of one prevents vanishing gradients while maintaining training stability. This technique allows models to learn recursive behaviors without the computational overhead of full history retention.
Chain of Thought Is Bounded. Chain-of-thought prompting is limited by the model's pre-existing knowledge and discrete token space. Recursive latent reasoning allows models to discover new algorithms from first principles, surpassing the constraints of human-labeled training data.
Hybrid Architectures Are The Future. The optimal path forward combines the embedding capabilities of large LLMs with the reasoning efficiency of tiny recursive models. Integrating recursive modules into existing transformer architectures will unlock scalable, high-performance reasoning agents.

The Shift from Scale to Recursion

The prevailing strategy in AI development has relied on increasing parameter counts to improve reasoning capabilities. However, recent research into Hierarchical Reasoning Models (HRM) and Tiny Recursive Models (TRM) challenges this paradigm. These architectures demonstrate that inference-time recursion, rather than model size, is the critical driver for solving complex, incompressible problems. A 7-million-parameter TRM achieved 87% accuracy on ArcPrize 1, outperforming models with orders of magnitude more parameters trained on vast datasets. This indicates that the industry is over-investing in brute-force scaling while underutilizing architectural efficiency.

Overcoming LLM Limitations

Standard Large Language Models operate as one-shot feed-forward networks, lacking the external memory required for algorithmic tasks like sorting or maze navigation. They are bounded by their layer depth and cannot perform iterative computations within a single forward pass. Recursive models address this by maintaining continuous latent states that function as a memory tape. This allows the model to store intermediate results and refine its reasoning iteratively, effectively simulating a Turing machine within the latent space. This capability enables the discovery of new algorithms from first principles, a feat impossible for chain-of-thought prompting which relies on pre-existing human knowledge.

Strategic Implications for AI Development

The success of TRMs and HRMs suggests a fundamental shift in AI engineering. The key innovation is the use of truncated backpropagation through time and outer refinement loops, which stabilize training while allowing for deep recursive reasoning. For enterprises, this means that high-performance reasoning agents can be built with significantly lower computational costs. The future of AI likely lies in hybrid architectures that combine the rich embedding spaces of large LLMs with the efficient, recursive reasoning modules of tiny models. This approach offers a scalable path to advanced reasoning without the prohibitive energy and infrastructure costs associated with current scaling laws.

Key insights

Tiny Recursive Models outperform massive LLMs on complex reasoning benchmarks using a fraction of the parameters. This demonstrates that architectural efficiency via recursion is superior to brute-force scaling for specific cognitive tasks.

Model Architecture →

Impact: Reduces computational costs for deploying high-performance reasoning agents, making advanced AI accessible to smaller organizations.
Standard LLMs fail at incompressible problems because they lack external memory mechanisms. Recursive models solve this by using continuous latent states as a memory tape for iterative computation.

Algorithmic Reasoning →

Impact: Enables AI systems to perform complex algorithmic tasks like sorting and pathfinding that were previously impossible for single-pass neural networks.
Outer refinement loops are the primary driver of performance gains in recursive models. Iterative self-correction at inference time yields significant accuracy improvements without increasing model complexity.

Inference Optimization →

Impact: Provides a low-cost method to boost model accuracy by simply increasing inference-time compute, rather than retraining larger models.
Truncated backpropagation through time with a depth of one is sufficient for training stable recursive models. This technique avoids vanishing gradients while maintaining the benefits of recursive learning.

Training Methodology →

Impact: Simplifies the training process for recursive architectures, making them easier to implement and scale in production environments.
Chain-of-thought prompting is limited by the model's pre-existing knowledge and discrete token space. Recursive latent reasoning allows models to discover new algorithms from first principles, surpassing the constraints of human-labeled data.

Reasoning Capabilities →

Impact: Unlocks the potential for AI to innovate and solve novel problems without relying on historical human data, moving closer to true artificial general intelligence.

Action items

Evaluate recursive model architectures for specific reasoning tasks in your AI stack. Identify use cases where iterative computation and memory retention are critical, such as complex planning or algorithmic problem-solving.

Impact: Identifies opportunities to replace expensive, large-scale LLMs with efficient, specialized recursive models, reducing operational costs.
Implement outer refinement loops in your inference pipeline to allow models to iteratively refine their outputs. This can be achieved by increasing inference-time compute without modifying the model weights.

Impact: Improves model accuracy and reliability for critical tasks by leveraging iterative self-correction, leading to better decision-making outcomes.
Investigate hybrid architectures that combine large LLM embedding spaces with tiny recursive reasoning modules. This approach leverages the strengths of both architectures for scalable, high-performance reasoning.

Impact: Creates a robust AI infrastructure capable of handling both broad semantic understanding and deep, iterative reasoning, future-proofing your technology stack.
Adopt truncated backpropagation through time for training recursive models to ensure stability and efficiency. This technique should be integrated into your training pipelines to avoid vanishing gradient issues.

Impact: Accelerates model training and reduces computational overhead, enabling faster iteration and deployment of recursive AI systems.
Benchmark your current LLMs against recursive models on incompressible tasks like sorting or maze navigation. This will highlight the limitations of your current architecture and quantify the potential gains from adopting recursive methods.

Impact: Provides data-driven insights into the performance gaps of your current AI systems, justifying the investment in new architectural approaches.

Quotes

“It's actually impossible for the model to map from unsorted list to sorted list. In a one shot basically.”

“The outer refinement loop scales, and there's a great breakdown. Basically, the sapient authors, which is huge kudos for this paper because there's so many innovations in this paper, didn't really do like scaling ablations on every single one of the inputs, but this guy Constantine at François Chalet's company, India, actually did.”

“The right answer is to take the amazingness here and take the amazingness here, which probably is already in Gemini already or some of these, it might be at least in some part. But when you take the benefit of both these TRMs and these giant models and you actually slam them together, I think that it's just going to take off.”