LangChain Founder on Agent Economics
Harrison Chase of LangChain analyzes the shifting economics of AI agents, arguing that value accrues to vertical domain experts who control context and harnesses. He outlines a three-step framework for managing soaring inference costs and predicts the rise of specialized memory layers.
The Shift to Vertical Agent Economics
Harrison Chase, founder of LangChain, argues that the AI agent market is transitioning from a race for general-purpose models to a battle for domain-specific value. While model capabilities improve rapidly, the core challenge remains non-determinism: agents are black boxes that require robust harnesses to behave reliably. Chase posits that the highest value will accrue to "domain experts" who can quantify success through proprietary evaluation metrics (evals) and verifiers, rather than to generalist infrastructure providers.
Strategic Implications for Builders
For entrepreneurs, the recommendation is clear: start at the harness layer. Context engineering and harness optimization offer higher immediate ROI than post-training models. As models improve, parts of the product will become obsolete, requiring companies to "erase" features that models now handle natively. However, the system connecting models to proprietary data, APIs, and memory will persist. This creates a durable moat for companies that build neutral, portable harnesses, allowing them to switch between frontier models without losing accumulated context or memory.
Managing the Inference Cost Crisis
A critical operational challenge is the skyrocketing cost of inference. Chase notes that LangChain spends roughly 30% of its engineering salary on coding tools, a figure that has stabilized despite increased token usage. This was achieved through a three-step framework: 1) Visibility (tracking usage via observability tools), 2) Cost Control (setting hard budgets), and 3) Harness Optimization (using prompt caching and intelligent routing). He emphasizes that routing should occur at the harness level, not the gateway, to leverage state awareness and prompt cache efficiency.
Investment Outlook
Chase is bullish on inference providers, vertical agents, and low-level infrastructure primitives like sandboxes and databases. He advises against investing in crowded customer support AI or robotics, where funding exceeds current capabilities. He highlights that "owning your intelligence" involves controlling the context, harness, and eventually the model weights. The future of enterprise AI lies in vertical agents that interact with horizontal assistants, creating a layered ecosystem where domain-specific memory and logic drive competitive advantage.
Key insights
-
The value in the agent stack is shifting from the model layer to the harness and context layers, where domain-specific logic and data reside. Generalist models are becoming commodities, while proprietary context engineering creates defensible moats.
Impact: Startups should prioritize building vertical-specific harnesses and evals over general-purpose frameworks to capture higher margins and stickiness.
-
Inference costs are rising faster than they are falling due to the complexity of multi-agent systems. Token spend is now a primary line item in engineering budgets, comparable to human salaries in some cases.
Impact: Companies must implement rigorous cost visibility and control mechanisms to prevent runaway agent loops from eroding profitability.
-
Model routing is most effective when implemented at the harness level rather than the gateway level. Harness-level routing allows for state-aware decisions, prompt cache optimization, and domain-specific model selection.
Impact: Infrastructure providers focusing on gateway-only routing may find their value proposition limited to simple, horizontal tasks like general chat.
-
Memory is the most unsolved problem in agent development. General-purpose memory solutions fail because memory requirements are highly domain-specific, requiring vertical-specific learning agents.
Impact: Investors and builders should look for opportunities in vertical-specific memory layers that can distill user interactions into actionable context.
-
Post-training open-weight models is currently most effective for simpler sub-tasks like classification and extraction, not for the core agentic loop. Frontier models still dominate the core loop, but this may change as post-training techniques improve.
Impact: Companies should use post-trained models for cost-efficient sub-tasks within their agent harnesses while relying on frontier models for complex reasoning.
Action items
-
Implement a three-step cost control framework for AI agents: establish visibility into token usage, set hard budget limits, and optimize the harness for efficiency via prompt caching.
Impact: This approach can stabilize AI spend as a percentage of engineering costs, preventing budget overruns from inefficient agent loops.
-
Build agent architectures that decouple context and memory from specific model providers to ensure portability.
Impact: This prevents vendor lock-in and allows companies to switch to the best-performing models as they emerge, maintaining a competitive edge.
-
Prioritize context engineering and harness optimization over model post-training when starting a vertical agent project.
Impact: This yields higher immediate ROI and allows for faster iteration before investing in the more expensive and complex process of model fine-tuning.
-
Develop proprietary evaluation metrics (evals) and verifiers specific to your domain to measure agent performance accurately.
Impact: Domain-specific evals are the key to quantifying agent value and improving performance over time, creating a durable competitive advantage.
-
Focus investment and development efforts on vertical agents and low-level infrastructure primitives rather than crowded horizontal AI applications.
Impact: Vertical agents offer clearer paths to monetization through domain expertise, while infrastructure primitives benefit from the growing demand for reliable agent execution environments.
Quotes
“The main way our business changes as the model gets better is we basically have to erase big parts of our product.”
“I think the reason we need neutral harnesses is that there is a lot of state that you build up either before you build the agent or as part of the agent running.”
“I think there's a tremendous amount of attention in that domain, and so I think absolutely this will catch up.”