Anthropic's Product Strategy: Evals, Labs, and AI Leadership
Diane Penn, Head of Product at Anthropic, reveals how frontier models require frontier products. Key insights include evals replacing PRDs, the strategic value of token spending, and building autonomous labs for discontinuous innovation.
Anthropic's rapid ascent from a niche startup to a $50 billion ARR powerhouse underscores a fundamental shift in AI product development, where speed, culture, and first-principles thinking dictate market leadership. The company's trajectory reveals that frontier models alone are insufficient; they require equally frontier products to unlock user value. The synergy between Opus 4.5 and Claude Code exemplifies this, demonstrating that model capabilities and product vehicles must evolve in tandem to accelerate adoption and create inflection points.
The Evolution of Product Management
Traditional product management artifacts are being redefined. Diane Penn highlights that "evals are the new PRDs," emphasizing that product managers must now construct rigorous evaluation sets to translate user feedback into actionable research directives. This shift demands a move away from pattern-matching legacy SaaS workflows toward dynamic, data-driven validation loops that measure emergent model capabilities. Product leaders must sweat the tokens as much as pixels, diving deep into failure trajectories to create sustained descriptions of pain points that researchers can directly address. This approach shortens the feedback loop between user experience and model training, ensuring continuous improvement.
Strategic Experimentation and Labs
Anthropic's Labs division operates on a thesis of discontinuous bets, utilizing small, autonomous pods to pursue zero-to-one innovations like Claude Code and computer use. This structure allows the organization to explore high-risk prototypes without the friction of large-team bureaucracy. Furthermore, aggressive token spending is framed not as a cost center, but as a strategic investment in experimentation. Teams that spend heavily on tokens today are simulating the workflows of 2028, uncovering use cases and building muscle memory for future AI-native operations. This communal discovery process, where experimentation is treated as a team sport rather than an individual pursuit, accelerates the identification of viable product directions.
Human Judgment and AI Collaboration
As models approach frontier intelligence, the role of human judgment becomes increasingly critical. Penn argues that AI should function as a thinking partner capable of pushing back on human assumptions, rather than a compliant assistant. This dynamic preserves human agency while augmenting strategic depth. Success in this environment requires leaders to remain hands-on, actively building with AI tools to maintain technical intuition. Hiring and retention strategies must prioritize first-principles thinkers who possess a tinkering spirit and low ego. Ultimately, the organizations that thrive will be those that foster radical ownership, enabling teams to navigate exponential change through collaboration and relentless focus on user value.
Key insights
-
Product management is shifting from static documentation to dynamic evaluation sets that directly guide model training.
Impact: Accelerates model iteration by translating vague user feedback into measurable research targets, reducing time-to-value for new capabilities.
-
Aggressive token spending functions as strategic R&D, allowing teams to prototype future workflows before costs normalize.
Impact: Early adopters gain competitive alpha by mastering AI-native operations and discovering emergent use cases ahead of the market.
-
Labs divisions thrive by using small, autonomous pods to pursue discontinuous bets outside the core roadmap.
Impact: Enables rapid prototyping of high-risk innovations without bureaucratic drag, capturing 10x market opportunities.
-
AI models should be configured to push back on human assumptions rather than simply complying with requests.
Impact: Improves decision quality by preventing echo chambers and ensuring human judgment remains central to strategic outcomes.
Action items
-
Implement eval-driven product loops by replacing static PRDs with dynamic evaluation sets that measure model performance against specific user pain points.
Impact: Creates a direct feedback channel between user experience and research, ensuring model improvements align with actual market needs.
-
Increase token experimentation budgets to allow teams to aggressively test emerging models and simulate future workflows.
Impact: Builds organizational muscle memory for AI-native operations and uncovers high-value use cases before competitors.
-
Establish autonomous labs pods dedicated to high-risk, zero-to-one prototypes that fall outside the core product roadmap.
Impact: Fosters a culture of discontinuous innovation, enabling the discovery of transformative products without slowing core operations.
-
Mandate hands-on AI usage for all product leadership, requiring executives to actively build and ship with AI tools.
Impact: Maintains technical intuition and empathy for user workflows, ensuring leadership decisions are grounded in reality rather than abstraction.
Quotes
“We actually have a saying on the team of evals are the new PRDs.”
“If you're willing to spend $100,000 a year right now on tokens, you are living the way somebody in 2028 is going to live.”
“You need frontier products in order to have frontier models and for people to feel the magic of frontier models.”