Scaling AI Adoption in Large Engineering Teams
Coinbase's Senior Director of Engineering details the strategic framework for driving AI adoption across 1,000+ engineers. Learn how to shift from skepticism to velocity using hands-on leadership, speed-run events, and automated feedback loops.
The Shift from Skepticism to Velocity
Large engineering organizations often struggle to move beyond superficial AI adoption. At Coinbase, a team of over 1,000 engineers faced this exact challenge. Initial attempts with tools like GitHub Copilot resulted in low retention; users tried the tools briefly but failed to integrate them into daily workflows. The turning point was a strategic shift from top-down mandates to hands-on leadership engagement. By having senior leaders personally code with AI tools, the organization identified specific friction points and demonstrated tangible value, transforming AI from a compliance checkbox into a core productivity driver.
Strategic Execution Framework
The adoption strategy relied on three key pillars: visible leadership, gamified momentum, and automated workflows. First, leaders demonstrated "show, don't tell" by using AI for daily tasks, from bug fixes to documentation. Second, the team organized "speed runs," where hundreds of engineers simultaneously submitted pull requests using AI assistance. This event generated 400 PRs in 30 minutes, breaking infrastructure limits and proving that AI could handle high-volume, low-complexity tasks effectively. Third, the organization built internal tools to automate the feedback loop. Raw user feedback is now captured via voice or video, processed by LLMs into structured tickets, and automatically converted into draft PRs. This eliminates manual triage and reduces the time from user feedback to deployed fix from weeks to minutes.
Measuring True Impact
A critical insight is the rejection of vanity metrics. Instead of counting lines of code, the team focuses on "ticket-to-user" cycle time. This metric captures the entire value chain, from prioritization to deployment. By reducing PR review times from 150 hours to 15 hours, the team achieved a 10x improvement in velocity. This approach allows leaders to quantify the ROI of AI adoption in terms of market responsiveness and product quality. The result is a culture where engineers are empowered to ship code faster, with less coordination overhead, and with a clear path for continuous improvement through data-driven cohort analysis.
Key insights
-
AI adoption fails when leaders issue mandates without personal usage. Hands-on leadership is required to identify workflow friction and model best practices.
Impact: Increases trust and buy-in from engineering teams, accelerating the transition from skepticism to active usage.
-
Time-boxed, high-volume coding events (speed runs) create social proof and break cultural inertia. They demonstrate that AI can handle large-scale, concurrent tasks.
Impact: Generates immediate visible results that shift team perception and drive organic adoption across the organization.
-
Traditional metrics like lines of code are misleading. The true measure of AI efficiency is the reduction in time from ticket creation to user-facing deployment.
Impact: Provides a clear, defensible ROI framework for AI investment, focusing on business outcomes rather than tool usage volume.
-
Automating the feedback-to-PR pipeline eliminates manual triage bottlenecks. LLMs can convert unstructured user feedback into actionable code changes instantly.
Impact: Reduces coordination overhead and allows teams to respond to user needs in real-time, improving product-market fit.
-
AI tools can analyze usage data to segment engineers into cohorts (e.g., light users, power users). This enables targeted enablement strategies for different skill levels.
Impact: Allows leaders to create specific playbooks for each cohort, ensuring that all team members, regardless of experience, can benefit from AI.
Action items
-
Implement a "hands-on" leadership policy where senior engineers must use AI tools for daily tasks and share findings in team channels.
Impact: Builds credibility for AI initiatives and provides real-world examples of effective usage for the rest of the team.
-
Organize a company-wide "speed run" event where engineers compete to submit the most valid AI-assisted PRs in a short timeframe.
Impact: Creates a viral moment of success that demonstrates the feasibility of AI at scale and boosts team morale.
-
Shift performance metrics from output volume (lines of code) to cycle time (ticket-to-deployment). Track and report on this metric weekly.
Impact: Aligns team incentives with business value and provides a clear baseline for measuring AI-driven efficiency gains.
-
Build or integrate an automated pipeline that converts user feedback (voice/video) into structured tickets and draft PRs using LLMs.
Impact: Reduces manual triage time and accelerates the feedback loop, allowing for faster iteration and higher user satisfaction.
-
Use AI to analyze tool usage data and segment engineers into cohorts. Develop targeted onboarding or training materials for each group.
Impact: Ensures that all team members, from juniors to seniors, receive relevant guidance, maximizing the overall adoption rate.
Quotes
“I do think that it's really important when you're doing this organizational transformation that you have a single person with incredible conviction at the leadership level who is also hands on the metal.”
“The worst thing any engineer could do is just be like, I decree you must use AI. Like, come on, no one's going to listen to you.”
“AI is an accelerant because there will always be more work to do, right?”