Capital One's AI Governance and Risk Framework
Capital One's SVP Abhijit Bose details how regulated enterprises can scale AI by shifting from token metrics to outcome-based evaluations. The discussion covers building internal multi-agent frameworks, implementing centralized risk scoring for code deployment, and leveraging platform engineering to automate compliance controls.
Executive Brief: Scaling AI in Regulated Enterprises
Capital One’s approach to AI transformation offers a critical blueprint for regulated industries seeking to balance innovation with rigorous risk management. Abhijit Bose, SVP of Enterprise AI and ML Platforms, outlines a strategy that moves beyond initial code generation hype to focus on measurable outcomes, robust evaluation, and automated governance. The core thesis is that AI adoption in banking requires a shift from "token maxing" to outcome-based metrics, ensuring that increased productivity translates directly into faster product delivery and feature enrichment.
Strategic Shifts in AI Governance
A primary differentiator for Capital One is the institutionalization of evaluation frameworks. The organization treats evaluations as core infrastructure, requiring all coding tools and agent harnesses to pass through custom and industry benchmarks before deployment. This rigorous testing regime allows the company to optimize for latency, cost, and quality simultaneously. Furthermore, Capital One has developed a proprietary multi-agent framework, featured at NVIDIA GTC, which powers customer-facing applications like their auto dealership assistant. This framework demonstrates that generic, reusable agent architectures can drive significant business value when paired with robust backend infrastructure.
Risk Scoring and Deployment Automation
The most actionable insight for engineering leaders is the development of internal risk scoring infrastructure. By analyzing code impact, historical outage data, and user sensitivity, Capital One can automate the deployment of low-risk changes while maintaining human-in-the-loop controls for high-risk scenarios. This approach leverages the bank’s inherent expertise in risk management to solve a common engineering bottleneck: the fear of deploying AI-generated code. Additionally, the implementation of a centralized AI gateway automates compliance controls, such as prompt injection defenses, removing this burden from individual developers and standardizing security across the organization.
Conclusion
Capital One’s model demonstrates that governance is not a barrier to AI speed but an amplifier. By centralizing platforms, automating compliance, and rigorously evaluating outcomes, regulated enterprises can achieve the "speed of well-managed." The focus must shift from experimental pilots to scalable, telemetry-driven systems that continuously improve through production data. For leaders in finance and other high-stakes sectors, the path forward lies in building the infrastructure to measure, score, and automate risk, rather than relying solely on human oversight for every code change.
Key insights
-
Token consumption and code generation volume are no longer valid metrics for AI success. The industry is shifting toward outcome-based measures that correlate AI usage with actual product delivery speed and feature enrichment.
Impact: Aligns engineering investment with business value, preventing wasted spend on high-volume but low-impact AI usage.
-
Evaluation frameworks are becoming critical infrastructure. Capital One requires all agent harnesses and code changes to pass through custom and industry benchmarks to ensure quality, latency, and cost efficiency.
Impact: Reduces production incidents and ensures consistent performance across different LLM models and agent configurations.
-
Internal risk scoring infrastructure allows for automated deployment of low-risk code changes. By analyzing code impact and outage history, banks can safely automate parts of the SDLC while retaining human oversight for high-risk updates.
Impact: Accelerates time-to-market for safe changes and reduces the cognitive load on human reviewers for routine deployments.
-
Centralized AI gateways automate compliance controls such as prompt injection defenses. This removes the burden of security implementation from individual developers, ensuring consistent governance across all AI applications.
Impact: Standardizes security practices and allows developers to focus on core business logic rather than repetitive compliance tasks.
-
Proprietary context retrieval engines are essential for optimizing agent performance. By indexing APIs, business policies, and operational data, organizations can prevent inefficient context window usage and reduce latency.
Impact: Lowers operational costs and improves agent accuracy by providing precise, relevant context to LLMs.
Action items
-
Replace token-based metrics with outcome-based KPIs. Define clear correlations between AI usage and product delivery speed, feature count, or customer satisfaction to validate ROI.
Impact: Ensures AI investments drive tangible business results rather than just increased code volume.
-
Implement a mandatory evaluation framework for all AI-generated code and agent harnesses. Use custom datasets and industry benchmarks to test quality, latency, and cost before production deployment.
Impact: Prevents low-quality or inefficient AI solutions from reaching production, reducing technical debt and operational costs.
-
Develop an internal risk scoring model for code deployment. Integrate data on code impact, historical outages, and user sensitivity to automate low-risk deployments and flag high-risk changes for human review.
Impact: Accelerates safe deployments and optimizes the use of human reviewer time for critical decisions.
-
Build or adopt a centralized AI gateway to automate compliance controls. Ensure that prompt injection defenses and guardrails are applied automatically at the platform level, removing this responsibility from individual developers.
Impact: Standardizes security practices and reduces the risk of inconsistent compliance across different teams and applications.
-
Invest in a proprietary context retrieval engine. Index internal APIs, business policies, and operational data to provide agents with precise, relevant context, reducing latency and improving output quality.
Impact: Optimizes LLM usage costs and improves agent reliability by ensuring they have access to the most relevant information.
Quotes
“how do you stay at the frontier of AI with the speed of well-managed”
“token maxing is not the right, you know, for example, like the right metric”
“for us, it's all about risk scoring”