Strategic AI Integration in Software Delivery
An executive analysis of Birgitta Bachler's insights on harness engineering, local model viability, and the shifting role of developers in AI-augmented teams. Covers the transition from deterministic guardrails to probabilistic fitness functions and the impact on team structures.
The Shift to Probabilistic Engineering
The integration of AI into software delivery is moving beyond simple tool adoption toward a fundamental restructuring of engineering practices. Birgitta Bachler, a distinguished engineer at ThoughtWorks, argues that traditional deterministic 'guardrails' are insufficient for managing AI agents. Instead, organizations must adopt a probabilistic mindset, utilizing 'fitness functions' to define acceptable output quality. This shift requires teams to explicitly articulate what 'good enough' means for their specific systems, moving from implicit team knowledge to explicit, measurable criteria.
Harness Engineering and Context Control
The concept of 'harness engineering' has emerged as a critical discipline. This involves expanding the coding agent's environment with custom tools, feedback loops, and context management. Rather than relying solely on static prompt instructions, effective teams use dynamic steering mechanisms. This includes integrating linters, custom scripts, and computational feedback directly into the agent's workflow. The goal is to create a regulated system where the agent's output is continuously measured and corrected, mirroring principles from cybernetics and control theory.
Strategic Use of Local Models
There is a growing viability for local, small-parameter models in specific workflows. While frontier models remain superior for complex planning and architectural reasoning, local models like Qwen 3.6 offer cost-effective solutions for well-defined, repetitive tasks. However, the adoption of local models is not plug-and-play; it requires careful evaluation of hardware constraints and task complexity. Teams should use large models for high-level planning and delegate execution of straightforward subtasks to smaller, faster local models to optimize cost and speed.
Evolving Team Dynamics and Roles
The role of the developer is expanding to include QA and product context. As AI handles more coding, developers must focus on verifying functional correctness and understanding second-order system effects. Pair programming remains vital not just for code quality, but for aligning team members on AI tool usage and harness configuration. Furthermore, the boundary between development and product management is blurring, with developers taking on more responsibility for feature adoption and user impact. Organizations must invest in observability infrastructure to support AI-assisted debugging, ensuring that agents have access to the high-quality data needed to resolve complex incidents effectively.
Key insights
-
Traditional guardrails are unreliable for AI agents; teams must use probabilistic fitness functions to define acceptable output quality. This requires explicit definition of system constraints and success metrics.
Impact: Reduces false confidence in AI output and improves the reliability of automated code generation pipelines.
-
Harness engineering involves dynamically steering agents through feedback loops and custom tools rather than static prompts. This approach treats the agent as a regulated system requiring continuous measurement.
Impact: Enhances agent performance and consistency by providing real-time feedback and context, reducing the need for manual intervention.
-
Local small models are viable for specific, well-defined tasks but require careful hardware and task complexity evaluation. They offer cost savings but lack the reasoning power of frontier models for complex planning.
Impact: Optimizes token costs by delegating routine tasks to cheaper local models while reserving expensive frontier models for high-value strategic work.
-
Developers must adopt a QA mindset to verify functional correctness in AI-generated code. This involves assessing second-order effects and risks rather than just checking syntax or unit test coverage.
Impact: Ensures that AI-generated code is not only syntactically correct but also functionally robust and aligned with business requirements.
-
Pair programming and mob coding remain critical for aligning teams on AI tool usage and harness configuration. These practices facilitate knowledge sharing and ensure consistent application of AI workflows.
Impact: Accelerates team adoption of AI tools and reduces variability in output quality by fostering shared understanding and best practices.
Action items
-
Define explicit fitness functions for your system to guide AI agent output. Document what 'good enough' means for performance, security, and maintainability in your specific context.
Impact: Provides clear criteria for evaluating AI-generated code, reducing ambiguity and improving the reliability of automated workflows.
-
Implement a hybrid model strategy by using large frontier models for planning and complex reasoning, and local small models for routine, well-defined tasks.
Impact: Reduces operational costs and improves speed for routine tasks while maintaining high-quality strategic output from advanced models.
-
Expand your coding agent harness with custom feedback tools, such as linters and pattern-matching scripts, to provide immediate computational feedback to the agent.
Impact: Improves agent accuracy and reduces the need for manual code review by enabling the agent to self-correct based on real-time data.
-
Train developers to adopt a QA mindset, focusing on functional correctness and second-order system effects rather than just code syntax.
Impact: Enhances the ability to detect subtle bugs in AI-generated code, ensuring that the final product meets business and user requirements.
-
Maintain pair programming and mob coding sessions to align the team on AI tool usage and harness configuration.
Impact: Ensures consistent application of AI workflows across the team, reducing variability in output quality and accelerating collective learning.
Quotes
“in 80% of the cases in the space, what we actually have to ask is, how do we increase the probability that?”
“it's just context engineering for coding agents, right? And the actual thing, the harness, so like cloud code.”
“if you want to build these sensors and guides and skills and all of those things, you actually have to know what good looks like”