The Evolution of AI Engineering and Open Source
An exploration of the transition from traditional software engineering to AI engineering, focusing on the agentic workflows, the necessity of organization-specific evaluations, and the shift in engineering culture. The discussion highlights the role of open source collaboration in accelerating technology adoption.
The Paradigm Shift: From Software to AI Engineering
In the rapidly evolving landscape of technology, the transition from traditional software engineering to AI engineering is not merely a technical shift, but a cultural and methodological one. The emergence of AI agents and agentic workflows—where Large Language Models (LLMs) act as decider nodes in structured graphs—is redefining how applications are built and integrated into SaaS platforms.
The Critical Role of Organization-Specific Evaluations
One of the most significant challenges in AI engineering is the non-deterministic nature of LLM outputs. Unlike traditional software, where a specific input always yields the same output, AI applications require a new approach to quality assurance. While generic benchmarks exist, the real value lies in creating custom evaluations (evals) based on proprietary organizational data. By leveraging subject matter experts to build comprehensive datasets, companies can systematically reduce failure modes and ensure the agent's output aligns with strict organizational or legal requirements before a staged rollout.
Redefining Engineering Culture and Teams
AI engineering requires a marriage of two traditionally separate worlds: the data scientist's comfort with statistical uncertainty and the software engineer's discipline in building scalable, production-ready code. This intersection has given rise to the 'tiger team'—a cross-functional, agile group that bypasses traditional command-and-control organizational structures to iterate rapidly on high-value projects.
Conclusion
For leadership and investors, the takeaway is clear: the competitive advantage in AI will not come from the model providers themselves, but from the ability to integrate these models into specialized, organization-specific workflows with rigorous evaluation frameworks. The speed of adoption is accelerating, and those who embrace the 'uncomfortability' of this new domain will be best positioned to lead the next wave of technological innovation.
Key insights
-
AI engineering differs from traditional engineering due to the non-determinism of agentic applications; multiple successful responses can have different bodies, making traditional unit tests insufficient.
Impact: Forces a shift toward statistical evaluations and tracing over deterministic testing, changing how software quality is measured.
-
The most significant value in AI evaluations comes from datasets unique to an organization's core competence and proprietary data, rather than generic model benchmarks.
Impact: Allows companies to create defensive moats around their AI applications by leveraging private data to ensure high accuracy and reliability.
-
There is a growing gap between data scientists who build prototypes in notebooks and software engineers who build production systems, necessitating a hybrid 'AI Engineer' skill set.
Impact: Increases demand for engineers who can navigate both statistical uncertainty and production-grade software architecture.
-
AI agents can now be integrated as a distinct client interface within SaaS applications, acting as an alternative to traditional web or mobile UI for interacting with APIs.
Impact: Transforms the user experience of SaaS, moving from manual navigation to a conversational, agent-led interface for complex data reporting.
-
The lifecycle of open source maintenance is being radically accelerated by the use of multiple parallel coding agents to handle bug reports, triage, and change log generation.
Impact: Significantly reduces the overhead of maintaining open source projects, allowing smaller teams to manage larger communities and more complex codebases.
Action items
-
Implement organization-specific evaluation datasets by collaborating with subject matter experts to identify critical failure modes and high-risk questions.
Impact: Ensures AI agents are safe and legally compliant within a specific business context, reducing the risk of incorrect or harmful outputs.
-
Form cross-functional 'tiger teams' consisting of veteran engineers and data scientists to bypass rigid organizational structures for high-value AI prototypes.
Impact: Accelerates the time-to-market for AI features by reducing friction between prototyping and production deployment.
-
Adopt a staged rollout strategy using feature flagging to move AI features from beta testers to 100% of users over weeks rather than days.
Impact: Mitigates the risk of non-deterministic errors affecting the broader user base by allowing for real-world testing and iterative refining.
Quotes
“In AI engineering, tracing and evals are sort of we've seen them being let's say tenx is important as normal engineering because the non determinism of agentic applications you can't anymore expect that like you can have multiple successes that have different response bodies”
“The things that are important when building an application that the model providers are not going to do are the things that are unique to your organization's area of core competence and the data that your organization has”
“To be good in a new field you need to be sort of okay with being uncomfortable and okay with being kind of bad at this new thing that you're doing”