# Context Engineering and the End of Code Review

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-04-14

## Transcript

The majority of developers today that are using agents, they're using agents for smaller tasks.
They're still very supervisory.
The best developers are the ones that truly understood the problem to be solved.
They just saw the code writing as a pass, right?
They knew how to then translate the English into code quickly.
I see the pattern that is emerging to try and tackle that is, okay, as you get the agent to code, Be explicit around architecture, even if it's not you who made the decision, but the agents.
Review that, and don't be afraid to tell the agent, scrap all of that and build again.
As coding becomes something that's very fast and actually taken away from us slightly, we do lose that time to think as we're building.
But what we do gain is that faster iteration, which allows us to gain feedback faster and throw away quicker.
I think one of the things that I find most interesting about the way AI has evolved is it's oftentimes around solve the next problem, solve the next problem, solve the next problem.
So I think the next problem to solve right now is...
Back in November, we hosted the first ever in-person AI native DevCon in New York.
This June 1st and 2nd, we're bringing it to London.
It's two days built for AI-nated developers and engineering teams.
One day full of hands-on workshops and one day full of practical talks on agent skills, context engineering, agent orchestration and enablement platforms, and how teams are actually shipping AI in production.
Join us at the brewery in London near the Barbican for all of that.
Plus, networking, parties, giveaways, and a room full of people building the future of AI native development.
You can also join us from anywhere in the world via the live stream.
As you're listening to this podcast, you get 30% off your ticket with code POD30.
Just head to AINATIVEDEVCON.io and we'll see you in London.
Hello and welcome to a milestone episode of the AI Native Dev.
My name's Simon Maple, one of the...
hosts of this show and joining me is Guy Pajarni.
I'm the co-host, Simon's co-host and sidekick overall.
And today's a milestone episode because, you know, Guy, we've already done, already shot and released.
100 episodes of the AI Native Death.
That's insane.
How does that feel?
Like, I don't know how we get anything else done.
I know, right.
All the time, just sort of coming here and talking to interesting people, which actually is not a bad reality.
Yeah, so we're a podcast company, right?
And we do this other thing on the side.
It's the fallback.
It's the safety net.
I don't know how safe it is, but it is interesting.
Amazing.
And so this episode is really going to look back.
at the previous 100 episodes.
And we're going to take sound bites from some of the most amazing speakers that, like you say, we've had on the show.
So it's going to be about 45 hours?
Something around there.
Yeah, it'll definitely be a week at least.
So, yeah, make sure you've got a long drive.
And we're going to add our commentary and also see how things have changed.
For example, let's run the first clip guide because I think we're going to find this very, very interesting.
If anyone who's predicting what that looks like in a year or two is going to be fundamentally wrong.
And I think that's really what makes it so exciting.
We don't know where we're going other than that we do know that we're going to keep building software while maintaining billions, if not trillions of lines of code that are already out there.
Thomas Domker, of course, the previous CEO of GitHub and now, I believe, founder of a new company, new startup in the AI space called Entire.
So if anyone's going to be fundamentally wrong about a prediction, in one to two years.
Let's go back, maybe let's say one to two years, Guy, to episode one, where you predicted something around AI development.
Yeah, I never claimed to be too wise.
No, no, no, no, no, no, no caveats now, Guy.
Let's hear what you have to say and then we'll see how accurate you were.
I think the biggest change that we should expect that I feel fairly convinced will happen is the move from code-centric to spec-centric development.
When you think about how you develop software today, you get some requirements, you write some code, you make 100 decisions in that code that never leave the code.
And the code very quickly becomes the source of truth, and we focus on evolving it and growing it.
I think in the world of AI native development, we will have a means of specifying what it is that we need, and AI will provide the implementation.
So, Guy, tell us your new take.
How accurate were you?
Yeah, I mean, I think, so something's spot on, some aspect of it, in that I think today it's a much less controversial statement to say that, you know, code will become disposable and that implementations will be, you know, something that is just kind of created for it.
And so that you will revolve around intent, you know, around what it is that you want to build.
So I think it was like not...
far off on that.
That bit was good.
Tell me about the bit that's fundamentally wrong, as Thomas would say.
I think the primary kind of change, I'd like to think evolution, kind of in my thinking, our thinking here at TESOL, is this move from spec to context.
When we recorded that episode and we were talking about spec-centric software, we talked about how you want to make sure that LLMs, this is pre-agents, now agents, will make sure to update to kind of capture.
You will convey what it is that you want to build, and then the LLMs will build that, and they will fill in the gaps.
That's what we talked about back then.
And while that is not wrong, you know, it's true LLMs need to do that, it's just such a small part of development.
When you think about your dev team and what you expect of them, you know, you don't really say, hey, make sure that every time you follow, you read this doc, and you follow the exact instructions, and you update that.
You know, sometimes you want them to do that, but generally you want them to...
to make good decisions, including choosing when to update documents and read them.
But that's just like a portion of what you want from a great developer in your team.
You want them to have all sorts of preferences around how they build and how do they collaborate on it and how do they test and make sure it didn't break and how do they balance kind of quality and how do they, and sort of speed, and how do they kind of use your existing infrastructure and how do they troubleshoot issues.
And so all of that is really about training the developer.
And so I guess my kind of catchphrase today is the thing that we've moved from speccing the program to speccing the programmer.
It's much more about how do you want a developer, a great developer in your team to behave.
And again, there's a portion.
Like, I don't think spec-centric software or spec-driven development, as is kind of the term today, goes away.
Like, I think that is important.
I just think it is a subset of how you want today agents to build.
And it's more important when you think about the new software development, it should revolve around training these AI agents to build the way you want them to build.
Such a good way of putting it.
And one way that resonates with me is onboarding.
And it's like if you were to hire an individual to work on your team, you would spend time onboarding them, training them.
They could be the most senior developer on your team that you've hired.
but they still need to know how your team works, your organization works, and take on those policies, ways of working to actually use their skills in the new environment.
And there's no reason why we shouldn't do that with an agent as well.
And it's that onboarding of that agent.
And very, very nice.
Yeah, absolutely.
And that would sort of add, it's like onboarding and then continual training.
You know, you want to make sure that they...
you know, as a group, you learn things.
So some of it is you individually learn, but a lot of it is the group learns, the team learns, and you want to make sure you accumulate that and retrain the team.
So both of those are human analogies, right, about how do you want people in the team to work.
You want them to onboard, and then you want them to kind of keep with it, and then sort of adapt and evolve and be aligned with.
the rest of the organization, how they build.
So we need to build that.
So all of that today manifests in the form of context engineering.
And I guess the unit of software today or unit of context today that is most discussed is skills.
And so I'd say if you take the clip, you can swap the word spec for skill.
And, you know, it will still be natural language words.
It will still be a capture of intent.
But it is not a program spec.
It is a programming spec.
You know, how do you want to develop software?
So it was quite a subtle thing that you were fundamentally wrong about.
I was fundamentally wrong.
I had to be fundamentally wrong about something.
Otherwise, we wouldn't be learning anything.
It would be boring, Simon.
I mean, we can agree.
You and I can agree that you were fundamentally wrong about that.
So that's fine.
Okay, next up.
Thomas didn't just mention that you were fundamentally wrong about something.
He also talked about the changes that developers need to go through.
Let's hear from Thomas.
Simon Willison on his blog recently had a statement, I'm paraphrasing, but it was like, we have to get comfortable with not reviewing every single line of code before we deploy them into production.
Because the sheer amount of code that a single agent can write 24-7, now you parallelize that into running 10 agents in parallel all the time.
you're just never going to be able to review all the code without one human doing that becoming the major bottleneck and effectively erasing some of the productivity gains from these agents.
Yeah, very interesting take there from Thomas and actually also supported by a couple of other folks on the podcast.
Peter Guagenti from episode three way back also talked about a potential future whereby agents might actually write code that isn't even human readable.
So maybe something close to a byte code or a machine style code.
Birgitta also from ThoughtWorks in episode 69.
Also said she ran into situations where she actually has done so much, almost like a vibe coding, but allowing the agent to write that code that she feels so disconnected from the code.
She isn't necessarily the most appropriate person to do the code reviews.
What's your take on this?
Yeah, I think it's super interesting.
fully aligned with Thomas's view.
In general, the human is the bottleneck.
There was all sorts of similar statements in the AI world.
Elon Musk talked about how he had some weird spreadsheet analogy on it, but he also said if you have a gazillion spreadsheets on it and just a few cells need to be filled in by humans, you've reduced it down to human speed.
And I think that's what we're saying with code review, that if humans are the bottleneck, then you get...
human traits like review fatigue, but then you also just see slowness, right?
This is the human speed, you know, is limiting how much can AI kind of code and build up and so you're just moving the bottom.
Like, I think this will actually extend further.
So once you get through and you deploy, now the problem will be that the system changes too quickly and we can't do like observability and root cause analysis and handle those.
We can't figure out that a bug needs to come back and and be resolved and then resolve that and deploy the fix.
So really throughout the end-to-end software development lifecycle, our aspiration really has to be, and I think we're seeing this in the advanced organizations, to really agentify every single step of that process.
And I think that is the only way to move forward.
And when you think about, okay, it's like...
but these things are unreliable, they don't know that they build the right thing, it comes back to context, right?
And, you know, at Tesla we talk about the context development life cycle, right?
And I think agents should operate the SDLC and the human should live in the context development life cycle in the CDLC.
They should.
really be a guide.
And I think the human analogy is really a manager.
And if you're, you know, at the moment, we're like the frontline manager with a player coach, you know, you're sort of giving the tasks, you're monitoring, and you're, you know, getting things done.
And increasingly, if you want to increase your scale and have more employees, in this case, AI employees, or sort of, you know, AI labor that is getting things done, then you need to become, you know, a second-line manager and a third-line manager.
And I think those are...
Those really predicate on your ability to define what correct looks like, to convey instructions, to identify that mistakes were made and be able to resolve them.
And so you need to move up the stack.
And the way you communicate with agents is through context.
It's not just context.
It's context.
It's the tools you make available to them.
It is the foundation to be able to observe what has happened.
And so that's why we want to think about context development lifecycle.
We think about kind of generating some piece of context, like, We talked about that in one of those episodes as well, about how knowing what you want is actually hard.
So figure out what it is that you want, what is good, what is correct behavior.
And then you want to be able to test and evaluate.
This is maybe the equivalent of...
testing humans, but in a little bit more software-ish of like your tests to know whether these instructions are well adhered to.
So we think about evals for it.
You want to be able to disseminate and communicate to that, to the horde of agents, you know, that you have the right communication to the right agent at the right time.
And then you want to observe what has happened.
And then once you've observed, you will have learned a bunch of things.
Also a bunch of like facts in the world have changed.
You know, maybe some codes have changed.
Maybe a pricing model, a model came out.
And so...
you want to learn from that, come back and sort of update your instructions.
And so all of that is what we think of as the context development lifecycle.
So I think that is very kind of aligned with this destination.
And I think it's interesting that the world of AI, you know, I like my crystal ball, you know, like I like these, you know, fundamentally wrong predictions and looking ahead.
But I think one of the things that I find most interesting about the way AI has evolved is...
it's oftentimes around solve the next problem, solve the next problem, solve the next problem.
So I think the next problem to solve right now is code review because we're filling it there.
I think subsequently we'll get into those deployment sites.
Because one of the biggest worries is, of course, the quickest, cheapest, short-term fix is just to allow things to happen and for developers to go, if I'm the bottleneck, I can just eliminate myself from being the bottleneck by just checking, yeah, this is fine, or just not doing it at all.
It's about doing things.
How can we speed things up, continue at the pace that AI wants to generate at while doing things accurately and properly and adding that context in to almost take things away from the developer so that the developer, the human, can really look at the things that are most important rather than as code changes scale up, just try and do things faster or by learning how to code review faster.
That doesn't make sense.
But adding this context actually can take a lot of that away from us.
And for us to be that second line manager, to understand it, to manage it, makes a lot more sense.
Yeah.
And I think the cloud analogy kind of works pretty well here, which is, you know, when we were in Waterfall era, a lot of these processes and a lot of these reviews were human.
And as we moved into cloud, we had to automate them, right?
We had to define.
to do policy as code as the meta name for it, but infrastructure as code to deploy your infrastructure, automated testing, and dev QA became a thing.
It wasn't a thing before on it.
And so we actually have gone through a fairly similar shift when we moved to clouds to say things are moving faster.
The bottleneck is the human.
Let's automate this action of the human.
And now we're doing another leap over there.
And it was mighty uncomfortable.
at the time to say, oh, what do you mean?
I'll automate this and I will deploy it and I will deploy more bugs.
And, you know, the time has shown that if you close the loop, then yes, you might even deploy more bugs.
But because you're able to identify those and then fix them and then come back and roll those out, the systems that were continuously deploying, you know, today's well-known, they are more secure.
They are, they have higher quality, you know, they have all of those.
But you have to go end to end.
If all you've done is removed the gate, right, and just said, just...
just let it flow, then you end up with a system that is more broken.
So a lot of that cloud learning actually applies here substantially.
you have to think about the end-to-end operation.
It is happening faster than in cloud, and so what we don't have is the benefit of trying it out and seeing it works.
I have one more small comment just on what you said about non-readable code, which is interestingly something that is now starting to surface as a safety concern.
So there's actually some significant call-outs to say, that actually a safety requirement should be that LLMs are not allowed to produce things that are not human-reviewable, like binary code or their own languages, even if it is more efficient for safety as part of being able to supervise agents.
So that's interesting.
I don't know if it's an over-exaggerated concern or not, but I think right now more likely in the prediction realm that we will produce readable.
output readable code and then it'll get compiled.
So following on from code review, I guess, next up we have Annie Vela, who's a distinguished engineer from Westpac and really interesting blog and discussion that I had with her around the future of developers.
Let's hear from Annie.
There are literally people who enjoy going through those sorts of lead coding exercises on there's a reason that website exists right there's leaderboards of people who can solve the most complex problems the fastest so that i don't think will be very important anymore that's a skill that a lot of software engineers have taken to be the core of being a really strong software engineer that is a skill i think we're going to need to learn to let go of both in interviews and personally, it just, it won't make much sense anymore.
Why would you need to be an expert at writing that level of code when an AI can probably generate something good enough for you?
Really important point.
And I think it's something that I think we're mostly, I think most of the industry is aligned on it, but it's possibly one of the things that we are least comfortable giving away because it's a thing that we enjoy.
It's a thing that we have become, it's part of what excellence is in or seen as in development in programming to write the nicest code the cleanest code that solves the deepest most complex problems and be fast about it and be fast about it is this going to be a problem for developers adapting in the future I mean I think for some for sure yeah it I like the mental model that says like some developers I'm generalizing here but like some developers are more motivated by the craft of coding while others are more motivated by the impact, right, or by the creation.
And so I think if you're motivated by the creation side, then agents are amazing.
You can produce so much, shortening that journey from idea to execution.
Even if you were or are very, very fast as a human developer, it still takes you that time.
And while in the meantime, you could have thought of...
other things.
It's the means to the end of the coding versus the enjoyable things.
So the fun is having an idea and bringing it to life and tuning it and orienting it and even making some architectural decisions and things like that right around what is elegant and what's not.
But like writing the code, a step in the journey, right?
It is not the goal.
While for others, writing the code is the craft, you know?
And I do think there's beauty in the architecture of, you know, like writing correct code.
well-crafted, short, simple code on it.
All of those are...
Good abstracted Java classes and interfaces.
Exactly, yeah, and also in modern languages.
But, you know, you...
So there's beauty in it, you know.
So there is...
There's probably room for it from an aesthetic perspective and a skill, but it...
It's slightly becoming like a factory manufactured furniture versus kind of handcrafted ones, right?
You can create things that are very local.
They might even be able to better represent very, very specific preferences, very specific decisions that are almost hard to convey in any way that isn't code.
But for the vast, vast majority of furniture, you're totally comfortable with them being.
factory made, right, with them being built by machines.
And so I think that is true for code as well and for increasingly sophisticated and complicated code.
And the advantage of doing it by hand is so minuscule compared to the advantage of being able to produce it quickly, cheaply, by more people.
Like, think about the loss of knowledge when you have one person with the idea and they need to convey that to...
know a person to convey that to a person you know to eventually get to someone who writes the code so dropping some of these middlemen um so all of those are just so much more valuable so the reality is i think there will be room for handcrafted code in some niches in some very specific spots and some artisanal element and i think it can remain a hobby just like today there's a bunch of these you know hobbyist hardware hobbyist like it's okay for that to happen but i think as a profession You've got to let go.
So do you think, first of all, the best developers five years ago will continue to be the best developers in five years?
What do you think the key skills need to be for a successful developer?
It's a good question.
I don't know what the percentage is of people that are more the best developers because of their ability to craft code versus the best developers otherwise.
I'd say that even...
like throughout the years, the best developers are the ones that truly understood the problem to be solved and the architectural trade-offs to be made.
And then they just saw the code writing as a pass-through, right?
They knew how to then translate the English into code quickly.
And so I think it's the same best developers.
But I think there's, if you were to contrast or like compare, you know, different developers, for some of them, the strength of code writing was a bigger part of that mix.
And that will diminish.
There's another element of just speed of decision making.
And so writing code is slow.
And because it is slow, you actually get to think about the problem and these architectural decisions and all of those as you code.
And I think that is an interesting thing that is lost a little bit when you are coding quickly because you need to decide more things up front.
And that's hard because sometimes you don't need to do it.
I see the pattern that is emerging to try and tackle that is, OK, as you get the agent to code, be explicit around architecture, even if it's not you who made the decision, but the agents.
Have the agent be explicit about the architecture, review that, and don't be afraid to tell the agent, scrap all of that and build again.
So this ability to rebuild is really more important than the ability to build.
How do you capture that?
It slightly comes back to context of, in this case, maybe it is sometimes specs and architecture.
It's a specific type of context to say, this is what we've built.
These are the emphasis points about how we built it.
remove the whole thing and write it in you know i don't know java yeah again rust something uh per your requirements i love that i love that thought where you know as coding becomes something that's very fast and actually taken away from us slightly we do lose that time to think as we're building but what we do gain is that that faster iteration which allows us to gain feedback faster and throw away quicker yeah we can throw away we can and we can use that feedback to build something faster with real you know stronger knowledge about what works and what doesn't precisely which which by the way like the a recent episode with chad fowler about regenerative software and the phoenix architecture yeah is really all about that amazing so next up let's jump into the world of from dev to ops via devops uh armand dadga You might hire the world's best SRE who knows Terraform and cloud inside and out.
But when they join your organization, they don't know anything about your organization.
So they don't know, do you run Windows in production, RHEL in production, Ubuntu in production?
Now, they might be power users at all of those things, but the context of knowing, hey, as an organization, we've standardized on RHEL, that's a critical piece of context that you have to be a useful SRE at the org, right?
You don't want to hire someone, your rest of my stack is...
windows and all of a sudden this person's writing linux for me but does it help me right they need to understand how does the organization work what have we standardized on what does my existing infrastructure look like for them to generate things that are then contextually relevant to my environment and you can tell from your from your youthful looks there guy that was that was back in episode 19 uh arman of course the co-founder of hashcorp um so Yeah, context is the difference between great AI and useless AI.
And it's not just development.
It's generally AI, right?
In ops, I think what we're seeing in, first of all, context is probably the most abused word right now.
It's like anything you need to know.
And I think the meta perspective over there is to separate intelligence from knowledge.
You can be massively intelligent, but what do you know?
And I think there's a bunch of things you need to know to operate in an org.
I really liked the episode with Armand, which was early in DevOps.
We also had a recent one with Mirko, who is the founder of Dash Zero, kind of a modern observability player.
talking about context and how it's important from an observability perspective and the ability to troubleshoot and do root cause analysis also in the world of DevOps, also highlighting the key is context.
I think Ops is an interesting place because when you think about who in your team, like which developers are carrying the pager and able to troubleshoot the things, or even like the superhero SREs in some organizations who are the ones that will be called to firefight a really bad production outage.
Those people, they have a very broad understanding of the system.
They know what are the types of applications, what are the patterns that are deployed, what is the infrastructure that they are running on.
So all the way from operating system to the types of scaling, historical, like recent or past outages, like known weaknesses within the infrastructure that have not yet been prioritized.
There's all this knowledge.
And because of all that knowledge, they can look at a piece of...
of a current data of like this system crashed or that, you know, those requests are slow or whatever it is.
And they're able to kind of quickly translate that into, oh, it's probably this problem and it's therefore probably this solution.
And so I think it is very, like context is very, very important because oftentimes.
The production system is where it all comes together.
It is the infrastructure you've deployed.
It is the system that you are connected to.
It is, of course, your code and your application.
And what you're seeing in, so definitely seeing in the world of DevOps that context is important and any player on the planet will talk about context when it comes to DevOps.
And what we're seeing is root cause analysis capability is the one that is manifesting as the probably the most useful in the DevOps world to be able to look at these kind of vast amounts of data that exist about how your system operates which is a strength of AI and agents to understand it combine that with context in the organization where you're not just blindly looking at the logs but you don't know anything else you're actually able to gather more of that information and then you go off to understand it and the advantage of root cause analysis is that it's kind of harmless, where you're actually seeing still very cautious use of AI, which was still the case like back in, you know, in episode 19, you know, with Armon, is the use of it for actual ops activity.
Like people are still very uncomfortable with an agent deploying a system.
slightly slightly more comfortable with writing you know some terraform code which is another interesting piece because like everything that you talked about up until the terraform code there is is really pulling sensitive information i guess but information that has come out of your ops infrastructure whether it's troubleshooting pulling data and logs and things like that but when we talk about creating the creation of terraform code So enterprise Terraform code, that's not just something that you find on someone's GitHub account, right?
Whereas typically there is so much more, you know, for a developer, there's tons of code that, you know, the agent can train on, the LLM can train on it and will provide you with good suggestions and good alternatives to what you're trying to do.
But is there really the right training data there for an agent to use or an LLM to use when generating enterprise quality and enterprise grade?
Terraform scripts, for example.
Yeah, I think that was definitely a holdback in the early days.
There's still a decent amount out there.
You know, there's a bunch of public modules and things like that.
You know, there's good documentation for it so it can follow it.
And eventually, you know, declarative languages like HCL, like the HashCorp language, or Rego and such are a little bit easier.
They're still probably behind a lot of commentary about Rego, the open policy agent, you know, not being...
agents not being great at creating it.
So there's probably room to improve.
I have seen substantial improvements on it, or like here, from various users.
And you see it probably for two primary reasons.
One is there's some data aggregation that happens just as these things are actually used in the real world.
And so there's more data accumulated from private sources.
Not all of it can be trained right away, but still there's signals that can be learned.
And the second is the ability to create synthetic data.
And so the labs are increasingly relying, not just on public data.
but on synthetic data.
And I do think that infrastructure as code has the same kind of attribute that regular code has, and sometimes even more, which is you can generate something and you can check if it actually works.
So you can deploy it and sort of see whether the result is correct or not.
So that becomes just a kind of a data generation exercise.
But still, that is one kind of limiting factor is indeed the public-private and maybe the price of generating it.
Another big concern or limitation is the ability to deal with time series data.
And so when you analyze observability data, It's very important, the sequence of things that happen.
And LLMs are a little bit more kind of meta thinkers of how it is.
And you've seen a variety.
We had Oli from Datadog on the podcast on it.
We talked about Todo, their little dog, but also their sort of time series model.
And you see some of the startups like Traceable in this space and all that actually look to...
create models or at least fine-tune kind of RL models around that.
And so the ability to deal with those volumes.
And then on top of all of those, there's just like sheer price.
Like when you're producing a lot of data, you're going to flow that through Opus and you'll be happy until it hits your bank account.
So you need to be smarter and there's many more attempts of sort of using open and cheap models.
So I'd say overall context is key.
in DevOps as much as it is anywhere else, maybe even more so.
And I still think the DevOps world, and we've had, like we've mentioned, three episodes over here, we've had a few other references to it, still ends up being a little bit more cautious, mostly because of some of these sort of technical limitations.
But I'd also say the blast radius of, you know, if they got it wrong, you know, I guess one of the horror scenarios that we sort of hear about agents running systems is they dropped a database in production, right?
Those are...
are kind of very hard to recover from type setups.
So it's an interesting, let's solve code review, you know, kind of evolve from there.
But it's still making progress.
You mentioned meta thinkers.
Let's switch.
This is so seamless guy.
Ian Thomas, who's a meta thinker because he works for meta.
He has some thoughts about the adoption of AI within.
the Meta organization and I caught up with him at Qcon AI in New York and actually to set this up Peter Guagenti who's to have mine also mentioned a stat that he gave was that Gartner say that 10% of enterprise devs use AI tools today but by 2027 that number hits 80%.
Let's see what Ian thinks about that.
We had a few people that were really keen and sort of experimenting and they were finding value of these tools outside of work.
And then we had a bunch of really senior engineers who were perhaps a little bit more skeptical.
And then gradually over time, we've seen adoption grow.
Last time I checked, we were over 80% of the weekly active users.
Wow, wow.
80%, over 80% weekly active users within Meta, which, okay, is a little bit biased, perhaps, given how deeply involved they are with AI, but over 80% weekly active users within Meta using AI DevTools today.
Is that surprising?
It doesn't, but I feel like I have a few grains of salt to kind of throw into that statement.
One is...
This is a world of kind of haves and have-nots on it.
And what you're seeing is this sort of big polarity between organizations that have embraced and invested and have leadership that is both bought in and able to execute on it and driven and sort of businesses that are in a state to push the use of agents on it, which is oftentimes disruptive, expensive.
It's like it's not an easy thing to doing it.
So those organizations are ahead.
And we're seeing a lot of other organizations, and we engage with a lot of them here, in which you'd sort of say this organization uses agents.
But really, when you peer through and we talk to a lot of these orgs, there's like a cohort of forerunners that they are embracing agents.
And some of them are actually very sophisticated in how they use it.
But then the vast majority of the organization is still very nascent.
Everybody's using code completion and the likes.
So that's one.
I think there's a difference between the 10% stat, which was also about a year earlier, but between that stat and the 80% stat just between organizations.
And Meta is still a very AI forward.
It is a large organization, but it is an AI forward, very driven, very technology organization.
The second thing is, I think we don't yet have good enough metrics to say, what does it mean that you're using agents?
And I feel and I see that the majority of developers today that are using agents, they're using agents for smaller tasks.
They're still very supervisory.
So they're delegating, but they're delegating...
small capabilities, they give very detailed instructions of like write this piece.
They use it for things that are immediately available or immediately useful like learning some use of an API or, you know, just sort of building things that are very repeatable.
They build small things with it.
What are your thoughts on adoption through looking at organizations' costs?
of using models or tokens from using models.
Some people are more and more doing that these days.
Yeah, there's a very controversial view there, right, of whether you should basically require people to spend more on the models.
I think it's a very short-lived thing.
So I do see it today.
People talk about how that measure is like.
just a means of driving adoption.
I think everybody that says it has a bit of an apology with it, in part because I think the models are actually getting very expensive.
And one of the competencies of being good at using agents is knowing when do you need to use Opus and when can you use Sonnet and when can you use an open model.
Those are actually part of the strengths.
And when do you give the right instructions so the agent doesn't need 1,000 turns to get it right, but it can actually get it.
get it done in one go.
And so there's actually an efficiency element.
The metric that I like the most, and I think maybe has a shelf life for a bit, is the notion of merged pull requests.
And then some people, Lemonade is a good example, actually introduce the complexity measure on top of the pull request.
So for starters, just kind of assess rate of pull request merge rates.
fine, maybe these are small pull requests, but as long as you kind of have established a point in time and you're kind of seeing the trend moving up, that is, I think, a better measure for developers to say, well, how many pull requests have you opened and how many were merged?
And from that point on, even if you cheated at the beginning and you can kind of get to like some small pull requests because you just started pushing things in that way, you can only do that.
a little bit at the beginning, and then you still have to accelerate that number over time.
So I think that measure is probably a slightly better one.
So next up, after thinking about a potential future where 80% plus in 2027 of enterprise developers and developers generally are going to be using AI, we're going to switch to Big Eater in episode 69.
It talks about how AI amplifies everything.
Good engineering practices.
Bad engineering practices, it does them equally as well.
You know, whenever it sees.
Let's hear from Birgitta.
The AI amplifies indiscriminately, right?
So if you have like a bad setup, you might just amplify that bad setup, right?
And if you have a good one, everything might go well, right?
Can you actually, in your organization, with your pipelines, with your processes, can you support an increase in throughput?
I think a lot of organizations are still underestimating this foundation that you need.
Really interesting because it kind of means, you know, you can't just put your cowboy outfit on and just create software.
You still need to use good practices.
You need to guide and have the right context to be able to get an agent to do what you want the way you want to do it.
And that's really kind of like the crux of what Birgit is saying, right?
Yeah, I agree.
I feel I really like the idea of AI as a multiplier and amplifier of it.
Actually makes me remember one of the early episodes we had with Des from Intercom, one of the co-founders of Intercom, who talked even in the support world about garbage in, garbage out, right?
Where you came in and if you were just like, hey, just replicate what's in, you know, like learn from my knowledge base and quickly you learn, well, your knowledge base has a whole bunch of outdated stuff and kind of things that are no longer correct.
It's the same for your code base.
If the agents just replicate the code base, well, a lot of the things that you've done before in your code base, you don't actually want to repeat.
So you really have to be...
mindful of that.
And so I think we've kind of gone through that phase a little bit in the AI world in which the thinking was, we will just learn from your code and we will replicate what development.
And you don't actually hear that much anymore.
I think today it is a lot more about context engineering and about the definition of what is the correct behavior to build out.
And then you want to codify that and you want to be consistent.
And even if the agents are building things from scratch, then they might build it.
You might want them to be explicit about the way and the decision that they've made as they've built it so that future versions of that agent is able to build them out.
So there's more and more of those types of definitions of what...
good looks like of history of decisions you know of building those out i still think that there's a a gap which makes sense just with time around looping in the learning into that of saying, fine, you've made these decisions.
What would you see in the deployed system that would make you think, well, that decision was maybe not very good or it's no longer correct.
Can I go and adapt?
So we still have to evolve that.
And also, I guess, what you need to change as a result.
Did that decision come from context that needs to be changed?
Or was that a human decision that needs to be changed?
It's pulling feedback or data in and understanding.
what was the root cause of that issue.
Yeah, I agree.
And actually, it's interesting to think about also making the migration to the change and making that be incremental.
I saw a recent post from Intercom about how they manage skills and all that, and they have a test generator or something that tests their systems as just a part of software development.
And when it finds a bug, it looks for that pattern elsewhere.
in the system and then it modifies it in those areas as well.
I don't know how thoroughly and autonomously they do that but I think that type of pattern has to be there.
It has to be a sort of self-healing, self-improving system and it all comes back to that like the whole SDLC, the end-to-end SDLC.
has to be handled by the agents.
And what we need is we need to really focus on what correct behavior is and what are the mechanisms and what are the things we need to review versus can be made autonomous around modifications to that context and then how does the context meet the code when you modify the context doing it.
And it's interesting because people, I think what we're seeing today is like we're moving so quickly, but nobody has had...
substantial mileage with this yet.
And so people haven't experienced yet what happens when their context goes stale.
You know, what happens when, you know, like a practice, like we think today about make sure that it applies the right practices, but what happens when a practice changes, right?
When a security issue changes.
And so that is very interesting.
And also the fact that we're never going to be as manual as we are right now with agents.
Everything is, you know, agents are getting better, but we're also trusting them more.
There's going to be more autonomy in future than less.
And in terms of that, we need things to act correctly and we need things to be as we need it.
And that's so, so important.
Indeed.
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
But for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
Okay, and then one of our...
Actually, one of our...
maybe most popular episodes in terms of the number of views was Olivier Pommel, Datadog CEO, who talked about the wonderful dream of no one waking up at 3 a.m.
because of a critical situation.
Let's hear from Olivier.
At the end of the day, we think just everybody will want automation.
Every single company in the world will want automation.
That's the dream, right?
The dream is you never have to wake up in the middle of the night ever again to fix an issue.
The issue has been for you.
And so the bar for precision needs to be very hard on anything that relates to ability.
And the good news is that I would say three, four years ago, being completely right about root cause most of the time was science fiction.
And I think now it's definitely within reach.
And we can clearly see on the horizon the moment where the technology is good enough to be put into the hands of customers in a large number of situations.
What's the worst that can happen, guy?
Yeah.
I think we spoke a little bit about root cause analysis and how that is the dominant case.
And I think it just really lends itself well to you have to sift through a lot of data, you have to build those.
And if you got it wrong, then you've wasted time as opposed to you've taken out a system.
And so I think those are useful.
And they have a little bit of the coding assistant version over there, which is like surface candidates for me, right?
Be an assistant for root cause analysis.
technology is improving to be able to even get you good inklings of what that is with context.
And so I do think that this is, and we touched on this already, very much the sort of the dream of not getting woken up.
There's a nightmare version of this that we're seeing and actually has manifested a lot recently with all the supply chain attacks, which is what we're also seeing is that attackers are becoming increasingly agentic.
And there is a marked rise in...
phishing attempts that are scaling and are going downstream and going into SMBs.
We're seeing agents finding vulnerabilities in open source libraries, and so there was a big push around a variety of supply chain vulnerabilities.
Supply chain vulnerabilities are multiplied in different ways.
In part, it's because attackers can find vulnerabilities more easily, and on top of that, they can actually phish.
Like the Axios vulnerability that happened right now had a phishing element, which sort of combines the other capabilities of AI.
But then also, the agents are pulling in lot of these libraries and they're done with less supervision so there's just this trifecta of whatever multifacta you know of reasons that drive it and so I think in the In the DevOps world, as is often the case, right, in the dev and DevOps world, it's a very positive message.
It's like you can do this and we can all sing Kumbaya together and, you know, sleep well at night and doing it because the systems will kind of be self-healing and we'll build that.
And self-healing is a DevOps term, right, that has existed for a while.
In the security world, there's a bit of a, you know, if you don't then type of risk scenario, which is we have to level up.
security handling to not just find vulnerabilities but actually fix them and then go all the way to deploy to production.
And that forcing function really, again, drives that sort of end-to-end speed.
What I like about it is that it's actually a case in which you should embrace full end-to-end agentic capabilities so that you are able to respond this quickly.
So again, an extreme version of the DevOps kind of cloud move.
in which initially it's like, well, it's insecure to deploy multiple times a day or whatever to have continuous deployment.
And then it became, well, it is insecure not to do that because the vulnerabilities will happen.
The systems do exist on it.
So the question is, how quickly can you respond and reach out?
And so I think we'll see a lot of that.
And I would expect this year to be security to be very much at the forefront as agents are being used for much more sort of real production cases.
So you'll see both of those.
You'll see the security issues because agents were not used in doing it.
Like people were just sort of slow.
Attackers are becoming faster than defenders.
And there's a lot of understanding and awareness of this in the security industry.
not real solutions yet.
And the second is you'll see security issues because of failures on the agent side and unreliability.
We've seen some leaks from Anthropic, whether it's cloud code that they've quickly clarified is a human error.
It's like it's the button pusher human who's being thrown under the bus.
And we've seen sort of the leak of Mythos, somewhat ironic, like super cybersecurity model having a cybersecurity leak on it.
But there will be more and more of those.
Yeah.
And do you think, you know, for the 3am kind of wake up.
Do you feel like that would turn more into a human in the loop validation versus actually, like you say, the important pieces, the activity, the action, the root cause analysis, the fix and getting all of that done quick.
Do you still feel like there's an amount of time where we need human in the loop to build that trust?
Or do you feel people will?
Yeah.
Be more immediate.
I think really the best setup is to make yourself error tolerant.
And so what you want is you want to have an environment in which you let the agent do it, but you build the infrastructure such that if the agent has made the wrong decision, it is reversible.
And I think that's really what we aspire.
I increasingly think about the mental model in which, you know, if you think there were some things that were tolerated in Waterfall and were no longer tolerated in the cloud era.
And the best teams in Waterfall were automating things, were finding issues early, like shift left in security is an example of that.
And then they were no longer tolerated in cloud.
You could not get by without shifting security left or by having automated tests of some capacity because you just couldn't deal with the speed.
I think there's similarly now a few things that are tolerated in the cloud era and will be not tolerated in the AI era.
largely due to speed.
And many of them come down to the same pattern of these were human actions, human reviews that had to be done.
And the best teams were automating these things in the cloud era.
But, you know, in the AI era, the average team needs to be doing that.
Otherwise, it would not be viable for you to build in this world.
Let's switch gears a little bit now to Mati Stanaszewski, who is the CEO and founder of Eleven Labs.
He has some opinions on how we interact with AI.
Let's hear from Mati.
Like in general, we think voice will be the future of interactions, the digital interactions of how you interact with interfaces of the digital world.
And it can carry so much more emotion, so much understanding than text.
So it will read your mind to an extent.
I don't know, like, you know, maybe go to Elon's kind of a Neuralink or some scary options over there.
Yeah, I mean, I think voice is definitely prevalent on it.
It's interesting because it's less about just typing speed and more about our human tendencies.
When we speak over here, there's ums, there's ahs, you know, there's you know, you know, and all those idioms.
The audience don't know that.
That gets edited out, guy.
So, you know, in our automation technology makes that go away.
And when you type something, then you bother fixing these things.
You pause, you think.
So it's actually not just related to your time of sort of typing down.
It is a lot more kind of related to how much do you need to phrase, how much can you ramble on on it.
And so it's been really interesting to see how voice and speech have become common in the world of AI.
uh in in the coding because coding requires that precision you know when you code you can't you can't say you know you can have an you know in the uh in that interpretation but when you are discussing things right and talking to an agent then that's okay the agent can remove those things and they can take that away and so it reduces the cognitive load it just allows you to to ramble on there's still value in precision when you're instructing something to the agent.
But if context is managed well, in this case it's inline context, you ramble something, the agent might create some instruction files.
That instruction file might be the thing that it passed to a sub-agent to implement things.
then it's, again, if you use the human analogy a little bit, this is sort of the person, the project manager, right, that you are talking to, and you can ramble on to them, and part of their job is to sort of streamline your ramblings to be able to translate that into real conversations.
So it's interesting.
I think Mati is right.
It was clearly a little bit self-serving to talk about sort of voice being the future of it.
You know, 11 Labs continues to...
sort of shine and thrive since we've had that episode.
Interestingly, they've actually expanded past audio.
And so now they don't just do audio.
But I think you'll see more and more of that text in the AI dev world as well.
You're seeing there's an interesting pattern in Cloud Code, and I think it probably will be, if not already, mimicked by others, to delegate to use subagents a lot more.
That, to me, is that analogy of the front.
agent that you have there is the project manager and you're interacting with it and then it will delegate activity to sub-agents to do the work.
And I will say that just like with project management, there's a caveat about losing visibility.
So when you're in the terminal and you're giving those instructions, it's a lot harder to know what happens when you're making that delegation.
So it's interesting, but I think generally correct that voice will be a...
And of course, I think Claude Code released a feature actually probably only a month ago or so.
If you put, there's a slash command now, slash voice, I think, or something along those lines, where then it actually listens to the mic and you can actually put from your laptop and you can actually just chat with Claude Code.
And it feels quite natural, actually, for me.
I enjoy doing that naturally with Claude Desktop, actually, anyway.
So it was a natural addition for me.
Definitely in agreement.
And it raises the, like, there's a very, it's a very different question to say, will a thing happen?
Like, what voice can be more dominant than text?
And who would be the players that win in that space?
So it is very interesting.
I will say that as we record this, you know, there's been some recent updates on the Claude side in which they've tightened up a lot of where can Claude be used.
And they really want their harness.
They want their interfaces.
And so we're probably, starting to toy a little bit with, as companies get more capable, and Anthropic right now is in that state, do they seek to form more of a closed garden around it?
And you just need to use our systems and run everything within our systems versus the other trends where you see, for instance, NVIDIA committing to open code and open claw and building things that are very open and composable system, which once again feels a little bit Cloud-esque, a lot more web-like.
And we had...
In an early episode, we had Matt from Netlify, the co-founder and CEO of Netlify, talk about the criticality of the open web and how do we build over there.
So they've continued to invest in that as sort of being a composable infrastructure while you're seeing Vercel actually definitely supporting some elements that are open, but also...
mostly investing in sort of the end-to-end experience and making a very good end-to-end experience within their kind of walled gardens, right, of it.
So it's just different questions.
What will happen and who will win or what does it take to win are just very, very different questions at the pace that this world moves.
Amazing.
Well, look forward to chatting about what's next in the next 100 episodes.
But for now, that's all we have time for.
Really fun to look back at.
kind of like, you know, think about some of the topics that were raised by such amazing guests that we've had on the podcast.
And I want to extend a huge thanks to all of you listeners who are really kind of the reason that these guests come on to sort of have these conversations with us, right?
And they...
just all the feedback that you get and the suggestions for different speakers and different guests that we will have.
So thanks.
Keep on listening.
Subscribe if you haven't already on it.
And if you have suggestions for guests that we should have in the next 100 episodes and beyond on it, send them over.
And of course, you know, to all these amazing guests that have sort of spent their time sharing their learnings, their perspectives, you know, with us, with you.
Big things.
Very humbled.
Very much enjoy the conversations.
Learn from the conversations.
And, you know...
At this pace, a lot of the kind of one of the critical elements of being able to deal with the space of change is community, is about learning together, sharing perspectives, sharing these learnings.
So we're very grateful for you for sharing on those perspectives and we'll continue to provide a stage for that to happen.
Seconded.
Well put.
Thank you very much for listening and tune in to the next episode.
Thanks, Guy.
Thank you.