# Agentic AI Strategy: Policy, Memory, and DevOps

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-09-16

## Transcript

I want to talk about 15 years, I think, that you spent on call.
That's a lot of 3AMs.
Do you feel like agents will very much be able to take that role from us?
With the basic investigations of on call, the log scraping, the diving into metrics and so on.
You know, if you look at something like AWS's DevOps agent, it's great at doing that kind of automation of, let's say, the more day-to-day aspects of on call.
And then what's left is the really interesting reasoning about why is the system behaving this way?
You know, what are we seeing in terms of large scale failures?
What are we seeing in terms of improving performance at big scale and so on?
And, you know, automation is going to help us there a whole lot, but it doesn't have answers to those questions today.
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
Join your hosts, Guy Pajani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
This is the AI Native Dev.
Yesterday, you were writing code.
Today, you're prompting agents.
Tomorrow, you'll be running a software factory.
Argentic coding is moving fast enough to give anyone whiplash.
This June, AI DevCon London sold out.
Hundreds of developers, engineering leads and VPs packed in to hear speakers from Anthropic, OpenAI, Netlify and GitHub cut through the hype and get to what actually works.
This November, we're bringing it back to New York.
Bigger, bolder and full of people actually building the Argentic future.
Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams, or setting the strategy.
Our early bird pricing ends on September 30th, so make sure you lock in your place before prices increase.
As a podcast listener, you can get 15% off your ticket using promo code POD15.
We have so much planned, and we can't wait to have you join us in New York.
Hello and welcome to another episode of the AI Native Dev.
And joining me today is Mark Brooker.
And Mark is a VP and Distinguished Engineer at Amazon Web Services.
Mark, welcome to the podcast.
How are you?
Well, thank you, Simon.
I'm great.
It's a pleasure.
It's a pleasure to chat.
And you're, of course, you're on the West Coast, right?
Are you in San Francisco?
I'm in Seattle.
In Seattle, a little bit further.
Yeah.
And so it must be raining.
I assume it's raining there, just like it is in the UK right now.
It is threatening to rain, I would say.
It's either raining or threatening to rain.
We've had our two weeks of glorious summer and we're back to Seattle weather.
Amazing, amazing.
I was on a call with someone from Kenya today and weirdly it was warmer in London today than it was in Kenya, which was, I doubt there are too many days where you could probably say that, but it's nice to boast every once in a while.
So, Mark, you've had an illustrious career so far.
Tell us a little bit about what you've been up to and some of the work that you do at AWS.
Yeah, so I've been at AWS for a long time, for 18 years now.
I've worked on a whole bunch of things, but the last few years has been really focused on agentic AI and agentic development tools and the infrastructure for agentic AI.
I was deeply involved in the development of AWS Agent Core, this kind of suite of tools for building agents in the cloud.
But most recently, I've been focused on building a team really looking at some of the future of agent building, agentic development.
So we own Strands, which is AWS's agent framework, open source agent framework.
We've got some really cool technology coming in the Strands space.
And then are building some new development tools, thinking a lot about the future of development and doing everything ranging from fairly basic research to a lot of code building, a lot of product building.
pushing the frontier of what it means to build software.
Amazing.
And actually, we haven't really talked about Strands too much on the podcast.
Tell us a little bit about what Strands is and who should be interested.
Yeah, so Strands is an open-source SDK for building agents.
It works with any model, any cloud provider, local agents, et cetera.
And the goal of Strands is to make it as easy as possible to build.
great production-ready agents.
And so the kind of minimal strands agent is just a handful of lines of code, but you can build out from there with tool integrations, you know, all built-in MCP clients, built-in open API clients, context management, all of the things that go into, you know, into building a great agent.
And the strands team is really trying to make that as easy as possible by being, let's say, very opinionated about good agent patterns.
And so you can extend strands, you can use it any way you want, but there's a really nice kind of paved path through strands that has these opinions of what we think right now it means to build a great agent built right in.
It's funny because I always think developers who have never built an agent before probably think there's actually so much complexity behind the scenes, but they can actually be very thin veneers around an LLM.
And I think it's very much encouraged people and listeners to actually build that.
And it sounds like this could be a very helpful way of actually doing that from scratch.
Yeah, for sure.
And I really just would encourage people to build.
build your own development agent.
I mean, it's not going to be as good as one of the major commercial offerings.
But very quickly, even in a couple of hours, you'll get this much better intuition for what's going on.
on under the covers of those agents, how they work.
And I think that kind of experience, and we will touch on this a lot more as we talk about, you know, how software development is changing, but that experience of knowing about what's going on under the covers, I think just makes people better users of even these great commercial products.
Yeah, yeah.
Amazing.
Mark, I want to go into a little bit of depth now, and I want to talk about something that was very interesting.
I think you mentioned in a book as well.
And it goes back to the time, 15 years, I think, that you spent on call.
That's a lot of 3AMs, first of all, right?
That's a lot to put anyone through.
That must have been painful.
You enjoy on call?
You enjoy those kind of roles?
I do.
And I just find them an incredible opportunity for...
Learning how the real world works, like learning how our systems actually run, you know, learning what our customers deeply care about.
And, you know, this is one of the challenges.
You come out of university and, you know, I came out of grad school with a ton of what I thought was practical knowledge, but a lot of it was quite theoretical.
And then on call was really this huge catalyst over my career for more deeply understanding.
how systems actually work, how systems actually fail, how we can stop them from failing, you know, how those things affect our customers and how we can help our customers out.
It's funny, Mark, because I always think a lot of people think they become a senior developer when they've done five or more years in development.
Whereas actually, I think it's more these types of roles where you get that real exposure into what it takes to build a real app that's, you know, reliable and et cetera, et cetera.
Is this still, you know, talking about the future of development with AI, do you feel like this is going to continue this?
type of on-call is going to continue?
Or do you feel like agents will very much be able to take that role from us?
You know, I think, you know, and this is going to be just a major theme of software is we're going to see a huge amount of automation of increasing automation of the rote things, right?
There's been this kind of failure.
And that's not new.
If you look at the whole history of cloud and computing, this has been a theme.
We used to get paged for single host failures.
But now, AWS autoscaling 15 years ago just started dealing with those things for us.
And load balancer health checks started dealing with those things.
And so we automated those away.
And we're seeing exactly the same thing with AI.
With the basic...
the basic investigations of on-call, the log scraping, the diving into metrics and so on.
If you look at something like AWS's DevOps agent, it's great at doing that kind of automation of, let's say, the more day-to-day aspects of on-call.
And then what's left is the really interesting reasoning about why is the system behaving this way?
What are we seeing in terms of large scale failures?
What are we seeing in terms of improving performance at big scale and so on?
And, you know, automation is going to help us there a whole lot, but it doesn't have answers to those questions today.
Yeah.
And I'd like to think that when I do wake up at 3am and other early morning times are available, but let's stick with 3am, that an agent has already been through the logs and been through all the information, you know, the grueling part of the debugging of, or not the debugging per se, but trying to identify what a root cause could be.
And has already provided that as part of, you know, the on call where I'm woken up and presented with all the information that I need to make a decision.
I'd love to know, as part of the, as part of the, you know, your writing in the past, you've said around, that you've done around 3,000 or you've read 3,000 to 4,000 post-mortems.
I guess, what did that teach you that you wouldn't have really been able to learn without being on call?
So as a typical developer trying to build, what did the post-mortems really open up for you?
One of them is a deep sense of humility.
You know, as engineers, we tend to be optimistic, which is great.
You know, we tend to think, hey, we can build this, we can solve this problem, which is fantastic.
And then, you know, you see how things fail and you see when things don't work and you get a dose of humility from that and a dose of sort of closing that loop of, okay, well, I'm going to be optimistic.
I'm still going to believe I can do this, but I'm going to do it in a better, smarter way this time.
One very concrete thing, for example, is over the last 10 years or so, there's been this increasing familiarity with a concept that is now called metastable failures.
There's these failures that large systems have where something will happen, whether it's an overload or a component failure, the system will go down.
immediate cause will be repaired and the system won't recover because it gets stuck in this sort of stable down state.
And, you know, deeply understanding that has been super important to understanding how to avoid whole classes of long and painful outages.
And I think a lot of our understanding of that, and certainly personally a lot of my understanding of that, has been...
come from looking at real world incidents, looking at when that happens, and then analyzing the dynamics of the system during those times.
And, you know, we could have approached that entirely analytically.
We could have approached that from the outside.
But I think we've learned a huge amount more by looking at how real systems fail, both, you know, our own and systems across the industry, and really trying to extract like...
What was it that kicked the system into the state?
What was it that kept it down?
What was it that led to recovery?
How can we make systems more resilient against those things?
How can we make that recovery faster?
And to your earlier question, how can we build automation that can make that recovery automatic when we're in one of those states?
Yeah, yeah, super interesting.
And I think that learning and feedback is super key.
And I think for us as humans, you know, pre-AI being mainstream, it's what we would have learned and what really would have leveled us up to that senior developer.
And I think your point on humility is so valuable.
I think you go in with a different mindset when you build code and you think, right, I know this is going to break.
I just don't know how or when.
And you look at it quite differently.
So I love that answer.
I think that's brilliant.
I'd love to understand, we as humans can learn.
from not just our mistakes, but our experiences, our view of other postmortems and things.
How about agents?
When we think about loop engineering, for example, and we think about how we can take these loops that will automatically understand how agents could have done better, potentially updating context, updating skills, so that when we write and build, applications that are similar, we can potentially circumvent, you know, work around certain problems or at least guard against them.
Is that realistic or is that kind of like a pipe dream in terms of the type of bugs that would be raised to a postmortem?
Oh, that's very realistic.
And very much that is, you know, what is happening today at the frontier of these agents.
a variety of techniques.
And so, you know, one of the techniques that you have is to say, hey, I've got all of these postmortems.
I'm going to put them in the training set of the model.
And so you can improve the model.
Another way is to say, I've got all of these postmortems and I'm going to give my agent a tool to search over those postmortems semantically and say, show me incidents like this one.
And then I can understand how I recovered from that.
And then at that point, that agent can write its own memories and say, I tried X, it didn't work.
I tried Y, and it worked.
And so next time, it will retrieve from those postmortems, it'll retrieve from its memories and jump ahead in that phase.
Increasingly, agents are also seeing, hey, I did this step and this step and this step.
I could encode all of those into a piece of code and build myself a tool and put that tool in my memory, this kind of procedural memory, and remember that and just pull that off the shelf when I need it.
And so there is building those libraries of context for agents.
There is building great, you know, allowing agents to build and manage their own memories.
There's model customization.
And then there's both manual and automatic tool building for agents and a whole variety of other techniques.
And each of these things can help us make agents learn from experience and get better and better at solving certain types of problems.
more reliably, but also more quickly and more cheaply.
And do you think that loop of learning and understanding, particularly around, you know, incidents that occur, do you think that's most valuable during an incident where it can actually learn and understand where possible root causes are or, you know, it's likely it's going to be something like this?
Because the last four were similar or I saw a similar footprint before.
Or do you think that learning can be taken down into the architecture decisions and the development decisions for future building?
Yeah, well, I hope so.
And that's a bet that we're making.
I think we haven't seen as much of that yet.
And maybe to take that back to the human level, one of the things that we've always done at Amazon is our...
Our product teams, the same people who are building our products, are on call for those products.
And we do that primarily because it drives exactly that learning loop, right?
You get that production experience, you get those customer anecdotes, and you bring them right back into development.
Whereas separating those things out into separate teams, although it does have its advantages, makes that learning loop much harder to drive and much more abstract.
And so when we get into the agent world, what we want is that our coding agents, our development agents, our software building agents are learning from production experience in exactly the same way that operations automation agents are.
And there's a path to that with memory, with sharing context, with sharing those memories, sharing tools, et cetera.
Even using that development agent as a kind of expert consultant for reviewing a software design.
But I would say as an industry, we are earlier on that journey than we are on the optimizing the individual pieces.
Nice.
I remember when prompt engineering was a thing where you used to be able to write something to say like, oh, by the way, another agent is going to review this.
So make sure you do it well.
And I love the idea of just thinking, oh, by the way, you're going to be on call for the code you write.
And you might be woken up at 3 a.m., Claude or Gemini or whoever.
So that could be that could be kind of fun.
I'd love to know in your experience, what can an agent not?
What's it not possible for an agent to learn from a postmortem?
And I guess this kind of like leans more into where we as humans need to be very understanding of where we play a role.
So from the postmortem, what do you expect an agent just to breeze over?
Well, I think the things that agents can solve are the ones that have clear, definitive outcomes.
I have a graph of...
orders on Amazon.com or I have a graph of EC2 instance launches and it should look like this, it does look like this, figure out why there's that gap and drive that closed, right?
There's a very clear goal.
The ones that are harder to solve are the ones that are, well, you know, I'm hearing on Twitter that customers are seeing a few more errors than usual or People just seem less happy with the product or things feel generally slow when I open it.
And so the more ambiguous things.
And so agents can help you dig into those and can help you find the metrics that might be driving those perceptions from customers or driving those anecdotes.
But generally aren't as good at really...
Understanding the end-to-end context of the work.
How are customers using the system?
What does it mean for it to feel fast?
What does it mean for it to feel reliable?
And, you know, as we know, encoding those kinds of things in metrics is often very difficult.
Yeah, very, very true.
I'd love to...
jump across now to a move that you made, actually.
After a couple of decades in storage, computing database, you moved across to agentic AI safety and policy.
Which is actually very topical, given the time which we're recording this episode.
Of course, Daria just mentioned about how Anthropic should slow down its development of models and is kind of talking about others to do the same.
I know Sam Altman agreed, so did Elon Musk.
First of all, I'd love to hear your opinion on that.
And then secondly, I'd love to understand...
Why the move?
I'd love to go deeper into that.
Yeah, I don't have an opinion to share on model development and the speed of that.
Very interesting set of trade-offs in that space.
What I will say is my focus here is on what I'm hearing from customers and what I'm hearing from developers.
One of the very common themes that I hear from customers is, hey, we built this awesome agent.
We use strands.
We built this awesome demo.
We put it on agent core.
It looks great.
The board is super excited about the demo.
But now we're really worried about how we get it into production, how we make it reliable, how we make our – so our customers can trust that agent to do work on their behalf.
And a large part of solving that problem is this idea of agentic policy, right?
What does it mean for an agent to do the right thing?
How do we very clearly define what an agent can and can't do?
You know, what of that belongs to the environment and the harness and the containment of the agent versus what of that belongs in the prompt, tool descriptions, etc.
And what our customers are asking for is ways to think very clearly, very deterministically about how to build a box around agents and say, I'm going to trust this agent to do.
This thing, this thing, and this thing, but it is not going to have any access to do, you know, these things.
And a lot of people approach that as a classic authorization problem, as you would with humans.
Say, hey, you know, this human has this authorization to take this step.
But humans have a level of judgment.
They have a level of feedback that agents lack.
And classic authorization languages lack a lot of the power that customers are looking for for describing what their agents need.
And so, for example, one of the things we did is we recently launched Dogwood, which is this new policy language.
Happy to go into depth on that.
But that is built as a specialized language for describing.
With mathematical certainty, what an agent is allowed to do, what order it's allowed to do it in, whether it can do thing A after thing B and so on.
And so it really all comes out from these customer needs and the fact that, you know, across the industry, people are saying, hey, agents are super exciting.
We're really excited about what we can do with agents.
We want to automate more.
We want to serve our customers better.
We want to spend our humans' time doing more important tasks.
And a key to doing that is having a great security story and a great policy story, a great isolation story, a great identity story.
And so we can build agents that we can trust to run autonomously and do critical things within our companies.
those boundaries, those set deterministic boundaries without having to make their own decisions or do anything in a non-deterministic way.
They need those boundaries that whereby they can say, yes, you can, you have access to this or you can do this or no, this is, you know, beyond what we want to provide you within this container or this space.
I think that's, you know.
so important and I think the last month or so has actually shown how important that is in terms of that belief and trust.
And as we go forward with autonomy and inter factories and long running processes, how do you see people using this kind of a tool to run their containers?
Do you see like configurations that people will essentially be assigning on various tasks?
How would someone set this up?
Yeah.
Well, one of the things that we know from decades of research into psychology and operations and operators is that humans are really bad supervisors.
They're just really bad at watching things go by and pushing the red button when something bad happens.
And, you know, there are multiple dynamics behind that.
You know, one of them is this classic research from the 80s.
There's this great paper called The Ironies of Automation about how, you know, as a system becomes better and better and better, the failure modes become weirder and less frequent and becomes much harder for the human supervisor to understand when they happen, recognize them and react correctly to them.
And so I think the only solution to this is that autonomy, right?
Pull the human out of the loop.
Make them not watch the loop go round and push the button when something bad happens.
Certainly not make them say, yes, yes, yes, after every code generation task, because who's going to do that?
But instead, pop the human out and say, Let's start by describing what good looks like.
Let's start by describing the set of things that you would like the system to do.
And then once we have that description, we will apply it to the agent without requiring a human to go around and do those things.
And so how do you do that?
Well, some of that is encoding of, it's encoding of common sense, right?
You know, as you develop a piece of code, you know, what kinds of interactions it should have with the system, right?
It's going to do, you know, git pull and git push, but only in the repository it's working in, right?
If it goes and tries to do a git push to a different project, well, that's probably bad.
And so we're going to encode that little piece of common sense into a policy.
And part of that is...
writing that kind of library of those common sense policies.
Part of that is this process of natural language to policy where you can say, you know, this can read anything on my hard drive except the stuff in my secret documents folder and have that automatically and reliably turned into a policy.
And then part of that is going out over a company and discovering all of the policies and practices and so on, and encoding those existing practices into policy.
And so to answer your direct question, I think the minority case is people writing policy directly.
I think there are going to be some people writing Dogwood or Cedar or OPA or whatever policy language they're using, but the vast majority aren't going to engage at that level.
and don't want to because those languages are not quite specialized and fairly difficult to learn, but instead are going to be engaging at that natural language level of saying, you know, here's what good looks like.
There could be me as a developer saying, here's what good looks like for me as a person.
Or it could be, you know, somebody in the CISO's office saying, you know, here's what good looks like for us as an organization.
Here's our standard and policy for handling customer content and our standard and policy for handling our code and then encoding those.
And often all of those things will come together.
And so I'll have some things from my CISO's office.
I'll probably have some things from legal.
I'll have some things from myself as a developer.
And all of those go together and provide this shell around my agent where we can strongly and deterministically make sure that the agent is following all of those policies by using one of these lower level languages.
Yeah.
I was going to ask, actually, whether you see this as something in your experience that is an organizational level or a department level.
from legal, from security, is it majority from those teams down to developers or is it closer to a 50-50?
I think it's closer to a 50-50.
There are some things that my security teams deeply care about.
The way that customer content is handled and where it is allowed to be in a system, for example.
And I want every developer in the company's agents to be held to, maintaining that standard.
And then there's some things that my security teams don't care about, right?
Like, can my development agent delete documents from my documents folder?
Well, my security teams don't really care about that because there's no, you know, there's no customer impact of that.
There's no organizational impact of that.
But as an end user, I care about that super deeply.
I don't want that stuff deleted.
And so I'm going to, it's going to be a mix of these things.
And You know, especially as we get down to individual developers, we know that across the industry and certainly, you know, even within teams and within organizations, development practices vary.
There's a wide range of what normal looks like.
There's a wide range of tools that people use.
And so...
Just like most big software organizations don't try and push down a you will use exactly these tools at this time type software engineering practice, they're not going to succeed at pushing down a complete end-to-end this is your agent policy because that needs to be adapted to the kinds of development you're doing.
And even day to day, right?
Like sometimes I'm writing back end Rust code and I want to set a certain standard and I want to write things like, hey, you aren't allowed to commit before Clippy is happy with this code.
And sometimes I'm writing, you know, a quick and dirty prototype of a web demo.
And I don't want to set any of those standards.
And I'm just like, okay, we'll just hack this thing together with whatever tools make sense because we're going to show it all for 15 minutes and then discard it.
And so I'm going to have different policies.
And so those policies do have to come from different places and be dynamic and be contextual.
Hey, everyone.
Hope you're enjoying the episode so far.
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
But for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
I want to go to another piece that you wrote about, which is the agentic software development hypothesis.
And you also mention a number of different kind of forms of this, which is the weak form, strong form and strongest form.
I'd love you to walk us through the agentic software development hypothesis first and we can kind of cover the rest.
Yeah, you might have to remind me exactly what I wrote in that post.
But essentially it comes down to this idea that the hard part of software building is, I'm going to say specification building.
And what that means is describing what you want.
And describing what you want in sufficient detail that it is, let's say, a complete description of what you want.
Once you have that specification, once you have that what you want written down, my belief is that we're going to see the building and maintenance of reliable, efficient implementations of that software become, over time, entirely automated.
We're going to see that first, and I think we've already seen that with...
Let's say these closed specifications, hey, I want something that implements this type of file format, and it's very well described.
And we're already seeing that.
And we're going to see that grow to larger and larger systems, systems with more UI, systems with more complex interactions.
But I think over time, all of that kind of specification down of software development is going to become automated and driven by an automation loop.
And the kind of stronger claim is that the vast majority of software can be specified crisply enough to enable that kind of automation.
Yeah, absolutely.
Now, in that same post, you mentioned that very few tasks have complete specifications and you also talked about oracles that can be deterministic and you mentioned oracles aren't deterministic so i wonder are we relocating the the the the tough piece of the problem uh to be in identifying an article that's deterministic or being able to create that complete specification is that still more of a human issue Yeah, for sure.
I think that is where a huge amount of the human work comes in, is creating those oracles, creating those specifications.
And in some sense...
testing is now the most important part of software development.
And, you know, some people will say that it always has been, but it's never been the shiny object for most developers.
But now the thinking about how do I build a test that says, you know, whether this piece of software is good or not is just really at the frontier of software development.
And some of that is going to be specification.
Can I write in a formal enough way to describe that in a deterministic way?
And again, like if I'm implementing GZIP or something, yeah, I can.
I can fully specify those.
If I'm implementing a UI for a customer, maybe I can't.
And so I have to use other forms of correctness oracles, other forms of evaluation that might be more.
open-ended, they might be customer in the loop, et cetera, where we get that feedback and feed it back in.
And if you look at the kind of automated reasoning level of specification, there's been this incredible progress in specifying software in languages like Lean.
And a huge growth in just addressing bigger and bigger chunks of software with that.
But as we expand those components to entire systems and those systems have complex interactions and so on, that becomes more difficult.
And so I also want to build these oracles that can look at a system and say, is this working well?
And that might be looking at those production metrics.
It might be looking at customer feedback.
comparing doing a fuzz test between two implementations of a database engine, for example, and a wide variety of other techniques.
Yeah, amazing.
And it's so cool to see.
I remember back in the early days, I think we had some folks from Amazon when we talked about Kiro, when that was first launched back in the early days.
And yeah, specs are such an important piece.
And I think it's definitely got a place in agentic development for me in terms of being able to truly describe.
a function, a feature, an app, and be able to have that as the source of truth going forward.
I think there's a super compelling piece there.
One thing I always kind of struggled with was delineating almost what is part of the specification and what is part of the test, because very often you can write something in a specification that is essentially describing a test.
And similarly, when you're...
talking about a test, writing or describing a test, you're actually talking about how something should function, which is kind of part of the specification.
And for me, almost like the functional specification as well as the tests almost merged into one.
And it's like, well, it could be both.
It could be the same thing.
But yeah, super, super interesting to see how that...
Yeah, you know, and testing is a, I would say, informal specification technique.
And, you know, always has been.
You know, I love the things the Kiro team has been doing with Spectrum and development.
There's been some incredible advancement there.
I think looking back on that, And the thing I feel a little bit unfulfilling about the current moment in Spectrum and development is there's still quite a lot of conflation between what are those requirements, those kind of declarative requirements, and the more implementation detail-y stuff, right?
Like there's still, oh, well, I want to build this in this language or for this kind of UI.
And ultimately, I don't want to care about those things as a system builder.
I want great decisions to be made, but great decisions to be made more automatically.
And the specification and the tests and so on should focus on what good looks like for the output of the software, not for the way that it works inside.
Yeah, no, super interesting.
I want to now switch a little bit to talk about, I think this is something that you said, a while back, I'd love to know if it's still true that you won't let AI write a word of your blog, but you're very comfortable with it writing AI-generated code for you.
So first of all, is that still the case?
Yep, that's right.
It is still the case.
What's the difference between blog content and code?
Yeah, it's a really great question.
And so I think it's a difference in intent.
And so I think five or so years ago, I would have said with strong conviction that the purpose of most code is to communicate with humans.
And the execution of that code is important, but not its primary purpose.
And so I was always very big on comments, very big on naming things carefully, very big on writing code in a way that communicates, especially the intent of the code.
I think that's changed.
And it's very quickly changing across the industry as, you know, as humans get out of that code level into more of that kind of specification level, as a lot of that coding becomes, you know, fully automated, is code is turning more and more into purely an implementation vehicle rather than a communication vehicle.
My blog, on the other hand, right, I'm saying.
This is for people to read and for people to engage with.
And in my mind, there's a kind of social contract behind that where I'm saying I've spent a bunch of time deeply thinking about these words.
I'm going to share these words with you.
And what I'm hoping you're going to do as a reader is spend some time deeply trying to understand my words.
Whereas if I get Claude to write a blog post for me in five minutes, and then let's say 10,000 people read it and they each spend 10 minutes reading it, there's this huge asymmetry between the amount of effort that I'm putting in and the amount of effort that my readers are putting in.
And I think that is unfair.
And if I take that into work, like Amazon's very doc-driven culture, I write a lot of documents for work about plans, about technology, about the things we're doing.
And again, that is a contract between me and the readers.
And I am expecting them to deeply read and understand and engage with my words.
And they're expecting me to have deeply considered those words and have full ownership for them.
And, you know, if they're Claude's words, I haven't done that.
And then the third part of that is, you know, if you have an idea and.
want to share it with me and want to share Claude's thoughts on it with me.
Actually, I think that's great, but let's do that.
Let's make that interactive.
Send me the prompt.
Let's have a chat with it.
One of the cool things about these agents is that I can go around the loop and ask follow-up questions and so on.
And so if that is the mode of communication we want to choose, let's choose that mode of communication.
It's not just land in the one that's in the awkward middle.
which is, you know, all of the human care taken out without adding in the interactivity and possibility for follow-up questions and so on, which go into that agentic interaction.
Super interesting.
And I think it's, yeah, for me, I really love the way you say, you know, you care about the fact that it's people's time.
And there's obviously one to very many.
People who are going to be reading that and taking the time to read that versus one person's time to actually write it well in your words.
Kind of something that strikes me here is the you've obviously got the way you say something and then the, you know, the core of what you're trying to purvey are both as important as the other to you there.
Or if you could get Claude to be able to.
write something in a reasonable way, but if it could really capture the message you're trying to share, would that be reasonable?
I guess the thing I'm saying is, what would it take for you to allow AI to be able to write on your blog?
Yeah, that's a good question.
And at some level for my personal blog, it is a philosophical thing.
where I just, you know, I want to say this is my, you know, these are my thoughts as I wrote them, with my mistakes as I wrote them, and so on.
I think for those, you know, kind of work documents, you know, if I could get Claude to write succinctly, you know, just describe the context, pull in the business context, and so on for me, I would feel quite, you know, feel quite comfortable with that.
And so maybe the distinction there is if you're using Claude as an expander, right, saying, here's a prompt, turn it into a six-page document, I think that's a waste of the reader's time.
However, if you're using Claude as a context provider, right, like add a paragraph with the business metrics for this week, I think that's fine because you're bringing in new information from the human.
And it is almost exactly the same form I would have done that.
I would have done that manually.
But I think for my blog, I still think there is a level there which I want to say it's a piece of craft.
Yeah, that's the word I was going to use, actually.
It's about, can you scale craft?
And I think sometimes craft is best.
left as a craft and and that's where people get the joy in both creating it and also consuming it knowing it's not just you know there's no chance of slop it's it's straight from there straight from the hand and I think in terms of you know feedback I've certainly had a player and I think it's very very hard to be able to capture that level of feedback that still you know offers in a genuine way the written word in the same way that an individual would write it.
So I really like that.
While on the topic of feedback, let's jump into the kind of advice or being able to suggest what our listeners can do.
In terms of feedback signals that agents can use, what's the...
one of the key most important things that a team can do today to improve the feedback signal, so to make sure that agents get the clearest signal to use to improve on future runs?
Yeah.
So I think the answer to that varies a lot about where in the agent, agentic development maturity hierarchy you are.
And so there are a lot of teams still in our industry who are just starting out, right, who are just getting their feet wet with agents.
And there's just a huge amount of, I think, the things that a lot of people would sort of roll their eyes at now that are very effective, like that agents.md file and writing some directions for the agents about what good looks like into those things.
You know, definitely not a cutting edge technique, but something that is proven to be effective.
I think a lot of teams early on try and jump to the end and say, oh, I'm going to do parametric memory or something when, you know, you can't ship code until Clippy is happy with it and it passes all the tests is a very effective thing with very little, you know, very little overhead.
Then, you know, as we go up those...
And the maturity spectrum, I think where things get really interesting is making sure that agents have a way to remember team context and remember past conversations.
And part of that is just a genetic memory, the kind of memory that is built into a lot of the agents.
They will remember things across sessions.
And if you say, don't do this again, they'll remember it.
Sometimes too well and sometimes too much, and that can be annoying.
But the problem with agentic memory is that it doesn't spread well across teams or even across people.
And so you need a way to capture those memories and to write those down in a way that the whole organization or the whole team can use them.
And this is where I'm seeing a lot of people use these.
agent wiki type patterns, which are quite popular right now, where essentially I have my team agents maintain a number of, just a small wiki of pages about what good looks like.
And that is shared, you know, between the agents.
And then there are tools like AWS Context that allows agents to build a formal graph of of the context in its environment, connect that back to memory, connect that to documents, and do that across an organization.
And part of that is that dynamic discovery and learning, and part of that is pulling in the right information from my organization where I already have a whole lot of that written down.
And again, people sort of roll their eyes at RAG and be like, oh, well, that's so 2004 or 2024.
But the bottom line is that giving agent semantic search over the information in my organization is still a super powerful technique.
And so there's just a lot of that.
I'm going to plumb that in.
I'm going to give my agent access to that.
And what we're seeing with the...
With the commercial agents, especially if you look at something like Kero or you look at something like Quick from Amazon, more and more they're making it very easy to pull in that organizational context and pull in that team context and share those memories through things like AWS context to bake in what these good kind of learning from experience patterns look like across an organization.
Amazing.
I'd like to wrap now by...
Another question around advice, but advice to individuals and advice to, you know, perhaps juniors.
And let's circle back all the way back to the front of this, to the top of this podcast episode, where we talked about the three to four thousand postmortems that you've read in the past and how that very much helped you and you learn from that feedback.
Your advice to juniors is own a project, talk to a customer, own a deadline, and going back to the front, also essentially learning from those mistakes that we as teams and individuals have made in the past.
But historically, these are the tasks that senior developers have historically done and often guarded.
Do you think it's reasonable to expect senior developers to now give these types of tasks to junior developers, or do you feel like they're going to hang on?
And I guess this is the age-old question about how we bridge that gap between a junior and a senior developer.
Yeah.
Well, we have to.
You know, I think the software industry is going to grow, you know, just a huge amount in importance and economic impact over the next decade.
And the bottom line is we're not going to do that growth and we're not going to have that growth if we pull up the ladder and, you know, the community of software engineers just gets grayer and grayer.
You know, we're not going to build this fantastic new industry on a...
you know, a bunch of folks with, you know, gray flecks in their beards.
And, you know, experience and seniority and, you know, having been around still matters a huge amount.
Deep expertise still matters a huge amount.
But we absolutely need to build this industry on a strong culture of teaching and growing and bringing new folks in, training and learning.
And it's tough to know exactly what that's going to mean tactically day to day, right?
Like what does that look like?
But I have no doubt that that is going to be a critical part of what our successes in industry is going to look like.
And what the success of individual organizations is going to look like is figuring out what is that path for bringing people in from things like universities who often have fantastic.
And very fresh theoretical knowledge and ramping up on the practical knowledge on the day-to-day.
And we've got to figure it out.
It's going to be a big differentiator for organizations going forward, their ability to bring in new people and bring in new ideas.
And the benefit of that is the folks that I see coming out of college who've spent a ton of time learning with AI, learning with agents, you know, without, you know, some of the preconceived notions a lot of us come towards them with, just come in with so many fresh ideas and perspectives.
And that's a fantastic, you know, addition to an organization or a development team too.
Yeah, I totally agree.
And I think it's...
like you say, almost like a joint responsibility from the organization and senior engineers to make sure that, you know, these folks are included, the juniors are included in a lot of the decision making and brought up to speed on this.
And I think it's a very understated, undersaid thing that actually junior engineers, they are truly AI native.
Like, you know, from our point of view, we've got years, decades of...
you know, development and, you know, this old way of tapping a keyboard and code comes out.
We're trying to learn the new AI ways, the AI native ways, but people who will, you know, come through, you know, the newer courses, the newer curriculums, which are truly very AI focused.
This is how they grew up.
This is how they learned it.
And I think we can absolutely learn from the, from the fresh perspective.
So very much plus one to that.
Mark, this has been a wonderful chat.
I really enjoyed the conversation and in particular, some of your insights.
I love the fact that you read 3,000, 4,000 post-mortems and things like that.
Mark, you've written a number of pieces, blogs and things like that about all of this.
You have a very insightful blog with all of this on.
Where can people go to read that?
Yeah, you can go to my blog.
It's at brooker.co.za.
And I tend to share things on X, on LinkedIn, and a couple of other social platforms.
But my blog and its RSS feed, very old school, is the primary source.
Amazing, Mark.
Thank you so much for your time and sharing all that wisdom with us today.
Well, thank you, Simon.
It's been super fun talking to you today.
Amazing.
And thank you very much for listening.
I hope you enjoyed that and tune in to the next episode soon.
Bye for now.
The AI Native Dev is brought to you by TESL, the package manager for skills and context.
Your hosts are Guy Pajani and me, Simon Maple.
Our producer is Tom Dowler.
The AI Native Dev is not just a podcast, it's a community.
And we host monthly meetups at the TESL offices in central London.
Visit tesl.io forward slash community to learn more.
And I hope to see you there.