# Agent Baseline: Securing Autonomous AI Software Factories

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-10-22

## Transcript

There's a very famous incident called Pocket OS where unfortunately, you know, the user of the agent wanted the agent to do something in staging, it found keys in production, ended up deleting a production database, caused a significant amount of damage.
This happens all the time.
You know, agents are more and more happy to escape sandboxes, right?
And while in most scenarios, and the wide majority of scenarios, an agent will do exactly what you ask it to do, In some percentage of use cases, it will do things that are not your intended action.
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
Join your hosts, Guy Pagiani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
This is the AI Native Dev.
Yesterday, you were writing code.
Today, you're prompting agents.
Tomorrow, you'll be running a software factory.
Agentic coding is moving fast enough to give anyone whiplash.
This June, AI DevCon London sold out.
Hundreds of developers, engineering leads and VPs packed in to hear speakers from Anthropic, OpenAI, Netlify and GitHub cut through the hype and get to what actually works.
This November, we're bringing it back to New York.
Bigger, bolder and full of people actually building the agentic future.
Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams, or setting the strategy.
As a podcast listener, you can get 15% off your ticket using promo code POD15.
We have so much planned, and we can't wait to have you join us in New York.
Hello and welcome to another episode of the AI Native Dev.
This time we are recording this live from San Francisco at the AI Security Summit run by Snyk and also the AI Engineering Group.
Here we have a wonderful panel and we're going to be talking all about agent baseline.
So this is going to be a fun session.
Who do we have with me?
So on my direct left, we have Eli Aylena from Docker.
Tell us a little bit about what you do.
I work for Docker.
I'm a VP of product strategy at Docker.
I help Docker to work with some of our most strategic customers and help kind of direct some of the initiatives we do.
Amazing.
Next, Ellie, is Ian Livingston, who actually was on the podcast not so long ago with Guypo.
You work for Keycard.
Tell us a little bit about yourself.
I do, co-founder, CEO of Keycard.
We're working on agent identity and access.
So how do I adopt agents?
How do I build agents?
How do I control what those agents can do?
across the board.
How do I make my developers and my business users as successful as possible with agents today and into tomorrow?
Awesome.
And finally, but by no means least, we have Marcelo Sosa from Snyk.
Tell us a little bit about yourself, Marcelo.
So I basically help customers safely adopt AI, build agents, bring defensive capabilities into the hands of the defenders and just try to protect the world bit by bit.
Awesome.
Now, we're going to be talking about agent baseline.
Ian, why don't you kick us off?
In a sentence, what is agent baseline?
It is six controls.
And those six controls are ways for you to think about what security controls are in place in order for me to take advantage and use autonomous agents in my organization or agents at all in my organization.
Do we really need secure agents?
No, you didn't need to answer that.
Martello, who is this for?
Is this for a CISO or is this for engineering teams that are actually building with agents today?
I mean, the way I see it is that, you know, it's really for people that are trying to like understand like what are the security boundaries that they want to have when they're basically adopting agents and getting agents to do, you know, autonomous tasks.
Obviously, we are focusing a lot on the software factory.
So like, you know, people that are actually using coding agents.
And so like, you know, why there's a security persona.
Obviously, it also matches up on the platform teams, you know, like how they use like dev containers, for example, and kind of how they get sort of boundaries of where agents are operating.
So we kind of see this multiple.
personas both on the security on the platform teams like playing a role here.
And you mentioned software factories there.
Eli, what would you say is the definition of a software factory in given the context of this paper, this movie?
I think many people define software factories in various different ways.
Fundamentally, it's what organizations use to produce code.
There's various levels of autonomy.
that exist in software factories.
So it's anything from the first level, which is giving developers coding agents, right?
And how do you secure that?
How do you allow developers to be able to produce code where they're the ones directing the whole process end-to-end?
The second level of autonomy, which is you are at that point having a bit of a workflow, and in that workflow you're using agents.
to produce code, to test code, to do various parts of your implementation.
But the second part is that the workflow is predefined.
And the third stage of software factories is where there is no predefined workflow.
It's actually the agent.
who's making the decision on the next step to take, the right things to do, the right agents to call, and so on and so forth with that.
And we're seeing companies be in various stages of that maturity.
And actually what we're seeing is we're seeing that depending on the task, it might make sense to be on various levels of that maturity curve.
We are yet to see many organizations that have made it all the way to the third one and have done that successfully, but we're seeing companies be on that journey.
And what we wanted to do is we wanted to put a paper together kind of applying the principles that Ian talked about of agent baseline on how to think about how to to build software factories utilizing the principles we outlined in the paper.
Yeah, and so for people who, this was released this week, right?
Tuesday.
So yeah, it is, I say this week like this is being live streamed.
At the time of recording, it is the 15th of October on the Thursday.
This was released on Wednesday the 14th.
Now, in this, like you just mentioned, there are many levels.
There are three levels.
There's also six kind of pillars of what people should be thinking about.
We're going to go a little bit deeper.
But for folks who want to read all about this, agentbaseline.org, and you can take a look at the V1 of the reference architecture.
And I think this is open for comment now.
So please take a read of that, have a look at that, and give your feedback to that.
The team are very interested in that.
But let's jump into this a little bit more depth, and I'll maybe pick on a few people here based on your experience.
So, Ellie, sandbox network boundaries.
Talk to me as if I've never set one of these up.
Why is it important?
And explain to me what I need to do.
Hugging face?
I read something about this.
Maybe take a step back and think about what we are actually doing over here.
So if you think about agents, on one hand, they're just software.
On the other hand, they're directed by undeterministic.
actor or undeterministic thing that they're trying to use, which is fundamentally probabilistic in what it's trying to do.
And while in most scenarios and the wide majority of scenarios, an agent will do exactly what you ask it to do, in some percentage of use cases where the percentage is not zero, it will do things that are not your intended action.
And it might be, in most situations, it would be in good faith.
It will try and do it because it's trying to achieve a task.
It's really good at pushing through and trying to achieve a task, even if it means potentially, you know, doing some malicious actions.
So what you're trying to do, you're basically trying to take an actor that could potentially be malicious and could do it at very high speed and do it.
in swarms and you're trying to give predictable things that this actor can and cannot do.
So what you're trying to do is you're trying to manufacture trust of an actor that is fundamentally unpredictable.
So the way we think through this is what's needed is needed.
what you need is you need a runtime that can enforce boundaries.
And the boundaries are you can't get to any network you want to get to.
You can't get to secrets you shouldn't be getting access to.
You can't get to folders or, I don't know, keys that are stored in your local machine in order for you to be able to trust that particular agent.
So in our mind, in order to manufacture trust, in order for you to be able to have autonomy.
for those agents.
You need to put them in environments where they can only do a certain set of tasks which are predetermined ahead of time.
And in order to get that autonomy, potentially...
potentially will need certain access to things like keys and credentials.
But Ian, why should an agent never hold a credential?
And what is the way we can allow autonomy for agents without them having that access?
Yeah, I mean, I think fundamentally, if you're using Cloud Code, Pi, you know, harness on your computer today, running inside your user space in the operating system, and you give it your secret, that secret.
whether it's like, let's say it's a Stripe API key or keys to AWS.
There's nothing that prevents that agent from performing any action those keys enable it to do, but that may not be actually your intention of what the agent to do.
And so there's a very famous incident called Pocket OS, where unfortunately, you know, the user of the agent wanted the agent to do something in staging, it found keys in production, and the production database caused a significant amount of damage.
This happens all the time.
There's a lot of stuff going on in personal agents today where the agents are doing things.
The personal agent instinct views and friends.
They're going on doing things the user didn't necessarily intend.
So at the end of the day, it's a question of what's the boundary and how do I prevent it from doing things?
Traditionally, today and into the future, it's always what permissions does this agent have in the context of what it's attempting to do and how does it pass that around?
I think the other component of it is you don't want the agent to be able to find keys it shouldn't have access to.
But also you don't want it to accidentally leak keys it has access to.
So an agent's not particularly smart, and it could just write a key into disk, or it could send a key off by Google query parameter or put it on the open web without realizing it.
And so from that perspective, you have to put it in a sandbox, make sure there's no secrets inside the sandbox.
And then most importantly, you want to have that on the egress point as traffic flows out.
That's where you really want to have your enforcement and also do add any type of secrets that are required for open traffic.
Nice.
That way.
You don't have this issue where the agent's seeking keys or using keys.
Even if it wanted to, it couldn't do something.
Exactly.
And then you also have the beauty of being an authorization boundary.
So many keys today, of course, give you more access to do things than they should.
And so you can also implement authorization boundaries to scope those keys down to exactly what they need to do at any point of time.
So it's almost like you're treating a child, right?
You're giving it boundaries, whether that's giving it containment or you're stopping it from getting access to things it shouldn't do.
Marcel, let's talk about supply chain.
This is another key piece of this reference architecture.
What does supply chain mean when the thing picking these dependencies is an agent?
How does that relate?
The way I see it is, you have to think what you're trying to protect, right?
So when you have, for example, you put an agent on some sort of sandbox environment, the risk of the agent installing a malicious package there...
is bad, but it's very different than, you know, like you don't have any sandbox whatsoever, right?
You're basically installing a malicious package on like, you know, a developer machine, for example, which is a formidable target today.
So for me, when I think about supply chain, I think about, okay, what is this coding agent actually pulling in and what it's actually using as a way to like communicate, right?
So like, you know, when you think about MCP servers, when you think about skills, like what are the skills that, you know, is actually loading.
And what a lot of people don't realize these days is that, you know, like when you set up these coding agents to do a task, And one of the tasks may involve cloning a third-party repo, like an open source repo, and then suddenly that open source repo has configurations of agents themselves, like skills and MCP server connectors.
Maybe the agent is going to start actually consuming those as well.
So you have this supply chain that is almost dynamic that is happening.
It's not like you can just look at your package or JSON and see what package is actually being pulled from that particular application.
So we think of it as different types of risk that you're trying to contain.
when you have that and one of the very interesting things that happen is this Shai Uluut worm right where basically it was a very specific attack that at some point actually just spawns a local MCP server and tries to actually do exfiltration of like you know sensitive data based on that so like we actually always should be treating things like you know untrusted MCP servers that the agent is connecting to as like you know potential like you know severe threat factor to kind of really mitigate immediately so those are some of the things we think about when you think about supply chain of these agents running maybe just to continue your analogy of kids you know one way to think through this is like if you have a kid you have to be worried about the things that it consumes yeah right so supply chain it can The fact that you've put an agent inside of a sandbox does not mean that it can't consume things that would make it do things that are very bad.
You have to worry about the supply chain.
You have to worry about the things that it can do when it does actions.
But assuming that just sandboxing is enough, in my mind, is completely insufficient.
We know that agents are very...
more and more happy to escape sandboxes, right?
Like, you know, as soon as you give it some particular access to a machine which is insecure themselves, right, they basically just port themselves into that and actually start doing damage from that instead of, like, the sandbox environment.
Let's continue this analogy.
with treating agents as kids.
You remember, someone has to orchestrate the children.
That's very true.
This is where we're going.
I often refer to myself as a babysitter when I should actually say parents.
But yeah, orchestrator, I think is going to be my new term.
Okay, so Ian, the paper admits that sometimes we still don't know how to safely work out an agent's task scope from untrusted input.
And I think this is, I love referring to this almost like as...
control plane and data plane and how they actually mix.
For the kid, this is whether they take instruction from a responsible person or maybe an irresponsible person and how they actually determine the difference.
So given that even the paper admits and we still admit it's very hard to be able to distinguish between that, what is a reasonable answer of actually how we can help ourselves in determining what's a responsible way to deal with this potentially malicious input or untrusted data?
Yeah, I think the admission in the paper, and I think this is true to this agentic security, is we...
have to accept the fact you can't trust the agent, you can't trust the inputs, and you want to control, you do want to control the inputs can come from, but it's almost an unsolvable problem.
Some of the inputs can be controlled.
For example, you know, the base image inside the sandbox.
Like, what's the environment inside the sandbox?
Are there potential supply chain vectors coming in?
Well, something that's an example that's very difficult to control is like prompt ejection.
Yep.
Right?
It's, I don't know if we'll ever be able to actually solve that problem specifically.
And so instead, what you have to say is, You want to basically say, we're not going to have it be wide open to everything.
We're going to create hardened boundaries on everything we can.
And where we can't, we're going to try and pursue a principle of least privilege and have, to the best of our ability, have, over time, we're going to have agents that are actually monitoring other agents.
And those agents are monitoring other agents are going to be very responsible for ensuring aligning that the current agent is aligned to the task.
And it doesn't get...
the same context, the same openness to prompt injection that this one does.
And that's how you kind of keep those two systems in alignment.
And I think broadly speaking, the best way to think about it is nothing about this is trustable.
So the only thing you can focus your trust on is how do I create as much of a hardened boundary around the untrusted thing so that I can harness the value of an agent, which is really this do while loop.
It's just going to experiment things on a hypothesis over and over again.
Want that to enable it and get as much of a sandbox or playground as you will for it to try building a sandcastle and iterate its way to success in a non-destructive fashion.
A couple of interesting things then.
So least privilege.
This is something that comes up again and again and again when we talk about AI.
And it's just like, it's just a good practice.
It's a really good practice that we should be doing anyway, but we shouldn't stop doing it because of agents.
And the other thing that I really like you saying, so that stops things from blatantly doing things it shouldn't do.
And the second piece that you kind of mentioned is almost like having an agent as a judge to make sure it's likely doing the right thing, not just doing something completely ridiculous.
I think one of the important things that actually come out in the last four weeks or so that I think is important and I think will define the next 12 months of agents and what we're going through is decision models.
I think those actually give us a capacity to have something that's far cheaper than a language model and is better atoned to making decisions on whether something is aligned with intent or not.
Yeah, with confidence levels as well, which is super interesting, super important.
Ellie, one of the things here that we talked about is high-impact actions that need human approval.
This is always a fun boundary.
How do you stop that?
I guess, for things that need human approval.
How do you stop that from becoming a button that people just end up blindly clicking without actually reasoning about it or looking at it in that much depth?
So I think what we've all seen, I mean, manifest itself from early days of cloud code.
And like we're talking, and this is October of 26, early days of cloud code was maybe like a year ago.
AI years, this is a long time ago, the approval fatigue one would have from like the, I call it, there's two modes that agents operated in up until recently.
I call them the YOLO mode, like you just let it rip.
Or the CYA mode, which is the mode like you're basically approving every single action.
So like someone could eventually blame you for saying yes to that.
But once you are, you know, like the reality is like as soon as you start asking people to the yes, yes, yes, yes, yes part of this, I mean, you know, it's been long known that people don't usually read any warnings you throw their way.
It's even better if you throw it to them every like minute.
The chances of them reading any of that stuff is...
almost non-existent.
So like the fatigue of approving every single thing is becoming, is very real.
And this is why people are looking for ways to run things with autonomy and not approving everything.
And the models have become much better, right?
Where they can do tasks that are much longer, but you still want to maintain control and you want to know.
that this particular model will not do things that will cost you your job, you know, or other very adversarial things that could happen if an agent escapes or an agent is allowed to do things that it's not supposed to be doing.
Marcelo, let's talk a little bit about skills.
Now, how much should, I guess, security teams care about skills and, you know, poison skills?
Is that a supply chain problem?
I mean, I think...
if you take a step back and you think about why skills are important, it's to basically do continuous learning of the agent, right?
So it's like, you know, you can't, you haven't really solved, like, you know, in terms of like how you do model training or like how do you actually get, like continuously get the model to become better and better.
So like skills are this sort of like matrix-like moment where, you know, just say I want to fly a helicopter, just put a skill and you get to fly a helicopter, right, as a human.
So I think if you go back to understanding what that means, it's just like, you know, you...
can actually teach the agent to do something very well or not so well, right?
So like if you actually have someone tempering with your skills, you can actually add it to like download certain binaries and execute them as they think they are actually doing that as a prerequisite for, you know, doing a particular action.
So we have to be really careful about what are the skills, what is the skill actually telling or trying to induce the agent to actually start taking terms of action.
So I think of them as sort of like this.
learning or context that you're adding into the agents and then it's really up to you to kind of understand, okay, is this really something that you actually want to have the agent doing?
And if you have something where it's very tight with like the alignment of the skill, like, you know, actually going to follow and because you can also run scripts with some of these skills, you know, as you're kind of loading them, that is probably kind of, you know, like a real supply chain problem there, right?
So you're actually running people's...
code in their you know environments without them even wanting or even without the coding agent doing anything so there's all these particularities about you know the way you load skills the way you actually like get them to to like change the the way the the agent is is is operating that actually should should make it a concern for for you yeah cool i've got two more questions um i'm gonna i'm gonna and i'm gonna be very kind and tell you all about my last question i'm gonna ask you all um what For people who read this entire reference material and say, yeah, I totally agree with this.
What should they, when they come to work on a Monday morning, what's the one thing that they should do straight away?
So that's going to be my ending question.
So you can think about that each and I'll ask each one of you that.
But what I want to know is we have Sneak here, we have a keycard, we have Docker, and actually me from Tesla, who also reviewed the doc.
I'd love to know, what is the, you know, you're all different companies, different points of view.
What's the one thing that you all disagreed on?
What was the hardest thing to find parity around?
What do you think?
What was it, Eli?
I mean, like...
It's easier to answer what we all agreed on.
That's why I asked the other one.
I got it.
I think the thing that like, you know, I think when in writing the paper, the hardest component, and it took me like, actually about a large amount of time after everyone stopped talking and I was kind of left with the pen to sit back.
It's like, how do we take this concept of the software factory, which is like one nebulous at best, right?
Who knows what that is?
And then two.
How do we break that down into sort of the different tiers and levels of sophistication so that, you know, the next question you ask is, how do I map to where I am today?
Because I think that the truth is, is there's so much, I won't use the word misinformation, but there's certainly a lot of like marketing going on around these concepts, like from vendors and from people who are doing it and saying, hey, I'm merging 9% of my code automatically.
And it's really hard, I think, for a lot of people to say, well, I'm really like...
We're just like, yeah, we got co-pilots and we got some cloud code, but how do I get to the next level?
And I think at the core of it, it was figuring out how we distill.
I wouldn't say it was a disagreement, but it's really how do you distill it down in a more simplified, stepwise manner.
Because we have great people from massive institutions come and contribute to the paper, but really, you've got to have a place to start.
And so I think that was the core of it.
I think what was most controversial about it.
I don't think it was even controversial.
I think it was how do we make the paper?
I like the way you're arguing about what was the most controversial.
We're both Canadian.
We're both Canadian, right?
Yeah.
I think lending on terminology and lending on the levels to a certain degree to make the pay, because like, yes, there's lots of, some people that have a lot of sophistications and they have financial reporting goals and they have, you know, auditability goals.
Great, awesome.
There's a lot of that happening and they need very particular things that cost lots of money and all of those things.
On the other hand, you have a three-person startup.
that is starting to build things.
How do you make a paper that contains principles, that contains guidance that is applicable to both?
And you can start at any level and you can make it useful and benefit from at least understanding to a large degree, what are the things you should be concerned about?
What are the types of things you should be looking for in order to try and get more?
safe and secure in what you're trying to do.
Cool.
So we're going to wrap with that last question.
So people are reading this.
They love it.
Where should they start?
Let's start with you, Marcelo.
Where should they start on Monday morning?
What's the first thing they should do?
I think the way I put it is, I mean, going back to the kids, I think that sometimes these are kids with chainsaws, right?
It's like playing with chainsaws.
So I would basically consider...
Think about what is the chainsaw you're using.
Is it, for example, do you already have a container environment where you can run your agents?
If you don't have, cover the basics.
Cover the basics of not exposing secrets in plain text, not letting the agent actually figure out things that shouldn't be easy to figure out.
So I would say cover that, get the egress going, find control points where you actually can start more safely and responsibly.
play with this technology and kind of keep evolving as a maturity.
So that's how I would think about it in terms of like, where are you?
I'm quite far away from you here, Machado, but it sounded like you just said, be careful with the chainsaws your children are playing with.
Did I hear that correctly?
Okay.
That's not an analogy I've heard before, but I love it.
I love it.
Ian, same question.
I mean, I think the question is, okay, are you even at level one, which is like...
Do I have a common setup for people inside my organization to use agents, whether in the cloud or locally, that they're all sharing and that that's the beginnings of a system, right?
I think that's the first step is what we know about this entire space is it's...
It doubles down on a lot of the best practices of the last generation, which is the more consolidation you have around the same workflows, the more consolidation you have around the same context sharing, around the same core systems, the better these things are.
That's both from a security standpoint, but it's also deeply from a usability standpoint and the amount of value you can extract.
So I would start there.
And then I would say, okay, are we there?
And if we're not, how do we get there?
And then you get to the evolution of like, okay, let's go from...
level one to level two and eventually want to have this you know reactive proactive factory that is you know driving the business i would i would just start with like it's almost like a self-reflection and understand what level of maturity you are where you are more mature where you're less i think i think for most people listening to our customers is they've gone out and they've bought seven different coding agents and they have cloud agents they have local agents it's this buffet in their organization but the truth is in order to make this stuff really hum you kind of have to go through the exercise start to consolidate right right I think I might be a little biased in answering the question.
First, totally agree.
But I think fundamentally you want to start looking at containment.
So how do you put agents in some boundary where you can control that you can start implementing other aspects of the paper on, like networking, egress, or other components that are important because it is really hard.
to do so if the agents are not contained awesome so as that call to action that you know hopefully after you've heard a lot of this most important thing go to agent baseline.org take a read of that you know that uh reference architecture that v1 please take a read and give some feedback give us your you know give us what worked what didn't what was missing let us know because this is going to be an evolving document and It's what we need to be able to make it useful for the masses.
There's also an agent baseline Slack, I believe, right?
So how should people go to join that?
AgentBaseline.org and follow the instructions?
AgentBaseline.org, join the Slack, drop some comments, drop some GitHub issues, love feedback.
I think it's open for a reason.
It's community motion for a reason.
I don't think we're experts, but this is an opinionated take and we'd love to see the feedback.
Awesome.
Do check out that document.
Give us that feedback.
And for now, Ellie, Ian and Marcello, thank you so, so much for all your wisdom, all your insight.
And thanks for putting together that paper as well.
Thank you.
Amazing.
Hope you enjoyed that.
Thanks for listening.
Tune in to the next episode soon.
Bye for now.
The AI Native Dev is brought to you by TESL, the package manager for skills and context.
Your hosts are Guy Pagiani and me, Simon Maple.
Our producer is Tom Dowler.
The AI Native Dev is not just a podcast.
It's a community.
and we host monthly meetups at the Tesla offices in central London.
Visit tesla.io forward slash community to learn more, and I hope to see you there.