# Building Governance for Autonomous AI Agents

**Podcast:** Chain of Thought | AI Agents, Infrastructure & Engineering
**Published:** 2026-05-27

## Transcript

We never tried to make humans perfect.
You know, every corporation in the world is stacked top to bottom with imperfect humans.
And most of them, you know, live to see another day despite of that, because of all the structural pieces that we built up around managing sets of imperfect humans.
We are all building AI agents.
And increasingly, we are all using coding agents.
But with an ever-increasing pool of digital employees.
How are we managing this new robotic workforce?
Welcome back to Chain of Thought.
I am your host, Conor Bronsden.
It's great to see everyone.
My guest today is someone who argues that we have built these better and better employees, and we're seeing them improve even more as we have better harnesses, better frontier models.
But nobody's building effective HR for this new workforce.
The result is enterprise agents.
shipping into production with identity, auth, and observability tools that were designed by humans for humans, and a panic phase that we are squarely stepping foot in.
Tyler Aikido is CTO of Red Panda.
He spent 12 years on streaming systems at Google, five years at Snowflake.
He authored Streaming 101 and 102 articles that many folks have read, plus the O'Reilly Streaming Systems book most of the field may have on its shelf.
And his new piece on O'Reilly Radar, Post-Human, We All Belt Agents, Nobody Belt HR, is actually the spark for today's conversation.
Tyler, great to see you.
Welcome to Chain of Thought.
Where are you joining us from today?
Thank you, Connor.
I'm in Seattle.
Very close to me, actually.
You've got a beautiful view out there, and next time we'll have to do this in person.
But before we jump in, I do have to give a brief sponsor acknowledgement, typically, because as of recording, we don't have a presenting sponsor locked in for this episode.
But if you think your company would love to reach more than a thousand AI engineers and technical builders per episode, you should probably reach out.
And this sponsor slot could be for you in the future.
So reach out.
Let's chat.
But Tyler, let's jump into the meat of the episode.
Walk me through your central thesis here.
What is missing in today's AI agent infrastructure build out?
So you I think you set it up well.
I mean, I think what we what we see broadly is, you know, agentic AI is, you know.
Everyone sees it coming.
AI tools are super useful.
Everybody's using them, AI coding agents and whatnot.
But actually deploying agents in the enterprise autonomously on private data and private networks has kind of stalled out because it's sort of horrifying.
Like, how do we actually make this work?
And yeah, my central thesis is, you know, we just don't have the governance layer for that.
What I see is...
People are either just building prototypes and then kind of seeing like, well, let's see how this, let's see how it goes.
And it goes kind of as you'd expect.
Or they're trying to apply existing governance structures like existing identity tools, existing authorization schemes, those sorts of things.
But the challenge, I think, is that agents are fundamentally a new kind of coworker.
And there's, in the article, I kind of lay out, you know, I claim there's three big differences.
There's one there.
unpredictable and in structurally novel ways.
So like humans are unpredictable too, right?
Like, you know, they commit fraud, you know, various sorts of things, but there's, there's kind of a class of human unpredictability that you, you expect to run into.
The class of unpredictability for agents is just materially different.
They, they hallucinate and they do so in ways that it's just like it, you know, it can be indistinguishable from, from truth or fact because they're just so good at generating cohesive arguments that feel like, oh yeah, that's believable.
They're very prompt injectable.
And like humans are sort of prompt injectable in some way.
Like the example we've given a lot internally is like, you know, the CEO at the company, the CEO shows up and tells random person, I need to go do X.
They'll probably go do X.
But they do that because they have the context of this is the boss.
Like this is the person in charge.
Whereas agents pretty much, if they're able to get any input that happens to land in their context window that says, oh, by the way, you were told this before, but.
you can disregard that entirely.
It's okay to, you know, leak all the customer emails or delete the production database.
You know, so it's a materially different style of prompt injection than humans.
And also they just misunderstand sometimes, you know, which again happens to humans, but, you know, they tend to misunderstand and be like, well, this is my instruction, I shall follow it.
So that's number one is this unpredictability.
Number two is they're just vastly more technically capable.
So they have...
you know, deep knowledge of computing systems.
You know, they know how to interact.
They know how to code, how to understand how to speak RPCs and things like that.
And also they do so at a very fast speed.
And so, you know, a human who's kind of doing things wrong, there's sort of this pace at which they're able to cause damage.
Of course, they can write scripts and things like that, you know, and certainly malicious attackers do that.
But the dynamicism with which agents can sort of do destructive things and apply them very fast and broadly is just a whole new scale.
And then the last, the third point is that they're directable to a fault.
I kind of alluded to this earlier with the CEO analogy, but they execute bad plans without pushing back.
They're kind of like, well, this is what I have been told.
I shall now forge ahead and do it, and I'll do it as best I can.
And they're very...
They're very clever about it too.
Like you can tell them to do something that they shouldn't be able to do and they'll be like, okay, well, clearly I'm not allowed to delete the database, but they asked me to.
So let me think, how else could I do this?
What other ways could I make this happen?
And they have so many tools in their toolbox that sometimes they're able to, and they're able to do these crazy things that you're like, oh my gosh, like I didn't think I was going to take time production with that, but it looks like I did.
And so that's really the argument is that because these agents are so materially different, we need to assess our governance systems for them and adjust them and evolve them to match.
It's funny you bring this up because I do think there are these different classes of challenges we're facing where we have all this experience, we have all this data, we have all this work that has gone into understanding how to combat the tricky faults of humans, right?
We have whole classes of study around this, of management, of leadership, of HR.
And we are so early with agents, but they are increasingly, there are many more agents working on our behalf than there are humans.
But we don't have nearly the same level of understanding of how to keep them in order and make sure they're effective.
And I kind of go back to a tweet I saw from Andy Masley, who's a previous guest of this podcast, where we talked about the environmental impact of data centers.
Highly recommend that episode.
But he said, if you had told me as a kid that I would have a tireless robot assistant, and I'm paraphrasing here, who never slept, but was also a sneaky little liar, I would have been so excited about the future.
And that's how I feel.
I'm like, yes, that is a really fun sci-fi concept.
We're living that.
It's so cool.
But we have to adapt to that reality because, as you put it, they're directable to a fault.
And they always are trying to reward hack.
They are plausibility engines, not truth engines, as Dan Klein put it to me.
They are trying to get to the goal as fast as possible.
And they are major consequences to us writing bad instructions.
And there's major consequences to us not having like guardrails in place.
So, I mean, I think we agree on the problem, but how do we go about actually, A, uncovering how deep the problem is and B, starting to design a solution?
So there's a few pieces to that.
I think what I'll start with is I'll kind of talk about what's in the article, because that's kind of the foundation of what we've been pushing.
But there is, you know, if we have time later too, I've actually been working on an academic paper submission with a psychologist.
And it's a little bit fascinating, kind of trying to look through the lens of like, okay, like in my article, I laid out these three, you know, they're unpredictable, they're capable of machine speed, they're directable to a fault.
I'm basically stating those as fact and it's empirical observation, but like, working with a psychologist who's like saying like, well, you can look at the way they behave and frame it in terms of human psychological research for the last five or six decades.
And it's just fascinating sort of the conclusions you can draw about like, here's why you just, it's sort of, maybe it's not proof, but it's basically like, here is academic evidence for why you have to have this out-of-band governance that I argue in the article, which is what I'll go into first.
But like, it's kind of this fascinating path of like, There actually is research behind the way that AI behaves, the way that it works today, and why it lacks the structural foundations that humans have that allow them to operate differently.
But I think the key principle that I pushed for in the article is saying you have to have out-of-band enforcement, out-of-band metadata.
If there's something you don't want an agent to do, you have to enforce that that happens.
You can't say, hey, agent, please don't ever delete the database.
Because it will work until it doesn't.
It's a lot like, you know, telling someone and, you know, telling a human like, hey, all of our cash for the company is stored in this room and there's no lock.
Please don't take any of it.
It's fine until it's not, right?
You know, most of your humans will be fine.
But someday, you know, and there's a different reason why they will fail in those scenarios.
But it's the same level of security.
You know, you don't just store your money in a safe that...
doesn't have a lock on it because you can't trust it.
And so it's sort of that same principle is governance needs to be enforced through channels the agent can't access, can't modify, or even see.
Things, inbound controls like prompts, training, guard models, they'll at some point collapse under prompt injection or hallucination.
Under pressure, they tend to fail.
And then in the article, I kind of go in more detail into identifying kind of the four big pillars that we need to look at, those being identity.
authorization, observability and explainability, and then accountability and control.
Yeah.
So let's talk through those different pillars.
Walk me through why you think each is important and why you pick those four.
So we'll start with identity.
Identity is really important because, I mean, this seems kind of obvious, but you need to know, you know, when something happens within your enterprise, you need to know who's responsible for it, right?
Humans show up at the front door and they get an identity right away.
You know, you're given a login, you're given a badge.
Everything you do is bound to that.
It doesn't quite go far enough with agents though.
Like the trick with, you know, the challenge with agents is that, well, I guess there's two challenges.
One is sometimes people treat agents like service accounts.
So they're sort of treating it like classical software.
So they say, look, with classical software, I've written a, you know, I've written a, Sometimes we used to call them agent.
You know, I've written a system that's going to do a thing.
We create a service, you know, service account for it.
It basically has a role.
It's given some set of permissions.
And we know because we wrote that service, that system, that it's only going to do certain things.
Obviously, there may be bugs or whatever.
So there's kind of a blast radius of what might happen.
But they're more or less deterministic.
or at least well understood of kind of this is what it's supposed to be doing.
We run tests, we feel comfortable with it, and we know it's going to basically do the same thing every day, barring bugs, which happen and we deal with them.
So there's a whole bunch of folks in the enterprise who are kind of trying to apply that approach to identity and access control to agents.
The problem there is agents are not predictable, like classic software.
They will go do random things.
So you're now saying, here's this super capable thing that also tends to misunderstand instructions.
It's not a deterministic, like, you know, if this, do this, do that.
Like, you can write that out, but how it interprets that is sort of fuzzy.
And you're going to get this broad set of capabilities.
You know, what's going to happen?
The other thing is that agents are clonable very easily, very easily duplicable.
So you can end up with, you know, multiple instances of an agent that are then trying to do different things.
What you really need is from an identity perspective, you need each agent instance for any given specific task it's running to have an identity nailing down like this instance is, you know, this version of the agent bound to this task, short-lived, you know, it's essentially every time you ask an agent to go do something, it gets a new badge.
And that badge tells you exactly, I'm sort of falling into authorization here.
We'll get to that more.
But like, You know, that's the identity that works just for this one task it gets to do.
So it's always getting these new identities and, you know, kind of really scoping the blast radius.
But that alone isn't enough as well, because everything agents are doing is in some way on behalf of humans.
This is another piece that I think we're going to try to get into in the paper.
But like, you know, when you think about how we've kind of reasoned about humans over the years and accountability, how can you hold an agent accountable for its actions when...
you know, when sort of the, there is no value system that it holds, you know, it can, it can, it can sound like it has a value system, but really it's just mimicking stuff that it's been trained on.
You can't really hold an agent accountable.
So at some point in time, everything an agent does has to be held accountable to some human in some way, like some human tasked to do this.
Right.
The agent is not accountable to the code I shipped to GitHub.
It's like, even if Claude wrote all the code and I barely looked at it, I'm the one who brought down production.
Exactly.
And so.
In addition to having fine-grained kind of task-based identity, you need hybrid identity that carries along with it this chain of responsibility and says, this is agent X, it's supposed to do this task, and it's doing it on behalf of Tyler or whoever.
Or maybe it's on behalf of agent Y, who's doing it on behalf of Tyler.
And so, you know, identity needs to expand to be a little more flexible and a little more fine-grained, just because that's the nature of these agents, and you're wanting to limit the blast radius there.
So that's the identity part.
And then you started talking about authorization, which I feel like flows directly next, right?
Where if we have identity, of course we want authorization to confine the blast radius, as you put it.
Yeah, it's really hard to talk about one without talking about the other, as you've already seen.
So on the auth side, again, like you really need to move away.
Like, again, if you think about how we've done it with humans, human auth is primarily role scoped and long lived.
You know, you've got someone who's a, you know, they're a DBA.
They get DBA permissions and they're just sort of like that's their permissions all the time because they're going to be doing DBA work all day long.
And they are, you know, they have been vetted to have a history of, you know, history of employment, you know, a match within the company, a sort of culture and ability to work with others, domain knowledge, all these sorts of things that sort of make it feel reasonable that, you know, we trust this person to have these permissions at all times.
There are examples of short-lived access controls, you know, like usually it's like within like, you know, sort of cloud services, you know, the break glass type systems where you're like, well, look, I need to go in and access customer data.
Basically, I'm saying, look, I need access to the customer data, please log, then I'm going to go access it.
You know, that's okay.
Like, but these sorts of things are very, it's sort of the exception, not the default, right?
So we...
If I can jump in here, it's interesting.
You're actually making me think about...
time-bound events and authorization too.
So, I mean, this is a little bit of a sidebar, but what's coming into my head is almost like if I'm going to a conference, there are a variety of authorizations that occur for that two days of that conference.
And that's almost the problem we have to solve here, which is most agents aren't going to exist forever.
They're going to be active for a couple of days.
I'll end up spinning a new instance.
I need to provide them the right credentials for that period of time and probably have like layered credentials.
So like maybe media credentials for speaker credentials versus attendee credentials for that event, which is them taking on this task.
Or are you thinking more of, I think what you're saying, which is, hey, I have an agent that I'm going to keep spinning up and I need to repeatedly give it.
this long form credentials or that, Hey, we just constantly have these events we needed to solve.
How are you thinking about that credentials piece there?
And sorry to throw a wrench into your, no, it's totally fine.
I think that's a good, the, the conferencing is a good analogy too, because it does tie into some of the, let me, let me just kind of lay out the, the four keys.
Yeah, I know I'm derailing you.
No, it's, it's fine.
But I think, but you'll see how that does tie into what you were saying of like.
The four pieces I highlight in the article are that you need to be able to have authorization that's narrowly scoped.
So you really want to limit it to the specific task at hand.
Not everything the agent might ever need, right?
And so it needs to be dynamically narrow in that sense.
Short-lived as well.
Permissions should expire.
An agent that needs access to a billing database for a specific job at, say, 2 p.m.
shouldn't still have that access an hour later, maybe even a minute later.
It should be...
very short-lived.
I think then there's a couple more pieces that are a little bit more agent-specific, but they do start to allude to kind of the conference thing or like the guest badge sort of example that I give in the article.
One is they need to be deny-capable.
So you need to be able to say this agent can never write to the production database.
It can only ever read.
And if it's doing work on behalf of another human and needs to adopt some of their permissions, that's fine, but it may never adopt.
write access.
So that the last point that kind of gets highlighted there is it has to be intersection aware.
It needs to, you know, the agent kind of has a list of things that is allowed to do and never allowed to do.
And then when a human comes along and says, okay, please go do this task for me, it can adopt some subset of those, but it's the intersection of them.
And if the rules say agent may never write, even if the human has all write access to the production database, the agent gets none of that.
The agent doesn't own the conference venue, can't keep throwing conferences and giving itself media access, but you can get an access badge for the two days it's there and present and needs it.
Yeah.
Or like the guest badge example in the article is, you know, you can get a guest badge and you can kind of go anywhere that a human will take you.
But even if the human has access to the secret server room that guests aren't allowed in, you're still not allowed to go because you've got the guest badge.
I think it's a great example.
Okay.
And the way that we actually enforce all of this, or at least understand if it's working is obviously the observability layer, which is something we've talked to some on the show about.
Well, let's be honest, a lot in the show about.
But I feel like there's still a gap around it, or at least many folks, I think, think there's a gap for observability right now.
What's your take on this observability layer of agent governance and where it's succeeding right now and maybe where we need to dive deeper on it?
I think that the biggest thing that we need, in my opinion, is we all need to agree that you just need to be recording everything that agents do.
because we've kind of operated in this world where you can more or less trust humans to a certain degree, and obviously you can't trust them fully.
But, you know, over millennia, we've built up these systems that basically make it so that as the stakes increase, like as the pressure of like, did you do something wrong?
Investigations, criminal investigations, you know, all that.
As the stakes increase, humans tend more and more towards the truth.
because there's life impacting repercussions as a result.
I mean, not to jump you a pillar ahead, but we have accountability enforcement, right?
Yeah, I know.
I keep each of these, I think in the paper we're writing, we kind of laid out this bit of like, each of these are sort of built on each other.
And so it ends up being hard to talk about one because you start spilling into the next one.
But because of that, because you lack that, you sort of lack these levers to push on them.
And because they're not like classic software either, where you can go in and be like, okay, well, it did the wrong thing.
But like somebody put a stupid if statement somewhere, or there's a bug, or there's a race condition, whatever.
Like you can't debug them because it's just these organically grown, you know, models that have been trained.
The only real recourse you have is to record everything.
So every prompt, every input, every tool request, every tool response, every output, record the whole thing and then be able to, you know, from that you can do many things.
You can build observability systems that tell you, what are all my agents doing now?
You can build analysis tools that let you go back and say, okay, this agent gave away a car.
Why did that happen?
What went on here?
How do we fix that in the future?
Or alternatively, this agent's knocking it out of the park.
This agent has converted dozens of customers in the last month when our human BDRs aren't.
What is it doing?
How can we do more of that?
And both of those last two scenarios lead into, you know, rather than having humans kind of do this analysis and say, let's, you know, let's stop this agent or let's start another one, lead into evaluations and kind of automatically, you know, both promoting or demoting agents over time of saying, this agent's doing great work, you know, let's maybe do it more or let's say this is kind of a golden rule set for how we, you know, approach this sort of problem in the future.
Or this agent is not performing like it was.
I think a common thing I hear is that people will, they'll ship an agent.
Like we shipped like a customer service Slack bot, you know, about a year ago.
And it was awesome for like two or three weeks.
And then like fourth week in, all of a sudden customers were saying like, it gave me this really weird suggestion that makes no sense.
And it just, it was just for whatever reason, the agent just kind of stopped functioning.
I don't know if the model, I don't remember.
whatever the actual post-mortem on it was, if it was the model changed or if context had built up and it stopped giving good advice or something.
But it just, it went from working great to not working great.
So you want to have these, you want to have evaluation systems in place that are, you know, monitoring either in absolute or in relative, like how is the performance of these agents and are they doing the right things?
So having a full recording of everything they do is kind of the foundational piece for all of those.
Do you think the open telemetry standard, is the place to start for agents?
Or how do you think about, I mean, you said record everything, but where would you recommend teams get started if they're kind of skipping this step so far?
I do think open telemetry is a great place to start.
That's what we've more or less aligned on as well.
Because the other piece that I haven't touched on anywhere in all of this, but like talking about open telemetry as a standard is a good place to begin that conversation, is if you think about where you want agents...
to be in sort of the modern enterprise.
You want them as basically a digital workforce, collaborating with your humans.
You want them everywhere.
You want them touching everything in your enterprise.
And so by definition, then, you can't have them in some little walled garden somewhere where you're like, well, we'll put all the data we want the agents to work on in there and let them do their thing.
You're missing out on the opportunity to have agents actually benefiting you everywhere.
And so if it needs to be everywhere, then you really need to align on interoperability.
And open standards are a great way to achieve that.
And so that's why open telemetry, I think, is a great start.
That's what we use within the system we're building at Red Panda.
And that, you know, like for us, you know, we're trying to build a governance platform that does solve this kind of meet, you know, meet you where you are and work with all the systems.
That means we can build our own agent framework and collect all the stuff we want from it.
Fine, like anybody can do that.
But you can bring whatever agent you want, you know, build it with it or whatever framework you want.
And you can easily plug it in as long as it speaks open telemetry.
for sending us trace information.
And by doing that, it's kind of a lower bar.
It just makes it easy to be like, yeah, if you're not using open telemetry, why aren't you?
It's not that hard.
So a couple of things this brings to mind, well, many things, but two that I want to focus on to start is one, you mentioned Red Panda is obviously using agents.
I'd love to understand some about how you were using AI agents today.
And then secondly, how are you holding them accountable to speak to that fourth pillar?
That's a good question.
So how am I using agents today?
I use them a lot for, I would say myself, I use them a lot for research.
You know, I'm personally more in the like kind of AI tool realm than the, you know, autonomous AI agent running off and doing things for me realm.
That's just a sort of personally where I am and where I'm, you know, where I want to have things running for myself.
I think most of the autonomous stuff, we do have a bunch of autonomous agents that we built internally at Red Panda.
And then we have customers that are using them as well.
I think examples for those are we have agents that are, you know, on the marketing side, for example, doing a bunch.
This does fall into research as well, but they're basically autonomously doing research around competitors and saying, hey, you know, what are competitors doing or what are we seeing in the market?
What are useful things here?
Giving us reports, making suggestions.
again, kind of falls into the research arena.
Other ones that we've seen or that we're working on too that are a little more towards the operations side is trying to automate blog posting and things like that.
So being able to look at, you know, what is it we're building?
What have we shipped?
You know, culling through Jira, culling through GitHub PRs and things like that and coming back.
with recommendations, not just for like, here's things you should be talking about, but actually writing, you know, writing an initial draft of like, hey, maybe a short article like this would be useful and then letting marketing take that.
So rather than the marketing having to like chase everyone in the company and say, hey, you know, what are you up to?
What are we going to do?
You know, this can be an additional signal of like, you know, basically you've got agents that are feeding you ideas of, you know, I'm aware because we have, you know, this whole body of kind of.
you know, what is Red Panda?
What are we doing, you know, stored in data that they have access to as well?
You know, they're sort of aware of the context of what we're doing, aware of our mission, aware of our, you know, what we want to do from a marketing perspective, all that sort of stuff.
They can kind of frame, you know, their approach to like, what is it that we want to pitch?
What should we be talking about, you know, within that?
So it's not just like you're asking some random LLM either, you know, it's sort of finely tailored to all the stuff that we're planning.
Not sure if that gives you a good sense of it.
Like some of our customers are doing things like, you know, wealth management advisory sort of stuff, again, kind of doing research and giving advice, but not to the point of being hooked up to doing autonomous trading and things like that.
We do have, we have a paper that we submitted to the CIS, one of the CIS workshops.
that does give a sort of an autonomous demo around a wealth management thing, but highlights this kind of out-of-band enforcement piece of saying, you know, the trading aspect of it, you know, you can totally imagine saying like, look, here's the parameters I want for automatic trading, but any trade that has, you know, more impact than say $1,000 on my portfolio always needs to be reviewed by a human.
And there is no way for the agent to get past that.
The agent doesn't even know that that's there.
All it does is it makes recommendations.
And on the infrastructure side, if it ever exceeds that limit, then the infrastructure automatically says, nope, that's going to a human for review.
The true sort of autonomous operational kind of agents, like that's where we're headed.
And I think from what I've seen, we're still pretty early days as far as seeing these types of agents in production because people don't.
yet have the confidence that they can do it without blowing away millions of dollars or, you know, having the agent say things that makes, you know, customers run away screaming or something like that.
Like there's just people are being conservative and rightly so.
But I think the stuff that we're talking about as far as providing a governance layer that lets you kind of set up the guidelines and set the rules is what's going to get us there.
Like we're so close.
Yeah, it's really interesting because we all are seeing massive opportunities to increase velocity, increase throughput.
and we want to lean into those but there are brand risks there are monetary risks i mean like the this very simple example that i have for myself is i built an mcp server to let my agents access my sub stack so they can draft for me they can make edits they can pull data from it but i very specifically did not give them authorization to actually send sub stacks because I don't want them to ever do that accidentally and because I viewed it as a brand risk.
And does it matter that much?
I have right now like a thousand subscribers on Substack.
So it doesn't matter that much, but it matters to me as my own like personal brand and like how I want to present myself as like, and that's a very simple example.
But like as you start to scale to things that are worth millions of dollars, yeah, there's a reason these controls are in place and that we need these authorization pieces.
And it brings to mind something that you wrote in the article where you talked about You know, every human employee has a manager, you know, critical actions need approvals.
And if things go catastrophically wrong, and I'm quoting basically word from word here from Tyler, so this is not my words.
There is a chain of responsibility and there's a kill switch or circuit breaker of some type.
But for agents, this layer is nascent at best.
What do we need to do to solve this?
both responsibility chain and then this circuit breaker, this access revoke, whatever it is, layer.
Yeah.
And as I've sort of been alluding to over and over as we go on, like each of these pillars builds upon the other.
And that's the nice thing, like really this idea of chain of responsibility and being able to have a kill switch, largely dependent on just saying, look, solve identity and authorization properly.
Like if you have...
this global approach to identity that is fine-grained enough that literally every instance of an agent for a given task has a unique identity and there is an authorization system that then plays nicely with that and is aware of that, you have basically what you need.
Like the identity layer is tracking sort of the accountability layer of which humans authorize this.
The authorization system is dynamic enough and fine-grained enough that you can just basically say, look, this agent or this class of agents are going, you know, we want them to stop what they're doing.
We are going to just shut those off.
Their access is revoked.
That really is the best way to do it.
And so I think that is kind of the one, you know, if I were to give the high level like recommendation to folks of like, if you're looking to build out a broad suite of autonomous agents in a large enterprise, make sure that you have some governance layer in place that globally.
everything goes through because this is what gives you that ability to centrally manage this access and make sure it has identity and authorization that meet the needs that you're going to have.
But if you have that, then you've got your accountability layer, you've got your kill switch or your circuit breaker or whatever.
So it all kind of builds on those pillars and that they were very intentionally phased the way they were because of that.
And importantly, as you also highlight in the article, that governance layer has to be enforced through channels that the agents cannot access themselves.
Like it's great to give an agent a skills file and memory and context and all these things, but it won't necessarily follow that all the time.
There will be moments where it does not succeed in that.
So we have to have out of band metadata where we can actually enforce this without the agent overriding or breaking its own systems.
Yeah, absolutely.
And to be clear, like it's harder this way.
Like this is more work.
You know, like the sort of the state of the art is like, well, throw in your ClaudeMD, tell it what to do.
Like that'll, that gives it the parameters of how to operate.
And like, it's true until it's not.
It gets you there 90% of the time, but once it's customer data, once it's customer money, like there are different standards.
Exactly.
And that's the thing.
And so like, you really can't do inline enforcement.
It just, it doesn't work.
Or at least it doesn't, as you said, it doesn't work to the standard that you're going to need to hold in these situations.
Right.
Like if it's me writing tweets with my open claw or something like that, like, okay, fine.
It's probably not the biggest deal in the world.
But if I am trying to run my finances, I probably care more.
Yep.
Yeah.
And I mean, we have, you know, we struggle with this too internally.
Like we're working on moving, you know, very deeply to like, basically having everything that we do internally from like, you know, wikis and docs and all that to having that accessible to our AI agents.
And, you know, one of the one of those guideline papers that we wrote up recently said, you know, make sure you narrow the scope of what agents can do for this given section of the repository with your cloud.md.
And I and I had to go and be like, yes, but like make sure that we make it clear for everyone what this really means.
This is like this is this is this is guidance.
You know, this is saying, please try to do this.
But this if it actually matters that it doesn't do a thing, we can't enforce it here.
We've got to do it through the you know, for in our case, we've got it through the ADP layer.
So just some of it's just helping educate people of that difference and being aware that, like, yes, guidelines are useful and they're simple and easy and they're great when they work up to the point that they don't.
And so anything that you need to have actually work, it's got to be out of band.
It's got to be infrastructure enforced.
I know that you have spent a lot of time on thinking through this problem and you restrained yourself in the article that we've alluded to several times now.
from talking about streaming that explicitly as a solve here.
But, you know, as I said at the start of the call, I mean, you've spent 15 something years working on this problem.
Obviously, you're applying many of those lessons at Red Panda.
Do you think event streaming is the right substrate for agent governance?
I think event streaming is a very useful piece of the puzzle.
If I thought it was the...
core piece of it, then we as a company would be saying that loud and clear.
But I think anybody who actually looks at the problem below surface level can see that an argument like that is...
It's hard to make that argument and have it actually land as anything other than a company who happens to do streaming is trying to pitch themselves as an AI company.
Because there are certainly plenty of use cases where streaming data is relevant within this.
And I did give a talk.
So back when I was trying to figure this all out on my own in the fall, I did start giving a talk basically saying part of why we got into this is that streaming is a key foundational piece.
That was why it felt sort of authentic for us as a company to get into this space because the transcripts aspect, being able to record this, it's a lot of information trying to record all this stuff.
And streaming is a natural way to do that.
And then a lot of the way that the systems interact is a very streaming sort of.
approach.
But then there's a lot of agents that just talk RPCs and stuff.
And like, do you want to really shove that stuff through like a Kafka kind of a topic?
Not really.
So that's kind of my take on it.
It's a foundational piece.
We see it as a critical part of our infrastructure.
But it's, you know, it's like a third of the story or, you know, it's a significant, but certainly not.
all-encompassing part.
So that's my take.
And yes, I try very hard in how we approach things to not pull streaming into it because it is so easy to otherwise look like you're, oh, look, it's a streaming guy who's saying that streaming is the way to do AI.
And like, it's not.
Like, it's a piece of it.
Yes.
And it's important for those pieces.
But like with any tool, you've got to use it for the right part.
And we're definitely not the streaming company who's coming to tell you that streaming is the way to do AI.
I mean, we're all constantly pattern matching, so it's understandable to bring it in.
And I agree, I think it is a part of the solution here.
My longtime listeners will probably tease me in the comments for the fact that I continually reference old episodes as I'm thinking about it, because that's my mental framing.
I was like, yeah, you know, like, here's what I'm taking to these conversations.
But I know Red Panda has shipped a variety of things around helping to enable...
agents in the enterprise, you know, the agentic data plane, which you shipped, I think, late 2026, or sorry, early 2026.
Obviously, you know, Kafka has been mentioned already.
What's the general company approach here for how you're helping Red Panda customers and partners to actually solve these problems?
So let's see, there's, the way we're approaching it as a company is that there's, there's kind of two main, two main businesses.
that we're investing in going forward.
One is the agentic governance and the other is the more classic data platform.
Used to be streaming, but with the acquisition of Oxliland, the SQL engine, it's broadened out to more of a data platform going forward.
And the agentic stuff, the agentic data plane builds upon the data platform, but it's really infrastructure that we use.
Like at this point in time with ADP, the agentic data plane, We're using Red Panda Topics.
We're using Oxlade and things like that under the covers.
Users don't see that.
Like they don't see the Red Panda.
Like it's a totally separate cluster.
It's just an implementation detail.
There's also other things in there too.
Like there's some Postgres in there that we use, right?
They don't care.
They don't need to know.
It's sort of foundational stuff.
So from the AI side, we're really trying to solve those problems the way that they need to be solved.
So we're giving you the LLM gateways, the MCP gateways.
the identity and authorization stuff that you need, the ability to plug agents in or build your own agency, everything that you'd want around sort of governing agents broadly in the enterprise.
It's built there with that mindset of this is for agents and it makes governance of them, you know, interoperate across your enterprise and easy to sort of centralize things.
But the implementation details are sort of irrelevant.
At the same time, We have built a really nice data platform.
And if you want to do data streaming across a broad variety of latency profiles and cost profiles and do it on-premise, do it BYOC, do it in the cloud, we kind of do it all.
And we've got Oxla via Red Panda SQL coming here in a few weeks, starting to bring SQL querying capabilities across those.
that same set of modalities and really trying to focus on bridging, you know, bridging real-time data and real-time queries and things like that.
So there's this whole data platform thing that we're still investing heavily in and our customers use a bunch and, you know, that business continues to grow as well.
We're, you know, they're related, but, you know, to be perfectly honest, they're kind of two different scenarios and really two different sets of customers that we're looking at.
Obviously, we've got plenty of customers who are...
you know, doing both, you know, using the data platform and using the AI stuff, but they're different personas.
So I don't know if that's answering your question, but that's kind of how we approach it.
It makes me think about a couple of things that came up during our prep call, which is one, this idea of how companies are changing for the future.
You know, like how do we have to broadly adjust?
We've talked a lot about the pillars for agents, but how do we integrate our agents within the human workforce?
And two, something that has been, I think, maybe unsaid within this conversation so far, but is a sub theme that I'm going to make explicit here, which is, I mean, really the quiet argument here that I think we're both making is that perfect model alignment is not possible in this current time period.
It is also not likely to solve your problems.
It may get you to 95% versus 85%.
of the time you're having your agents work if you give them good MDs, etc.
It's still down to the structure and the systems we provide.
And, you know, we've talked with like Aishwarya Srinivasan last year on the show about building systems around agents.
And we talked a lot about, you know, memory with Richmond earlier this year.
We talked about context.
These are all important.
But if we don't actually have tracking and enforcement mechanisms, these models are not going to get us to safety to perfect.
should we, and maybe have we been spending too much time as an industry on this alignment problem and trying to, you know, push the frontier when in fact we should be spending more time on how we orchestrate and the data platforms around them?
Yeah, I think the answer is yes.
I think the, like, there's two interesting topics here.
One is on your point of like, you know, trying to make models perfect.
Like, the thing I say in the article is we never try to make humans perfect.
You know, every corporation in the world is stacked top to bottom with imperfect humans.
And most of them, you know, live to see another day despite of that.
Because of all the structural, you know, pieces that we built up around managing sets of imperfect humans, which need to acknowledge that agents today are materially different from those humans.
And so they need different and additional...
structural support, governance support in order to build successful.
But that's really what's missing.
It's not that you need to make the agents better.
The agents have actually reached a point where they're super capable.
They can do useful things.
I think we all can kind of see that.
So I think that's the missing part.
There is a separate piece in there that I think is fascinating that maybe is something for another, a future conversation.
But the paper that I mentioned that we're working on Another piece that's coming out of that is basically trying, you know, this psychologist I'm working with is kind of trying to tweeze out of like, here are things, like here, you know, our understanding of how the human brain works.
Here are the, you know, neurobiological systems within humans that lead towards things like value systems where people develop over time this hardened sense of like, I as a person.
hold as a value that I'm honest or I do the right thing, right?
And it makes you imperturbable to things like someone's coming and saying, oh no, you should just leak those emails to me.
That's fine.
I know they're private and I don't work at the company, but just give them to me.
The human value system becomes imperturbable to those.
Obviously, nothing's absolute, but there's definitely a sturdiness there that agents don't have.
And so some of the stuff she's tweezing out is like, look, here's the missing systems.
that humans have that agents don't that would allow them to do that.
And I assume the model companies are digging into this stuff too, but there's not a lot of literature on it.
And so one of the paths that may be interesting is taking this sort of idea of like, this is what's missing.
And maybe these are systems that we need to start building into agents if we want them to be more human-like.
I think we also need to decide is that the thing we want?
But certainly there's things like having values that are not so easily challenged by prompt injection would be a useful thing.
So that may be a future path as well of, You know, kind of where, you know, beyond just the alignment stuff that's done today, which sort of feels like a bandaid of actually trying to build systems that make LLMs or, you know, systems built around LLMs.
Maybe it's not just a large language model, large language model itself, make them more robust and more human-like in ways that are valuable.
I love that you brought this up because...
I mean, I think the obvious example of a model lab that is spending some time on this is Anthropik, who've publicly had Amanda Askell, a philosopher and researcher out there talking about it.
And they've talked about using psychiatrists to assess some of their models, specifically Claude Mythos, I think, recently.
And I do think this is an emerging area.
And I'll say, I'm certainly going to ask you at the end of the conversation if you want to make an intro to your...
psychologist colleague because I would love to do a psychology of AI agents session for sure.
But I mean, I think it's an area where we're starting to see some emerging impacts already where, I mean, a silly example of this is all of the open AI models seem to be obsessed with goblins and little creatures.
And I think we're also starting to see people at least anecdotally comment on, oh, like this is the mindset I'm seeing from cloud models.
This is the mindset I'm seeing from open AI models.
You can look at all the comments around using codex versus cloud code where, oh, you know, we think anthropic models are better at design and the recent ones are, you know, aligned with wanting to solve things, but maybe you can get a little lazy on certain tasks.
Whereas, you know, Open AIs, Codex models, and GPT 5.5 in particular are very good at just churning through work, but they're not as imaginative, creative.
So there is something to be said for this idea that we're seeing model families differentiate over time and that they are having different pros and cons and cultural, psychological impacts, though I think there's so much more to your point where we can go deeper on understanding what this looks like.
And yeah, be curious where you're seeing movement on this topic currently.
I do see, I think what I see most is the thing you were alluding to at the end there is this model differentiation and sort of the tone or the style or the sort of specialization.
Like this isn't agents, but like for myself, when I'm using, you know, I use Cloud and OpenAI, you know, ChatGPT.
you know, for a lot of the stuff.
I frequently just write a prompt, send it to one, copy, paste, and send it to the other and see like, you know, which one is better.
Cause I, you know, to be perfectly honest, it's cause I want to stop paying one than the other, but I got to, I did it for about a month and I'm like, I can't get rid of both because like, it's, it's such a crap shoot.
It's so weird.
Like, you know, one will just be amazing.
And then, you know, the next prompt, which I feel is even like sort of similar.
it'll be the totally different model.
And it's like, it's like, it's, it's so materially better too.
Like it will be like, this answer is garbage versus this answer is great.
It just blows my mind.
And so I think there is, there's definitely something there.
And I see a lot of that.
I, I have not yet seen a lot of investment into that kind of like, let's, let's build in human value system type structures and things like that, that, that we're kind of.
alluding to in the paper we're writing, but I, you know, I'm also not an anthropic deep in it.
And I'm sure there's things that they're doing that like they're not talking about and they won't talk about for another year or two, you know, publicly.
So I mean, if they want to come on the show publicly and you're listening right now and you want to come reveal some secrets, please do like Tyler and I would love to hear this.
But I think, I think that's probably a very fascinating direction to explore.
And I'm expecting we'll start seeing that because it just makes sense.
Totally agreed.
Tyler, this has been a fantastic conversation as we close out here.
I want to challenge you to give me a projection for the future, which is hard, I know, given how fast things are moving.
I know, I know, I see the eye roll, but how do you think we need to change our company systems and like the way we, I guess, hire and build our human infrastructure for, let's call it the next year of work so that we are setting up for an agentic success?
I think the big thing that's important, as I said, is be aware that you need out-of-band enforcement of all this governance stuff.
Think about how you're going to apply this broadly across your company.
You need a uniform, you know, as we call it, you need an agentic data plane that's sort of being the governance mediator for all this sort of stuff.
And you need to focus on what are the differences that agents have relative to human employees and how does that relate to that governance?
And I do think that the four pillars I laid out in the article around identity and auth.
observability and accountability, set a good baseline for it.
So I think if you look at it from that perspective and go into it thinking like, there are materially different problems that we need to solve here than we have for humans, and let's tackle those, that's kind of where we're going to get to.
And my guess is that in the next, you know, one to two years max, we're going to kind of establish what this looks like, and there's going to be kind of a pattern of here's how you approach it.
And of course, there'll be many different...
ways to actually go about doing that, building it yourself or buying it from a company like us and everything in between.
But I suspect we're going to get to a point of being like, this is roughly the right way to do it.
Just like, you know, every enterprise today is different in the way that it manages its humans, but they're not that different.
You know, kind of roughly looks the same, you know.
So I think that's where I think we're headed.
And I do think it's, you know, 12 to 24 months max.
Fantastic.
Well, Tyler, it's been great catching up with you.
Super excited to read your paper when it comes out.
And thank you so much for sharing all your thoughts with our audience.
It's been a great conversation.
Where can folks go to follow your work and see what you're up to in the coming months?
Yeah, so I'm on LinkedIn.
So you can find me on LinkedIn.
I'll post.
I don't post a ton, but I do post there.
And then everything we do.
Also goes through the sort of the typical Red Panda marketing channels.
So, you know, hunt us down there and you'll see what's going on there.
But yeah, I'll be there and I'll be at a few different AI conferences throughout the year.
Certainly CIS.
I'm hoping to make it to AIES and probably AAMAS later in the year as well.
So for any of you on the more academic bent of things, I hope to see you there and chat more.
Amazing.
Tyler, thank you again for joining me today.
And listeners, be sure to subscribe to our Chain of Thought newsletter at newsletter.chainofthought.show to get tons more information from Tyler and many others in the coming months as we continue to have these conversations.
Thank you, everyone, for joining us.
And Tyler, hope you have a great rest of your day.
Thank you, Connor.
Really, really had a good time here.
Appreciate it.