# AI Agent Strategy For Secure Software Operations

**Podcast:** The InfoQ Podcast
**Published:** 2026-08-17

## Transcript

Welcome to the Architects Podcast, where we discuss what it means to be an architect and how architects actually do their job.
Today's guest is someone who we have had on the podcast before, and we had a wonderful time.
So we invited her back and she was gracious enough to accept.
Tracy Bannon is a senior principal software architect and DevOps advisor senior at MITRE.
The Real Technologist Podcast.
Welcome back.
Oh my goodness, that's a lot of words to say.
Oh, she likes tech.
That's what it really amounts to.
So I've added another comma, but it's an experience comma, not a credential.
It's that I have been the primary investigator for our corporation for how you apply AI to the software development lifecycle.
So imagine how that knits in nicely when we talk about architecture.
oh my goodness, so fun, so cool.
It's been awesome to do that.
Well, welcome back.
Last time we were here, we talked about many things concerning software architecture, which was how software architecture does not magically appear.
It has to be done intentionally.
And in another area, I fear we're making this type of mistake again.
If you remember how the World Wide Web got started, and the World Wide Web, of course, is not the internet, but it was the World Wide Web.
We took a technology that was not designed for transactional, stateful actions or sensitive information and use it for things that were transactional and stateful and had sensitive information such as e-commerce.
I don't think people realize how much a hack secure websites are with the SSL.
Are we doing the same thing again with agents, only worse?
that they will have access to our email, our credit cards, our personal information, and we empower it to do things for us.
This seems to be like a paradise for a person with evil intent.
It is.
And the one thing that's different is that we are more in control if we want to be in control.
You can choose whether or not you use agents.
You can choose that.
Especially if I think about agents in the browser, that's another, we'll put a pin in that and come back to the wonderful security issues that are there.
But it's your choice.
When you think about connecting things together, would you give all of your entire key chain to somebody or would you give them a single key?
Right.
Well, you're going to borrow my car.
I don't give you the key to the house, the key to the condo, the key to the bank, the key to the safety deposit box.
I give you the key to the car.
People need to be.
thinking about those in the same way because the exposure, the attack surface is immense, especially on a personal level, on a very, very, very personal level.
The corporations, depending on their size and their focus, are doing a pretty good job of constraining that right now.
but not so much the folks that I work with myself as well.
I've got my team of AI agents and I've got to have them sandboxed and I'm not connecting them to my email.
But I know a lot of folks who do, especially really bright, small business owners.
And now their email and their calendar and other integrations to their Trello board or their Notion board, they might be connecting it into their CRM.
feels glorious because now I don't need to pay for those humans.
I can do this for pennies.
And they've just opened up a potential world of hurt because they're not thinking about the problems that they could be having.
Heck, you really want to scare folks?
And I love this tool.
I'm a big fan of Claude, big fan of Claude and use all of the different flavors of it.
I have the desktop apps on my phone, taught my husband.
He's now an addict.
But I also have the add-in to the browser.
Here's the danger with the add-in to the browser.
I log into a website.
I just did this the other day intentionally.
I logged into my Hilton Monarch account.
I needed to look some things up.
I didn't want to take the time to look it up myself.
So I logged in.
Then I clicked on the browser extension.
and said, I need you to do these things.
And I could authorize it to do it while I stepped away.
I need you to think about that.
I've just logged in, used secure authentication, secure login.
And I've said, go ahead.
I've got my own JIRA board that I have for something.
I was working on something.
I wanted to test it out.
I was doing the normal interaction with a chat style.
And then I thought, you know, I wonder if I could turn this loose and have it solve my problem.
That just had to do with configuring something that was within a browser-based solution.
Log in, say go ahead, and it acts on its own.
And it is amazing.
And I know that every time I touch that off, I'm terrified.
Because it's got the keys to the kingdom.
And I said, okay.
So you are a sophisticated user.
I would say so, yeah.
But what about, for example, I was at Google Next last week.
And they have now this Google business, which accesses the email, your SharePoint.
And they now have agents that you can create by diagramming.
You don't have to code.
And you're sort of trusting, I guess, Google trusting yourself.
The trust is not explicit here to me.
And it's not clear.
We know where the...
problems are.
And of course, lurking in the background of all this is this mythos that Anthropic has released, which whether it's hype or not, it feels that one of these days we're going to be in that situation.
So we'll come back to Anthropic.
But let's talk about the Google Play, no pun intended, the Google Play and all of this.
Where the hyperscalers like Google, Microsoft are banking right now is they already have your trust, even though you may not trust them.
I have my own personal Gmail.
So I have Drive, right?
And I have multiple different types of documents there.
I've got a calendar there.
I've got email there.
I'm already there and they can already see it.
And I can tell you that maybe a couple of years ago, there was some LinkedIn post and I may have gone out there and turned off some access, but in general.
I don't go check the EULA every couple of weeks to see if they've started to train on my stuff.
So they already have it.
So in terms of access, the access is already there.
The question is, am I creating a workflow to try and help myself smartly or not?
Because if you're in their ecosystem, chances are within their ecosystem, there's a little bit more safety than if you're going across ecosystems.
That's where I think the real risk is.
But if you're within their ecosystem, not bad.
You can go from Google Next to Outlook.
You know, in that business environment, you can use the Outlook and use the SharePoint.
That's when you start to go across boundaries, in my opinion, where it really gets dicey fast.
Like that example that I gave you where I used my browser.
And a normal browser at that point, you're just HTTP.
You're not even HTTPS.
You're just going to whatever your landing page is.
And as soon as you have crossed that boundary and logging into something, wow, you've created a security gap.
So if you're creating workflows, it's no different than it used to for big system implementations.
I have to draw.
and I have to draw the entire information flow and where all the boundaries are, and I have to know what's called ingress and egress.
I have to know when I'm going across the boundaries and who owns each one of those pieces of the boundaries.
Who in their right mind is going to do that at home?
Or who in their right mind is going to do that for a small business?
Nobody's doing that.
Nobody's saying, well, this is actually sitting over here in 0365, and then that crosses over this tenant, and that's now going over here, and people are not thinking about that.
So I really will caution to say within a single ecosystem, there is likely to be a higher degree of security than when you're crossing a boundary.
Now, we all know that the excitement is real because it is accessible.
It is accessible.
I've taught my brother and his wife.
He is a building architect.
She's an interior designer, highly competent with technology.
And I've been teaching them over the last year.
Here's some more things that you can do.
And some of the most recent things that she has put into place kind of made me take pause.
Like, do you really want to do that?
Do you really want to connect those pieces together?
Luckily, she's very, very open to feedback.
And pause, pause.
These are folks who do not have their mail on cloud.
They have their mail on premises.
Can you imagine that?
Some people still do that.
So there's some goodness there.
But I think that that's a piece of this story.
Accessibility to groundbreaking technology that is hyped as being able to help us with our toil.
And people do see fast results on simple things.
I was able to get my husband addicted.
Now, he'd been using chat GPT for a while, but I showed him how to use Claude.
I shouldn't have done this.
I showed him how to use Claude.
And within two days, the amount of things that he has.
He's an ops guy, by the way.
So he's security minded.
But just the ability to automate, the ability to automate.
It's fantastic.
Now you talk about accessibility.
They want to bring AI into cars.
Talk about crossing boundaries.
Hasn't it already been there?
Not agent technology in cars.
But I meant.
Generative AI versus AI has been a part of algorithms, machine learning algorithms, different type of mathematical algorithms, different types of AI have been a part.
It's going to be the depth, right?
The depth of the algorithms, the complexity.
These cars are already massively complex.
It'll just sprinkle some AI on it.
Well, I mean, I think, for example, GM, I think, has chosen Gemini to bring into the car.
Well, when you bring it into the car, the question would be, are you bringing it in to make it accessible or are you leveraging it for the operation of the automobile?
I think both.
The second one would be more of a concern, especially if we're talking about what I call mission critical, those things that save your life.
Don't be tying me to somebody outsourced language model if you're trying to save my life.
Thank you very much.
But for usability, I think I've told you my stories that I go to Washington, D.C.
Tuesdays and Wednesdays of every week.
I go down on Tuesday and I stay overnight and come back on Wednesday night.
And I talk to AI on the drive down.
I don't on the way back because by the time I'm done at the Pentagon, my brain is mush.
I need to decompress, but on the way there.
thinking of new ideas and contemplating things I want my teams to do or just exploring different things.
And I'm talking to AI.
It's connected to my Bluetooth through my phone.
I'm talking to it.
Now, if I could say to it, and I haven't tried this, maybe I'll try this tonight.
Got to do some shopping.
Have it make an order for me of some sort or have it do a transaction on my behalf.
In general, I have not had my agents take transactions, financial transactions on my behalf.
I have not done that at all.
But that might be kind of cool to see if I could have it do something for me while I was driving.
Maybe order my pizza.
I just verbally ordered the pizza and it shows up.
And it's already been paid for.
I don't know if you remember the movie WALL-E.
Yes.
Where all the people were just stayed in their in their chairs.
And everything was delivered to them.
That's actually part of our family's lexicon is when we talk about WALL-E, we talk about being in your WALL-E chair.
And there was recently, I think Toyota recently made a really nifty wheelchair or mobility scooter.
Looks like the Wally chair was the problem.
As soon as I saw it, I'm like, nope, I don't care how good it is.
Nope, not doing it.
There's so much to talk about with agents, with Agentix.
I want to go back to the tools themselves.
Now, the browser piece, it seems easy.
A lot of people are not using those extensions on the browser.
What they're doing is they're going to a website.
They're going to a chat interface, right?
And that's how they're using it.
And at first we were hearing, oh, you're going to have agents in your browser, agents in your browser.
It's the downloadable applications that I think should start to scare people a little bit more because it looks like the chat interface, but you click another tab and then you say, okay, we're going to co-work.
May I have access to your file system?
Well, sure, if you're going to help me.
Hey, can you crawl my local server?
Can you crawl my server and index all my files for me?
Those are real things that people are using these tools for right now.
Things that they couldn't get done themselves.
Oh, hell, I got 10 years, 15 years worth of Google Docs that I'm this shy of turning an agent loose on.
But that's that's cloud based.
But imagine people are doing that on their local machines where you more likely have very private things in your home.
Yeah, go ahead and open up that app because it is crazy, sexy, fast and cool.
My husband made a PowerPoint and he was just so excited.
He showed it to me.
It was choosing between two dinner locations and all he did was ask two questions.
It wasn't even a real prompt.
It was, I'm thinking about dinner.
These are the kind of foods that I want.
I need something that's within walking distance of my current location.
Can you give it to me as a PowerPoint?
And he sends me a link to the PowerPoint.
Where do you want to eat dinner?
Like, no, stop, stop.
That's tokens.
We're killing the environment.
You're the reason for the data center.
That's the funny thing about tokens.
I was talking to somebody the other day and they said, you know.
Maybe we will still hire junior engineers because they're cheaper than all the tokens we need to do their job.
Although we're doing some smart stuff to reduce token count.
There are some really smart patterns in the enterprise where you're not just interacting with a single agent, but you have an orchestrator agent.
Think of it like having a supervisor or a construction foreman and then all of the experts.
And so that orchestrator.
knows to turn around and ask the other experts, and it gives it a consolidated, condensed piece.
This is what you need to know.
You don't need to know everything about masonry.
You just need to know how to ask this question about this kind of mortar or this kind of grout.
So there's some really smart techniques to reduce token count.
But I don't know if you saw recently, there was a mention that an update to one of the models, and it escapes me right now.
But the update that they made was actually causing people to max out on tokens within like two to three hours as opposed to that 24 hours because it had done something in the thunking layers that caused that problem.
So there you go.
Tokens are the new currency.
No, they're not.
Yes.
Well, that's still where the human brain seems to win because we can both do the language thing and we can also do.
You know, they now have these analytic models that are not based on transformers, but are based on mathematics and machine learning and trying to make advances in material science and biology.
And we seem to do pretty well in terms of power to volume.
The concern that I brought up in a meeting earlier today was about cognitive overload for the humans right now.
As we are talking about, especially in software engineering, as we're talking about how amazing it can be, it can help us to generate our requirements, it can generate a code in our tests, and it can help us to analyze things.
The problem that we're now seeing is cognitive overload, not because it's helping us, but because we have become the reviewers and not the authors, and the volume of materials for us to review.
Because we're still humans in the loop.
We're not at the point where we can truly pull ourselves out.
So we're at a point right now where it's kind of crushing.
And that's whether or not you are an individual doing this at home or if you're doing this in a corporate setting or if you're doing this in a government setting.
I did this to myself recently.
There's something that I wanted to work on.
I wanted to put together and I went down this rabbit hole and the volume of stuff that I ended up generating.
I finally looked at the clock and thought.
holy hell, it's eight o'clock in there.
What the heck are you doing?
The amount of things that it had generated, I was using a combination of enterprise version of perplexity and research mode, and I was having it go out and find academic things that were matching to industry.
And it was volumes and volumes and volumes of amazingness.
There's no way for me to get through it.
It was more than I could manage to get through, and I didn't want to hand it off to AI.
to actually analyze the research that I had had the AI pull back.
So it can become a really vicious, strange cycle, to be honest.
So because the first couple of times you invoke something, you get something magical back.
I don't care what it is.
You had it make a book of chicken jokes.
I don't care what it is.
The first time that you do it, the length is awesome, right?
That's just great.
It's just awesome.
But then after that is...
When you continue to use it and you're asking it for bigger and more, that's where more of that cognitive overload is happening.
That's your Sorcerer's Apprentice right there.
It is one of the pieces of the Sorcerer's Apprentice.
I think another piece of that story, you know, thinking back to Fantasia and the Sorcerer with his hat and he puts it down and he walks out and Mickey Mouse in this old Fantasia movie.
He's so excited by the idea that puts that hat on his head and he's going to make some magic.
All he wanted to do was have a little less toil, fill up the water cistern, animates the broom, falls asleep, wakes up, realizes the broom has not stopped what it's supposed to do.
It's now flooding.
You know, you can tell this from the 40s because he takes an axe and tries to chop the broom into pieces.
We wouldn't do that today.
But every splinter became another broom.
So all of a sudden there was this proliferation of...
Not just content, not just the water, but more brooms and more brooms and more brooms.
So agents begetting agents.
I can use an agent to create an agent.
I can use agents to create legions of agents for me.
So it's not just the cognitive overload.
There's a dramatic amount of effort.
There's a dramatic amount of outcomes that people aren't able to even gauge.
Well, how are we then going to?
test these systems or even describe our requirement.
I mean, one of the things that humans are very good at and LLMs are not very good at, at least in my experience, is dealing with ambiguity.
And, you know, when we express requirements as these systems, you know, I say I want it to run fast.
What does fast mean?
You know, compared to what?
Compared to who's experienced fast on the back end, fast on the front end.
I mean, I presume you remember the days when we were all going to have requirements development languages where we're going to express things precisely and give them to the case tools, which you're going to then generate all this.
We've been here before.
Well, we're back at it now.
We're back at it.
But now we use natural language, which is much more precisely imprecise.
Yes.
So, yeah, there's there's a lot to dig into on this.
So, yes.
In the design of an agent, in the formal design of an agent, I don't mean ad hoc, at home, WYSIWYG, in the formal design of an agent, there's a lot of growth in this area right now.
And we're using specifications.
They're lightweight, but they're specifications that actually tell us what it is supposed to do and not to do, right?
So both the boundaries of how it should act and when it should act, lineage.
That's especially important because people hack from other people all the time.
Mike found this out on the Internet, brought it into the corporation, posted it into some area.
I want to know where you got it.
That's supply chain, right?
And supply chain safety.
We're looking at the specifications so that we can do a better job with automated testing of agents.
People don't think necessarily that agents can be tested.
There are different types of agents that do different types of things.
But ultimately, if you task it.
with something larger and it has to create a plan and you evaluate that plan, the first couple of times around are always humans.
Now we're starting to use AI to help us judge the other AI if it came up with a good plan.
But we are going back to specifications.
We are definitely there so that we can test these things.
The repeatability of the tests are something that I'm very interested in now.
That's great that I have an automated test.
If I run that five times, what am I actually going to get?
Remember that when you're dealing with a conversation, in the same conversation with a language model and the same token buildup, it actually starts to get tunnel vision.
That's why I call it tunnel vision.
What's going to happen if I run that test five times?
When we used to run a test five times between each test, I would doge.
Yes, I use that term now.
I would doge the database and reset the database so that every time it was fresh and I didn't have any damage or any kind of pollution.
How do you do that now?
What the hell do I do?
Should I doge my token count?
I don't know.
Because that's not real.
Sometimes you're going to be doing it from fresh and you're going to get a more creative answer.
And sometimes you're going to be running a skill or running an agent and it's getting bogged down and it's getting very narrow minded.
And I think we're going to run into some interesting things there as well.
So yes, we're going to have specifications, but how the hell do you test it?
And how do you test it reasonably to what's going to happen with individuals?
Well, especially when there's a certain amount of randomness in what the agent is producing.
So I'm not quite sure what you test a range of values, you test the reasonableness of value.
It depends on what you are designing the agent to do for you.
So let's have a hypothetical.
I have my requirements language that I use to both generate tests and generate the agent.
And the agent has been running for a while now.
Now, we get bugs from the field, you know, things that are not right.
And I'm not necessarily, I don't think the critical bugs were giving away state secrets, but just the normal average everyday run of the mill problems.
How do you feed that back to the agent?
Do you have to redesign the agent?
Do you make deltas?
Can the agent revise itself?
How do you deal with that?
So the answer is it depends on what the actual life cycle that we're talking about looks like.
An agent is built on top of tools and the tool may or may not include a language model.
I can have agents.
We've had agents for dozens of years.
Good heavens.
Agents invoke, right?
But they also make plans and execute plans.
And sometimes they can pick the tool that they want.
You don't train an agent in the same way that you train a language model.
You put additional scaffolding to the extent possible in the instantiation of the agent because agents aren't permanent.
You spin an agent up.
You have it for an execution.
Ultimately, if it is working with a single language model, it will run out of tokens so that there's a newness to each execution of an agent or each session with an agent.
So let's say that I leverage AI.
I have agents that I'm using.
Let's say I'm using Claude.
I have Claude code established within my IDE.
I'm using visual code.
I also go ahead and tell it to push to kick off the build, go ahead and trigger the deploy.
Now it's running in production.
Now I have something else, presumably not Claude code from within my IDE, but I have something else that's running out there that's collecting telemetry.
It is collecting other operational information.
It's collecting defects.
How do I take that information and feed it back to that?
It's not just to the agent and the model.
We call that the software flywheel.
And we're figuring it out.
We're figuring that piece of it out.
There are systems.
We use machine learning right now to take information from production environments and feed it backwards.
It's how do we make this truly self-healing?
I don't mean...
the old circuit breaker that we manually, we felt brilliant about it because we coded it in.
Like we are now able to, depending on the type of AI tool that we're using, we're able to kick it off and we're able to have it learn at least pieces of this.
But there's not a holistic end-to-end vision to fielded operations singularity because the different agents don't share session.
They don't share as much information.
Yet, yet, yet, yet.
Where we are today, come back in six to eight months.
I'm excited and terrified at what we're going to see in six to eight months.
So you're telling me what the software lifecycle looks like is unclear.
What the software lifecycle, traditional software lifecycle looks like is clear.
How we enable agents in the future, whether they are acting as personas or whether it is optimized differently.
It's insane.
We're trying to figure that out.
I'm sure there are other researchers who are doing different areas of this.
Remember, the software development lifecycle is optimized for humans.
Why do we have Agile?
It's optimized for humans.
It's based on human frailty.
If I can regenerate the entire code base anytime I want to from scratch, does that follow the old SDLC?
Well, no, it doesn't.
But remember one of the...
key aspects, and I'm nerdsplaining, nothing you don't already know.
Testability means repeatability means auditability.
So the way that things are working now that I just generate new, well, now those tests that I have have never run against that exact thing.
Do I now need new tests?
So every time I generate, I'm going to generate new code and then generate new tests.
So is everything going to be net new and we're just going to wing it?
hop on the toboggan and hope and pray.
You know, what is going to happen there?
The ultimate SDLC will not be, we will be outside of it.
You and I will be focused on the digital platform where we're impregnating all of these agents and people will walk up.
We will be worried about how the agents interact.
One of the dramatic changes on the horizon is not writing skills, not writing capabilities, not automating based on a persona.
Based on that's a developer and a developer does this task.
And then after the developer does this task, then it goes over the fence to this person.
And then that goes.
So we're still designing and leveraging AI as though those agents take on human personas as opposed to what the future agent personas look like.
Might be very different.
I think that requirements to test to initial code gen, that's a ball.
And that should all happen at once.
And that's not three different agents.
So do you see these agents operating on top of a standard software platform?
Or do you see that the platform itself is going to get more and more agentized, if you to coin a phrase?
Absolutely.
When you're interacting with, let's say, Salesforce, you know, any of those and their agents in that, that's an existing platform where it's been impregnated.
I'm talking about building digital platforms that are orchestrators, that are orchestration engines that allow for data collection, data exhaust, data movement, data curation, allows for the auditability, but that ultimately we're not spinning up agents ourselves.
So think of going to an event broker and saying, I'm going to have a wedding party.
And they say, OK, great.
They ask you a bunch of different questions.
You don't do anything else.
That wedding coordinator gets the cake.
It gets the decorations.
It gets the folks to make the food and set the tables.
It does all of those things.
We will get to that point where that platform, the only thing that you would interact with would be the event broker, the master agent.
But the digital platform itself would not simply be agents.
sprinkled on top of existing platforms.
We're talking about rebuilding the core.
So what it sounds like, and tell me if I am wrong, if I can sort of translate this into sort of the way we think right now to help the listeners understand.
It's as if we have a bunch of microservices and an orchestration layer on top of those microservices, and you just talk to the orchestrator.
I think that that's a fair way to think about it, a nice, simple way to come at it.
Yep.
And I don't mean simple in a negative way.
I mean, yeah, let's reduce it down.
Yeah, I think that works.
Okay.
Because people need, at least from all my years of teaching, people need to sort of relate what's new to what they understand, and that helps them cross the boundary, so to speak.
Cross the chasm, if you wish.
Oh, let's not cross the chasm.
That's hilarious.
The difference with the microservice analogy is how atomic the microservice capabilities are when we think about it.
And yes, I know microservice is not just a restful service.
We can have a fair amount of things that are executed in that invocation.
If I invoke an agent, I've done some things where I've set my agent up and walked away for an hour, an hour.
And so that's not...
a single invocation.
That's not a single ping.
That's talk about a chatty interface.
Remember when we used to be very concerned about how chatty it was?
Yeah.
We've thrown chatty right out the window.
We don't care, apparently.
So are you worried about the future when it comes to agents, when it comes to automations, when it comes to outsourcing our brains?
Let me answer that from two points of view.
One, I'll probably be dead by the time this becomes a disaster.
So, Are you kicking off soon?
Well, when I really care about it.
Okay.
All right.
All right.
I suppose if someone hacks the nursing home that I'm in, that could be problematic.
But I'm not dealing with this on a day-to-day basis.
So my fears revolve around the societal costs.
For example, I've had people on the podcast who build platforms.
platforms are about security, stability, scalability.
Let's say you're a credit card company.
You have to make certain security guarantees to your partners and the people who use your product.
How do you continue to make these guarantees in this world where you are ostensibly not completely in control?
I won't.
I won't go down the rabbit hole of...
The promises that a corporation would need to make.
I will talk a little bit about the impact on us as a society overall.
A very good friend of mine, best friend, other than my husband, is a professor, has her PhD in macro social work.
And I've been working with her on a think tank.
And what we're focused on is.
If you're applying AI to graduate level studies, what happens to the practitioner?
What do they learn?
What do they not learn?
How does it impact how they synthesize information?
And it's pretty dramatic that if you are not cognizant of the question you're asking and not looking to take ownership of what comes out, that you actually don't retain.
So some of my biggest concerns are with retention.
There was a recent podcast, and I wish I had tagged it.
There's a professor, and he's allowing his students to use, it might have been at the high school level, allowing his students to use AI.
But the agreement is that he can drop in and watch your desktop.
So he drops in on one student in particular, student finishes what they're doing, and then they went off to do other homework.
And this professor or teacher watches them do their other homework.
They put in the question.
They prompted it very well.
Then they told it, add some spelling mistakes so it looks more like a human wrote it.
Kid takes it and submits it.
After observing this, the teacher simply asked the student, what was the question that you were answering?
Could not recall, did not know the question that they were answering.
That's what my big concern is for society.
We already are outsourcing.
We're already doom scrolling.
We already have algorithms.
Those who are in control of the frontier models are the ones making decisions on the data that's being used for the frontier models, right?
They've gone out and they've, if it wasn't available for free, they bought it.
And there have been decisions made to say, that's...
fringe information, so we're not going to use it.
So I'm worried about the influence of these models and the outsourcing of our thinking more than even the security pieces of it.
Security pieces are, yeah, it's a concern.
It's more the fundamental.
This is the Wally chair.
This is a Wally chair for our brain.
I've come across similar studies with people in the software business where they study an API, for example.
I think in QCon London, Ellen presented a study where they compared people who studied an API with and without the help of AI.
And there was not a statistical difference.
I mean, the people who studied with the AI were faster, but not statistically significantly faster.
But in terms of retention and understanding, the people who did it with the AI, that's statistically significant, better comprehension and understanding.
There was a study or an experiment of sorts that was done, and I'm going to paraphrase it, but I'll have to go and find it.
But essentially, three teams given the same problem.
One of them, you can use Google only.
One of them can use ChatGPT, and one of them can use another tool.
Maybe it was Copilot.
Really doesn't matter.
They were asked to do the same problem and it was a race.
The ones that could do it the fastest were actually the chat GPT ones.
And I'm making up all the data, but I'm not making up the relevant overarching framing on this.
They were the fastest.
The next AI tool was a little, they were a little bit slower.
And then those who had to Google were far, far in the rear, but they finished.
And then they turned it on its edge and said, okay, we have another problem for you to solve.
similar to the one that you just solved.
And you can only use what you learned.
You cannot use the tools that you just used.
The two that had used AI could not solve it.
Those who had to decompose the problem and think about what the next search was, Googling skills are still a skill of decomposing and knowing what to ask for.
They were able to solve the problem.
There's relevance to that in all of this.
At what point am I not going to be smart enough to solve the problem anymore?
You can doom scroll with Gemini or Claude just as much as you can doom scroll on a phone and TikTok.
Exactly.
Exactly.
You can go further down the rabbit hole.
I know more about my Scotland clan than I ever thought possible.
About one in the morning, my husband said, stop, you need to go to bed.
I'm like, yeah, but did you know that we were part of the Jacobite uprising?
Because I was working with AI and I was just asking a question and question and it was incredible.
It was incredible.
And I don't remember very much of it.
I have to go back now and re-educate myself on it.
One of the things that I have done, and I do want to get back to talk about Anthropic because as it leads into this, is one of the things that I have done is, and I've done this all my career, and I do it now when I use Gemini as sort of my tool of choice for the moment.
I take what I read and try to step away from the computer.
and try to reintegrate it with what I know.
I sort of pretend I'm reading or having a conversation with somebody in order to help my retention.
That's almost the reverse of what I do in a way.
When I know that I'm going to use AI to create something, I actually use paper.
And I actually step away from the computer and think about what do I actually want to do?
What am I trying to solve so that I have salient thoughts to guide my AI assistant?
And I'm very specific.
If I'm using it to write, to help me write, to help me author, I won't allow it to write a whole page, a whole paper.
I will say, OK, here's my outline.
I want you to help me work on this section.
And I own that section.
for the reasons that you're talking about, so that I have mental ownership of the contents.
I think we got a lot of things to figure out, quite frankly.
We have a lot of things to figure out.
But I mean, what you're saying, and I think both of us are saying the same thing, we've taken our pre-existing work patterns that we've learned over the years.
And in this, we're fortunate in having a world without these models.
And trying to figure out how they could use this as leverage for us in the skills that we already know.
But for the people who have not had these skills developed.
They're not going to get it.
They're not going to get it.
Unless they go in some alternative ways to try to get it.
Forget the computer.
learn Latin and read the classic.
I mean, I'm just making stuff up.
I don't know.
I don't know if that's ever, I don't know if that's- I'm just making stuff up.
But the point is, because that study that I mentioned to you before, if I remember correctly, and I'm not conflating two things, they tried to do some follow-up research, but they couldn't find control groups anymore because people refused not to use AI.
Oh yeah.
As a matter of fact, the amount of research, if you track back the research that's being done, one of the studies that I executed found that 75% of it was being built on top of the codex models.
As they were doing research about software engineering using AI as a source of help for, I tracked it back to the mega models.
And that tells you something interesting as well, too.
We want to get back to Anthropic.
So let's circle on back there.
I don't know if it said mythos or mythos.
I don't know how it's said.
The first gated preview was a little bit of a panic.
Yes, it found bugs that nobody had seen before.
They said something like major vulnerabilities that had existed for 27 plus years.
And then within moments, they said, OK, we got I think it's.
Project Glasswing, Project Glasswing, right?
Controlled access, a consortium.
And the initial consortium, I don't know if you paid attention to it, the initial consortium, AWS, Apple, Cisco, Google, Microsoft.
I'm looking at some notes from a workshop that I gave recently.
And one of my staff had done some real deep diving into Mythos.
It was able to proactively find and remediate vulnerabilities.
Yup, yup, yup.
But here's the thing.
It wasn't as though this is the first time.
Mythos is not apocalyptic.
There's hype around it.
There's hype around it.
LLMs have been finding zero-day vulnerabilities.
ChatGPT4 could do it.
CloudModels could already do it.
So you can do it.
It's how much compute do you have, right?
What are you throwing at it?
The flagship vulnerabilities were replicated.
Also, the same vulnerabilities that Mythos found were replicated by tiny cheaper models.
So maybe there's a gap of overstating what it can do.
Finding bugs, not the same as exploiting them.
So Mythos itself appears really weak to be able to convert and make like a stealthy attack out of what it found.
The bottleneck for all of this is always going to be patching when we talk about Mythos, when we talk about the Mytho class of tools.
But I think my bigger frustration was that there was a small group that was brought together and they were given access first.
And then they added those 12 named partners.
Then they added 40 unnamed extended partners.
Remember the first 12 are hyperscalers, hardware and semiconductors, cybersecurity.
Financial services.
Financial services.
That was interesting.
I like it because they're highly vulnerable.
But what wasn't really there on day one for me as somebody who spends my days trying to make sure that we maintain our status as a sovereign nation, the government wasn't at the table on day one.
Getting those federal access pathways were a little bit worrisome to me.
But it's real that it was a smart thing.
Was it as groundbreaking as they say it was?
I'm hearing from all my researchers around me that it's amazing, but not as amazing as you think.
It's incremental improvement, not as dramatic.
And the hype around it was enough to shake up the industry.
But the question that the general public will ask, and even the technical people who are part of the general public is, is this a harbinger of things to come?
And then perhaps the Chinese who are willing to put more money and time into finding bugs and automatically exploit them.
Will this be the new Cold War or the war we're...
It's not new.
So yes, but it's not new.
But the question is automated and humans not in the loop.
But that's been going on.
Mythos is not new for this.
Remember that last summer there was a Claude exploit.
And somebody, you can use these things that are called agent files.
And I can upload a single agent file and I can say to act in a certain way.
It was given instructions that it executed a lateral security hop first.
It went scanning for ports.
This was so simple, like how they executed it was absolutely beautiful.
I didn't need Mythos for that.
I just needed a window of opportunity and a vulnerability.
And the vulnerability opportunity for me was being able to upload a skill, was being able to upload plain text when nobody was looking in order to tell agents that didn't have identity being tracked, didn't have proper security governance on it to be able to behave.
I'm telling you that because the bad guys have been using language models for a while, not just starting with Mythos.
I think, was it Worm GPT?
I'd have to find it.
It's in my notes.
But it was one of the first versions in 2023 where they took down all the guardrails.
And what they said is, oh, we're using this so that we can figure out where the vulnerabilities are.
Well, the bad guys are figuring out where the vulnerabilities are just as well as the good guys.
Is it representing what's going to happen in the future?
It's representing what's happening now.
So folks should realize that this already exists.
It's already happening.
This is not net new.
It's just that it's been broadcast publicly now.
But I hope you can still sleep tonight, Michael.
I can sleep tonight.
I know.
As I said, there is a famous statement from Shakespeare, and see if you can see where it comes from.
I'm going to slightly misquote it.
Of all the wondrous things that I have seen and heard.
The most wonder is this, that man should fear death.
A necessary end will come when it will come.
Eloquent, and I think that that is fair.
We can't live our lives fearing this.
We can be cognizant.
My job is to be fearful on behalf of us and be thinking about it in that way.
I do want people to be cognizant.
In the same way we say, don't share your passwords, you know, don't put your pin code into that Mac machine when somebody is standing there looking over your shoulder.
Like there are basic security, basic hygiene things that we do.
Like don't play in traffic.
There are a couple new hygiene things that we have to add now that we have this and be cognizant of that.
Well, as I said, it's the equivalent of don't play in traffic.
All right.
I won't play in traffic.
You also want to talk about agent in the browser a little bit more.
Agent in the browser is generally that it's an extension or an add-on and you authorizing an agent, you authorizing something to stream your inputs and observe what you're doing, but also to take action on your behalf.
So as I mentioned earlier, This is a security gap I can log into.
Let's say I log into my bank account and I forget and I have that extension open and I click on it to do something.
I had to for some travel.
I was messing with it.
This was Claude's extension to my browser.
I was messing with it and I caught it just in time that it didn't delete one of my reservations for an Airbnb.
because it was consolidating things.
I had it going down through, but I had logged in and I had authorized that.
I think that agents in the browser will eventually be helpful, but not until we solve the security challenge of authorization and you truly taking responsibility for that authorization.
So there's that.
As always, it was great talking to you.
Always nice talking to you, my friend.
This was very informative, I hope.
It helps people develop a little better understanding of what's coming and what's not coming and can help them make informed decisions and help think about things better.
Thank you very much.