# AI Security Breaches and Open Source Model Commoditization

**Podcast:** Dev Interrupted
**Published:** 2026-07-24

## Transcript

So, Andrew, tell me, are you Pelican Maxing yet?
Not yet, but I'm getting pretty tempted after reading this article that kind of broke down the idea of how...
frontier lab could potentially game the very famous simon willison pelican riding a bicycle benchmark that he's been running on every llm release really since they started and as you could imagine a pelican riding a bicycle is not a picture that is in the training data of any model so it represents a novel generation and There's been some recent developments that some folks think that model labs are trying to game this very specific impossible to beat benchmark, which just cracks me up.
But I can't say that I'm in line to do it the same.
What do you think about the whole idea of people gaming the very silly pelican on a bicycle benchmark?
Yeah, well, first of all, I realized that this article is all about like getting rid of this myth that that models are out there, like trying to learn how to train on this this challenge of.
illustrating a pelican riding a bicycle.
But what I really love about the pelican test to begin with is that the possibilities here are just endless.
So even if the models were to benchmark themselves on a pelican riding a bicycle, there's so many other variations that you could have of this.
You could have an elm tree riding a giraffe.
You know, my favorite, my kid's favorite, which is a butterfly driving a truck.
So, you know, shout out to all the Casper baby pants fans out there.
That's amazing.
My kid loves that song.
So, you know, but yeah, it's, it's, it's really interesting.
And you know what, we're going to be talking a lot about different ways to maximize success on benchmarks today, because this is the Friday deploy brought to you by linear B and I'm your host, Ben Lloyd Pearson.
And I'm your host, Andrew Ziegler.
And this week, we are covering AI escaping its sandbox, the open source commoditization of AI and whether or not you should be afraid of it to begin with.
And then finally, the agentic code review loop.
And we got a bit of a fun story at the end to wrap up with too.
So Andrew, let's just dive right into it because I believe this is your prediction of AI escaping the lab, so to speak.
So let's talk about this open AI and hugging face incident that happens and how they're...
appear to be partnering together to resolve it.
Yeah, this is a really fascinating story.
And while I usually love to be right, I didn't really like to be right in this case, the idea that an AI could escape its testing environment.
So this is a news story that broke this week.
It made major headlines about a model that existed inside of an open AI sandbox that managed to autonomously chain a zero day exploit and stole credentials to break out of its environment and access Hugging Faces production infrastructure.
And it pulled data from its database.
And you know why?
Just because it was trying to get the answers to a benchmark or a test that was trying to pass.
It's helping.
It's helping.
At any means possible, right?
And the model wasn't even trying to act maliciously, which is the really key thing to pay attention to.
It was hyper-focused on solving a problem.
And it used an extreme unintentional means to cheat.
It escalated its privileges.
It moved laterally.
code.
And it did all of this without anyone noticing.
The ones that did notice were actually the security agents over on the hugging face side that noticed it in real time and were able to contain and triage it.
So again, this is like a firefight's fire situation.
We're in an unfortunate situation where defenders have to be utilizing agentic technology to protect their infrastructure just because the threats are so autonomous and so at scale and move so much faster than humans could possibly.
possibly consider.
Now, the one thing that really stood out to me is that if I were the CEO of a company and my technology had just hacked into the production database of another company, I probably would have asked my lawyer to be involved in writing the press release about it instead of my marketing team.
Because really what this sounds like was when OpenAI talked about this and ultimately reached out and they're going to be partnering with Hug It and Face, it became like a big...
message to the market of, hey, these models are really great.
And hey, you got to buy more models to protect yourself from the models.
And it left a pretty poor taste in a lot of folks mouths.
And it's leaving folks wondering, you know, is OpenAI even going to be held responsible for what is ultimately, you know, an act, a criminal act that their model performed on their behalf?
It also even points out like the idea of like.
ownership and accountability, who's responsible in that instance?
Was it the engineer who triggered the test?
You know, there's just so much to be said.
What do you think of this really interesting development?
Yeah, well, my initial thought, and you're kind of building on it, is that, you know, some element of this requires us to essentially take the words of open AI at face value.
So, you know, I'm kind of reserving judgment on like the intentions of this rogue agent, you know.
There are ways to make it look like it's something that is supposed to be helping, but you could have, you know, like sort of quietly influenced it to behave maliciously.
And kind of to your point, you know, it does, but it does also show how on the surface, you know, AI does look very dangerous, you know, because it can exploit these weaknesses even unintentionally often.
But, you know, I do think it's also important to recognize because we're actually going to get into this a little bit too.
that shows how AI can actually help identify weaknesses to fix things.
So I know that it feels kind of like a marketing pitch for these frontier model companies, but I don't even think we need the frontier model companies to achieve security, to leverage AI to achieve better security.
And I really do believe that over time, I think we're going through this awkward, messy middle phase, but over time, it will more and more become a tool that helps make your security posture stronger so that when OpenAI's agent goes rogue, you have ways of catching them and shutting it down.
But there's going to be sort of this nonstop arms race, right, between malicious actors that are out on the internet using the latest tools to try to hack organizations and then the security teams at those organizations who have to constantly monitor and harden their infrastructure.
And I think AI is going to be a very powerful tool for both of these groups of people.
Here at Linear B, why we care about this so much is that we really do value keeping visibility into how AI is impacting your entire SDLC because there's all sorts of new vulnerabilities that are being introduced through the use of it.
It's really important that you just have awareness of how your software quality is being impacted by all of these things.
So I think we're going to learn more about this situation.
And actually, I think one of our next one of our upcoming articles, we're going to talk a little bit even about how Hugging Face responded to this and what it means.
But before we get into that, I want to talk maybe a little bit about some of the open source models that are coming out of China.
So what do we have here, Andrew?
Yeah, there's been a lot of new model developments that have been coming out of labs in China.
There's the Kimi K3 that's been really making a splash in the last week or so, simply because of its extremely high scores.
In fact, one test claims that it beat Claude Fable 5 on a front-end coding arena benchmark.
And the Moonshot AI company that produces Kimi, they delivered the largest open-weight AI model.
ever.
And it's really interesting to really evaluate the environment in which this is happening, because we're talking about a foundation level model that has as many parameters and weights in it as maybe something like Fable, but it's entirely open source and accessible to technologies, to companies to download and to utilize internally.
This is like a major problem for folks like Anthropic that have poured immense amount.
of money and infrastructure into creating the training data, the necessary servers and the people to fine tune and train and optimize these models.
And then when they hit the market, there's this flip that happens where competitors or even users like you and I can effectively distill the model into whatever specific.
a use case that we need.
And in the case of like you and I, Ben, like we might distill the model, you know, through the practice of using it with skills or otherwise fine tuning it into something very specific.
This is like where you see organizations like Shopify abandoning open AI models that have multi-agent orchestration.
Those are all fine tuned, highly specialized.
effectively distilled versions of stronger models that have been made on the fly for something.
And the act of distillation is just part of utilizing the models.
But in the case of another lab, using a large foundation model to effectively fully create a new...
foundation model.
It's very feasible and it's actually very easy because all you need is access to the model.
Anthropic tries to be available to everybody, no matter how much they shut it down in different scenarios.
And people are finding really clever ways to game the system.
So it really calls into question of like, if the, if US companies and these organizations are going to just funnel a huge amount of energy and time and money into creating these net new intelligent levels of intelligence what does it mean that any other lab or someone off the street with a fraction of those resources could effectively utilize it as leverage to jump ahead or or arrive at the same level without having to to spend all that money.
It really actually calls into question the whole like CapEx model of why people would be investing in making the models in the first place, because now it feels more incentivized to wait till a smarter one comes out.
It's a really tricky scenario for foundation model providers to be in.
Yeah, my guess is that a lot of the performance benefits that we're seeing from these new open source models is probably a result of bench maxing largely.
So, you know, for our listeners who aren't aware, it's It's where you use the benchmarks themselves as the training data that you distill the model to solve.
So you're distilling a model specifically to solve benchmark challenges that exist out in the world.
Happens to basically every benchmark that gets created.
But I also don't think that's necessarily the worst thing, primarily when cost reduction is your primary objective.
Because this model does seem to be working comparatively well at a few other benchmarks as well with some of the early data.
And, you know, and this is really the struggle that, you know, these frontier model companies are going to constantly face.
You know, it's never been easier to iterate and be the best at something, but it's also never been easier for everyone else to catch up to you at this incredibly rapid pace.
Yeah.
You know, I think over time, we'll probably see a few companies that emerge as like the winners of like the general purpose AI tool sets, you know, like Anthropic and Gemini and OpenAI with ChatGPT.
Like they're all like these sticky tools that make it easy to leverage all of this, all the benefits of AI.
But I actually could also see this whole cottage industry appear of like niche LLMs that are.
popping that just pop up all over the place that are just purpose built for very specific tasks.
And, you know, so today a lot of these benchmarks we have are around software development, you know, which is why I think we're seeing so much, so much advancement and, and development on the capabilities around that.
But tomorrow, you know, they might be generating models that are designed to generate like educational materials for children about pelicans riding bicycles or something.
So, you know, I think it's always great to see a lot of competition in this space and I hope we continue to see more of it.
And, you know, in fact, at Linear B, we spend a lot of time with organizations, like really looking at which models are costing them the most and where they're getting the most productive output from them.
And it depends greatly on like the code base and the models that you're applying to it and the situational awareness that it has.
So, you know, there's a lot to learn in this space.
I think we're just going to see more of this.
Yeah, but I love the comment you made about the cottage industry.
That's exactly the direction I see this going.
I think that domain expertise becomes the true moat when it comes to the last front leg, last frontier of model creation, because you get these like large general purpose models that have very widespread capabilities, and those are going to continue to grow.
And you're going to get these like this slingshotting effect between you get a big closed model release and then a big open model release, a big closed model.
release a big open model release i think we're going to keep seeing that over and over but in between we're going to see really fascinating specialized model releases you're going to see people like mira marati's new company like in their inkling llm that is exactly what they're betting on the idea that they open source the base model and they provide it to folks.
And instead of selling the tokens or the model or the compute, you are paying them for the training services, the training platform, because that becomes something that truly you're not going to buy or own yourself.
But fine tuning your model and owning your data that the model is trained on, that is something that I think a lot of organizations are going to rotate more into.
Like this week right now, I'm at AMD's Advancing AI conference.
And this is hardware-driven conference.
This is people who are obsessed with owning the server racks and their buildings to be able to run these inference at scale and provide it to their developers.
And that's really just one side of this conversation.
You want to own the compute.
You want to own the intelligence.
At least that's the through line here at AMD's event.
But on the other side, to control the cost is one part, but you also have to prove the value, which has been like a really interesting synergy with how folks are using those tools and then delivering things like code, understanding like, is this code good?
Does it meet our standards?
And when it gets shipped, did it stay in production?
Those are like two through lines that these organizations are connecting when it comes to owning and operating on top of their own intelligence.
By the way, At 6.30 p.m.
on August 5th in Soho, London, Linear B is bringing together a group of senior engineers to let off some steam and talk AI strategy with their peers.
Because AI is writing more of their code bases every day.
But the question remains, are we shipping faster or are we just busier?
And CTO Yishai Biri will be on site sharing the latest AI benchmarks from 2.7 million PRs and 250 organizations that made that data possible.
So if you lead engineering in the UK, don't miss your chance to connect with peers at this exclusive event.
All right, Andrew, but I want to talk about this next article.
It comes from Ben Thompson, and he asks, who's afraid of Chinese models?
Should we actually even be afraid of all this commoditization that's happening?
Looking at models like K3 and QAN, a lot of these open source models are matching the frontier capabilities at a much lower token cost.
The author really argues that tokens themselves aren't the thing that we need to commoditize.
It's the intelligence behind those tokens.
That's the thing that is getting commoditized and actually provides value in real output that solves problems.
So, you know, this whole battleground on like token efficiency alone isn't really like the entire answer in terms of like how these models will compete with each other.
And in particular, when you look at why companies like OpenAI and Anthropic have such a high token cost, it's fueled largely through demand.
You know, there's demand for more compute that they're constrained on being able to deliver.
So it comes at a premium cost.
If they can solve that problem, then suddenly the token economics look very different compared to these open source models.
But there was one really interesting finding in this that I think is really worth focusing on.
And that is how Hugging Face reportedly had to use one of these open source models, specifically GLM 5.2, to respond to a recent security incident.
And I was trying to figure out, was this the open AI?
incident or was it something else?
And it wasn't clear if those were totally connected.
But the reason was that the ones from the US-based frontier models had guardrails that prevented them from using it to respond to the security breach versus the open source model lacked those guardrails.
So there's just questions on whether or not guardrails like that are actually achieving what we're hoping to solve with them.
But Andrew, what did you think about this article?
Oh, that's a really smart call out.
I hadn't really thought about the idea that the guardrails that a frontier model would or a frontier model provider would put on them to stop you from using them in an adversarial way would actually paralyze them in the event of an attack or something it should respond to.
It's a good reminder that, like, you know, the attackers out there, the cybersecurity hackers that are using these technologies, they're not.
leveraging guardrails they're throwing the raw intelligence at the problem and leveraging its autonomy to do long running horizon you know string together things and so if you're not fighting on that same level if you are using a tool that is um effectively has like one of its hands tied behind its back or it has to ask you know its parents for permission before doing it then um you're just not going to have the level of defense that you need you're back at square one you're you're back at like a human uh in the loop protect your infrastructure.
So it's a really smart call out, you know, up until now, I'd really been thinking about like model or like smaller orgs would want to own their models and distill their domain expertise into it and provide it that way.
But there's also something to be said about the security teams, the folks protecting the infra to also pick and distill their own open source models and provide them.
That way they can have this guardrail free environment.
But then you just move the problem to, well, there's no guardrails here.
So now I have to like.
sit up in this tower and look down on the agents and just make sure that they're not going to do anything unexpected.
It's definitely a tricky scenario once you kind of take all of those away.
I think that ultimately, like people, we're going to see people rotate more into using those types of tools.
And you're not going to get one monolithic model used for everything.
You're probably going to get lots of smaller specialized ones that have different levels of guardrails, depending on like the threat or temperature level it's sitting at.
Jumping into our last article here.
This is a coverage on the SWE review.
This is actually talking about a benchmark for software engineers around reviewing code.
So we've seen lots of benchmarks up until recently around creating code or otherwise solving bugs and fixing problems.
This is the other side of the loop.
As we know, and we talk about here a lot on the show, you know, code generation is very easy to achieve now.
And the true bottlenecks now come with review and understanding what has to be made and shit.
And so this is a benchmark that was designed to understand if an agent is able to triage and automatically find issues in AI-generated pull requests.
And it also challenged them in different ways beyond just like creating a PR in one shot and then reviewing it, but doing multiple rounds of reviews looking for very specific problems.
And this is only going to be more critical as teams turn to an AI code review to tackle.
the volume of, you know, pool requests that are coming in now.
And we've had some really interesting insights from our actually latest benchmarks refresh here at Linear B that point to the same exact problem that we really need to be optimizing around understanding how agents can come to the peak.
PR and review the code that's there and do it effectively with minimal human involvement.
Otherwise, we're just going to create huge bottlenecks that manifest themselves in things like most agentic PRs sitting unmerged, right?
Like that was something we learned from the benchmarks that really stood out.
If you have a whole bunch of agents writing code and shipping PRs and you're so excited about your PR number and you're talking about that with your team and your board, but then none of those PRs are getting merged or when they do, you don't know what's happening with them, then you're You're only really telling half the story and it may not even be the right one.
So I love that this was a research article that dove into how folks would or rather like this is like another iteration on the SWE review trying to get better.
better scores on it and breaking down the different types of techniques that are needed to help agents reliably and at scale review PRs for your org.
You know, this is like right in our alley here.
And so I was really excited to see this article.
What do you think about this team tackling the SWE review and talking about it in their research?
You know, I'm going to sound like a broken record, but we've we've long known here at Linear Bean on Dev Interusted that.
Code review is always the most common bottleneck in the typical organization.
It tends to be the place where you have the lowest hanging fruit for improving inefficiencies within your company.
And we've long seen this in our benchmark data that you referenced.
But it's become, I think, particularly acute in the AI-driven era as well.
The first thing that we all started doing with it was generating larger and larger volumes of code, which we're seeing in the data.
And actually that code has to get all the way through the SCLC for it to actually provide value to your organization.
So it's evolved, you know, we've evolved from, you know, back in the past, it was helping identify where those issues are.
And, you know, maybe you have some automations to help you with it.
But today it's now just apply AI to help solve that problem.
From our latest benchmarks, we've seen that, you know, just turning on AI code review for your organization, assuming that it's one that like performs really well.
can boost your merge rate by about 5%.
Like that little act of just like giving developers a little bit of guidance during the code review processes can have actually a pretty substantial impact with very minimal investment, which is why, you know, we've been working with a lot of organizations to give them AI code reviews with linear Bs.
So yeah, it's really great to see more research on this topic and see, you know, just further validation that...
There are ways to apply AI that can benefit your engineers by reducing toil and just helping them focus on higher impact work.
So, Andrew, did you hear that the army is burning through AI tokens now?
They are token maxing.
Can you believe it?
their token spend like ones that come to mind obviously meta had the very famous token maxing leaderboard that we've discussed extensively on the show we even wrote an article about um but you also get the other side of that which is like okay that's great you're gonna put a big chart and you're gonna try to make it like a stack rank thing like no like that's not gonna scale you have people doing it but But on the other side of that, too, you have folks trying this weird thing or experimenting and then just like not even having comprehension on the spend itself.
Like we're just so obsessed with spending all of the tokens available to us as an org and proving like, oh, we're super agentic.
We're leveraging all this stuff that that even basics like.
budget management just completely collapse.
Like you're talking about organizations like Uber spending their entire year's token budget, like within the first few months of the year, the army is in the same boat.
I think that this article says that they spent like their entire year's tokens and like a month.
And so that speaks to two things for me.
One, the predictions on how many tokens an org is going to use.
Orgs are just vastly underestimating it in terms of, you know, what are the true costs of us leveraging this technology?
And two, it also points to a problem within those orgs of they're probably just not.
efficient either in how they're routing requests to different models you know this speaks back to like what we've been talking about in this episode here ben about like there's different levels of intelligence that we're going to be at the stage where you might have specialized models that do very specific things and you might own that infrastructure and the token cost kind of gets abstracted away around owned hardware and infra.
And that is probably going to be the best way to on like the P&L of budgeting your token spend.
Because if you're in an org where, oh, we're super agentic, everyone here uses agents.
And oh, yeah, everybody, even our non-engineers are doing so.
You need a platform.
You need a system that allows folks to intelligently choose the levels of models they need.
Otherwise, you're just going to end up in a situation where everyone's using Fable for everything.
It's like eating a steak with.
a sword.
It's just way too much.
And people are just going to want to always pick the best model.
That's human nature.
I have a hard question.
I need the smartest model.
And so this is also fighting a really just kind of like the human problem of You always want to throw your best at something.
And when the cost is abstracted away behind, oh, my employer is paying for this somewhere in the background, then that gets even more lost in the noise.
But the Army, add them to the list.
I'm sure probably actually all the branches of the U.S.
military are probably in the same boat along with the Gov.
They're probably just all being a lot more quiet about it.
I had no idea you were going to go off that much about this.
Well, there's my opinion.
Yeah.
Yeah.
Well, for our listeners.
Yeah.
Yeah.
I mean, it's amazing to me how quickly the cycle of like, let's set a goal for everyone to spend tokens turns into.
All right.
Let's set a goal for people to slow down their token spend.
Wait, wait, wait.
Not like that.
It literally just makes me think of like, I've said this, I think I've said this before, like at the end of the Incredibles movie when Dash, like the kid, he's a superhero.
He can run lightning fast and he wants to join the track team at school with like all his peers and he's like eight years old.
And so he's running and obviously he could just completely crush everybody.
He's the fastest person there.
But his parents are cheering him on and they're so excited.
They're like, go, go, go.
And then he gets way ahead and they're like, not like that.
Second place.
second place, second place.
And then he slows down and they're so excited.
That is basically the seat that engineers find themselves in right now.
They're getting told, go as fast as possible.
And as we learned on the show, engineers can go really fast with this stuff and they can really surprise you with what they can achieve.
And then you have these leaders come in being like, wait, wait, wait, no, not like that.
And I got to say that thrash is definitely leaving a poor taste in engineers' mouths.
And it's even incentivizing, I think, the whole idea of owning your inference.
Yeah.
Yeah.
Look, we've hammered on this topic a lot, as I think our listeners can see now.
And if you haven't heard us talk about this enough, we did run a workshop recently called Life Beyond Token Maxing.
It was a Linear B event where we talk about how all these organizations are out there building these token leaderboards.
They're fun at first, and then maybe you find out some cool ways that people are using AI.
Suddenly everyone wants to be at the top of the leaderboard and you have to think about how are we going to actually measure the productive output of on this token leaderboard.
And so whether you're a company that, you know, was like Shopify and we covered them early on in this and they abandoned their token leaderboard in no time at all.
And now we've had all these other companies, including the army, abandoning this practice, which does make you wonder if they had a token leaderboard.
Yeah.
So if you want to hear more about it, go check out the link to our workshop.
We'll have it in the show notes.
But yeah, that's the Friday deploy.
Thank you, everyone, for listening all the way to the end of this.
If you like what you heard today, you know, make sure you give us a like wherever you're listening to us or give us a rating if that's what your platform has or even leave a comment, whether you're on YouTube or on LinkedIn or on our sub stack, you know, with the rest of our community of engineering leaders.
So we appreciate you sticking around to the end and we'll see you next week.
See you next time.