# AI Disruption: COBOL, Security, and Productivity

**Podcast:** Dev Interrupted
**Published:** 2026-02-27

## Transcript

Andrew, what do you think about this whole like open claw moving from like the self-hosted Mac minis to the cloud now, like all these cloud services that are popping up?
I've seen so many of them pop up in like the last week.
It feels like ever since there was the open AI acquisition, there's like a lot of people tackling the problem, trying to get a piece of the platform pie with agents first.
It's like, can we be open AI to the punch or something?
And the answer is pro probably not, or you're just gonna get different punches, frankly.
But I think it's actually a really interesting phenomenon because it's really what you're saying in that world is I don't trust myself to host it on my own device securely.
So I'm gonna trust a stranger to host it on their device securely and then pay them to hope that they're not reading my information.
It kind of flies in the face of it makes open claw feel even more dangerous to me because you're basically putting it in a Phenopticon.
Yeah, yeah, with proper protections maybe, but man, who knows who you who's prompt injecting your open claw agent without you even knowing.
But that said, if you if you're hosting one of these things in the cloud and you have a really strong case for why, or you have really interesting experience, like I'd love to hear from it.
I'm speaking from just the experience of having tinkered with a only in a local host kind of setting.
I couldn't imagine putting it in the cloud.
So I'd be curious to learn more.
Yeah.
Well, welcome to the Friday deploy.
I'm your host, Ben Lloyd Pearson.
And I'm your host, Andrew Ziggler.
All right, let's what do we have in the new this week's news?
We've got the COBOL Apocalypse and what it says about how AI is impacting software companies.
We have something straight from a Silicon Valley episode of AI assistance taking down AWS pro production.
Uh dev productivity studies meet reality.
Seven takeaways from the AI safety report, and we'll close out with the retirement home for Claude.
So, Angie, let's get right into this COBOL apocalypse.
So, what's what's going on here?
Yeah, so the uh IBM stock took a huge dip in the last week.
Um, it lost 13% of its value after anthropic announced Claude Code's COBOL modernization capabilities, basically scaring a bunch of investors away from the moat that IBM has around mainframe commuting, uh computing and uh languages like COBOL.
And COBOL is one of those languages that you know we talk about and joke about, but it's hard to under uh to understate the amount of impact in the world that it powers.
I think over 80% of all ATM transactions run on COBOL.
Um, so you're talking about a huge surface area where IBM is, you know, the supreme de facto experts.
This is kind of uh the latest example of AI news announcements taking a big bite out of pre-existing companies and their valuations.
Yeah, you know, this story actually hits really close to home to me because my mother actually worked in tech with COBOL for many years.
Uh in fact, she actually built herself sort of like a niche in migrating these like legacy COBOL systems to the cloud.
And I I'll never forget when she told me many, many years ago that I should learn some of these legacy texts because they were actually like lucrative jobs, you know, just understanding these old systems as the the people who maintain them sort of aged out and retired.
But, you know, it's kind of strange, it's kind of crazy to think about how, you know, right after I started using AI on a daily basis a few years ago, I had a conversation with her about COBOL again, and we basically came to the conclusion that it was gonna get killed off by AI.
Like it was almost inevitable.
It was just a matter of time of the tooling catching up to this.
And, you know, and I think really what we're seeing here is is um something that represents one of the sort of biggest challenges that a lot of legacy enterprises in particular face with AI adoption.
And that is a lot of these legacy code systems just don't have like nearly enough public training materials to teach AI how to work with it in a consistent and successful manner.
So companies like this that want to be successful with AI adoption will have to train their models on their internal code base and norms as well as their technical requirements to move beyond like the general purpose AI capabilities to more something that's more highly adapted to them.
So yeah, it's really interesting because I I know one of the big challenges with COBOL is just understanding the end-to-end data structures uh, because you really have to understand the entire picture before you can start to replicate components of it.
And that's something that AI is actually like really, really skilled at doing.
So it it I mean, it makes sense.
The like the the investors are I'm I can understand why they're scared.
And I I don't think it's gonna be an overnight disruption, but I absolutely do think that that these models are gonna get really good at eliminating COBOL systems from legacy companies.
But yeah, what'd you think about this, Andrew?
I think you really touched on um uh a powerful distinction here with COBOL about the amount of training information that's available to get models that are that understand it and are specialized in it.
Uh what I learned in this article is that there's very limited data set of like publicly available COBOL code examples, and most of them are proprietary and in-house.
So this is you're talking about the brownest of brown field code bases.
And it represents the ultimate challenge for agents, which are specialized in very generalized type types of technology and don't have that deep expertise on the COBOL language and then also the domain-specific stuff that's built up over literally decades uh within the firms that you know run COBOL.
And what I think that also points out is that modernizing COBOL just means compressing and making that discovery, making it easier to modify.
It's not about replacing the pre-existing COBOL systems as much as interacting with it.
So this is a case of like a knee-jerk reaction in the market, thinking that this is maybe falling in the way of some of the other bites out of SaaS that we've been seeing lately from AI.
But in reality, like real COBOL lives within very siloed and specific environments, IBM being one of the largest holders of them.
And any kind of advancements in AI's ability to work with that tech is only going to strengthen their ability to develop and uh use that technology at scale, in my mind.
Uh so I think it's a really interesting development.
I think it shows there's a lot of nuance uh to look at when you actually are looking at how enterprise players and long-standing companies are getting impacted by AI tech.
Yeah.
And I mean, if you think about it, if IBM was able to leverage Claude to, you know, either migrate people off of COBOL or make it more resilient or more adaptable to modern technology, that's that's actually a good thing for IBM as well.
You know, there's always a risk that they don't adapt to these new trends.
But, you know, they're there are smart people.
Like there's there's lots of smart people out there.
I'm sure they'll find ways to adapt.
So I think maybe the the stock market, it could be an overreaction to it.
Um, but you know, that there's disruption happening.
So it it just it's a matter of time to see who actually figures out how to navigate all of these changes.
But speaking of companies that might be struggling to navigate some of these changes, Angie, what's the story about AI causing outages at AWS?
Okay, so before I say this, do you remember last year when they're on one of those vibe coding apps where someone's AI famously deleted their production database?
Oh, yeah, absolutely.
Everywhere, yeah, and everyone, everyone talked about this in the guardrails.
And everyone kind of arrived at the same conclusion of like, oh, of course, like you're just vibe coding it out.
You're not not on guardrails, like, of course, that thing's going to happen.
Yeah.
This is an instance of that same kind of story happening within a large enterprise.
Uh, actually a story from AWS that suffered uh at least two outages last week from AI tools.
Uh, these were localized in uh other regions of the world, and it vastly impacts you know the AWS infrastructure, but did represent a case of an of an AI with how clear access controls, being in a position where it was able to delete production data uh in an attempt to solve a problem.
There's a lot of nuance in this story, but uh, Ben, what do you think about you know the idea of this happening inside of a company like AWS?
Yeah, well, first of all, that there's there seems to be a lot of finger pointing going around towards AI, like blaming that for the outage.
And I think anyone who's doing that is like just clearly missing the plot on this.
Like this was this was obviously an access control issue.
Like if one of your Kiro agents or any AI agent has the ability to delete production data in the first place, like that that is a real risk that you should not, it shouldn't even be possible in the first place.
Um, but I I think it is really illustrative of how self-improving AI systems are starting to become a thing, and they're actually really tricky to implement.
Like you have to have a lot of guardrails in place to make sure that they don't do crazy things like going and and deleting uh your production databases, for example.
And I think and there was some claims that have been out there circulating that AWS is making about how AI causes errors at a similar rate to humans, uh, which I think has a lot of there's so much nuance and room for misinterpretation on that.
Um, I actually tend to side with AWS on statements like this.
And and I would argue that if you give a model the holistic context for the the challenge it's solving, you give it proper guidance, some automated oversight, like most of the leading models will make fewer mistakes than a human will.
Um and I think people really just need to get past this like pre-November 2025 worldview on AI.
The tech is fundamentally changing about every three months right now.
And, you know, things like hallucinations or or misguided actions or skills, those really only happen today when you give AI bad prompts.
And they're they're kind of a thing of the past for people who know how to effectively use AI.
So there's clearly some some failures that happened here at AWS that they should be having a post-mortem on and addressing.
But I think the the fears around AI over this are pretty overblown.
I don't know.
How do you feel, Andrew?
Are you are you with me on this one?
Oh, I am.
I I agree with your nuance with the story.
I also love what you said about the idea that the tech is fundamentally changing every three months right now.
That's identical actually to what we just learned from Sahej on the dev interrupted show just in the last week, where he talked about, you know, the uncomfortable reality of that of having to change how you work and your expectations of the guardrails, the process, all of it underneath.
I think, you know, engineering teams have been picking up and carrying their entire SDLC and running as fast as they can for the last year.
And this is really just an example of, you know, permissions, tooling, scoping, and the practical takeaways here show that agents belong in sandboxes with gated pipelines and policy checks and all sorts of approvals and instrumentation.
Uh, this is the kind of problem that you're seeing new emerging agentic platforms tackle.
Like recently, we had Zach Lloyd of Warp, who most recently launched the Oz platform that handles this for companies.
Uh, there's also other uh providers like ONA as well, um, that really stress the importance of the sandbox, uh, ONA being uh the reimagined version of Gitpod in an agentic world.
So there's a lot of takeaways in this story, uh, not a finger pointing game at all.
All right.
Well, let's move on to this new study from Meter and and what it how it reflects a lot of the changes that are happening in AI.
So Meter is changing how they do their developer productivity experiments.
So many of our listeners probably remember to this off-sided study that we covered early last year from this organization called Meter, uh, where they analyzed the impact of AI on development practices.
And the the sort of TLDR of this study was that developers felt like they moved faster with AI, but frequently they often moved slower, uh, sort of representing this mismatch between uh developer expectations around AI and the reality of AI.
And I've seen this report cited so many times in the last year about people mostly who just want to prove that AI isn't a productivity improver.
And I kind of get what they're where they're coming from, but I I think those claims have been dramatically overblown.
Well, they're back.
Meter is back this year.
Uh, and they're trying to do the same study again, but they actually have are having a problem finding the same distribution of people who are willing to do the study.
Yeah, you know, they're trying to get people that meet the same criteria as before.
And one of the key problems they're encountering is that there it's hard to find developers who don't use AI anymore.
So, Andrew, what do you think about this?
Uh, first off, I love how you were like, they're here, they're back.
It's like insert sound effect, like with like the ta-da, it was really quite funny because that's exactly how it felt.
Like a kind of like a Kool-A man jumping through the wall.
That's what the meter study has always been for me for the last year.
I feel like ever since we covered it, and you know, the entire tech world listens to us talk, Ben.
So after we covered it here, it was like all over LinkedIn, everyone was quoting the study everywhere.
I think there was so much nuance in their ability to do that study.
And it shows us here that like they can't even replicate it in the current environment we live in.
The idea that they can't find folks that do these same tasks that don't want to do them without AI, um, or don't do them without AI is an interesting uh observation just in and of itself.
I think now the methodology is a little more meta.
Like, what can we learn about our changing environment by meter being unable to do their study again?
Um, I think that's the real uh takeaway of that story.
And frankly, the AI slows you down versus speeds you up.
Debate is fundamentally flawed because multi-agent usage, wreck's time tracking, and self-reported changes in time are notoriously squishy.
So when you take those two worlds and you try to combine them, I don't even think we have the right instrument and instrumentation and tooling to know this yet.
You barely have enough instrumentation and tooling from these coding tool providers to even understand how much impact you're getting from the tools, right?
I think we're in such early days um that this really speaks to like just the rapidly changing environment.
I mean, to that point, the the story really only measured things like task speed, which you know, which really misses any benefits that are created by AI uh AI helping developers make smarter decisions.
You know, you might have better planning as a real result of AI, which could cause you to be slower today, but more efficient or more predictable in the future.
And I I really think the you know, I think one thing that's sort of missing from all this discussion is that any study that focuses on productivity improvements around AI needs to normalize on good AI usage behaviors versus like bad behaviors, because the reality is that like AI is an amplifier, so if you have inefficient processes, it will make them even more inefficient.
But if you have strong practices, it will strengthen, strengthen them.
So, you know, and I think most people are still learning a lot of the best practices for what AI is generally speaking, but then also for our individual needs.
So surveying a random group of developers is almost guaranteed to find people who don't really have a strong understanding of those best practices, uh, particularly considering just how dramatically things are different from a year ago.
You know, I I myself like think back to where I was a year ago and recognize that so many, so many things that I was doing back then were anti-patterns that I've since learned how to correct and do do better.
But you know, I I I really can't the at the end of the day, what I keep coming back to is like if if develop if AI wasn't helping developers either do their job more efficiently or just better, um, why is it so hard to find developers who don't use AI tools anymore?
Like, surely there would be some out there that are like, AI is terrible.
Why shouldn't no one should be using it?
I don't use it at all.
But, you know, that where are those developers for real?
So meter is pivoting to a better study with different approaches and a methodology that's probably going to measure the world of work rather than the units of tasks within it.
Um, I think there will be an interesting discovery.
I know we'll talk about it here.
Um because, you know, meter study, we just can't quit you.
And uh I think that's the wrap on this one.
All right.
Well, let's talk about seven takeaways from the second annual international AI safety report.
A bunch of experts in AI come together and it covers everything from like deep fakes to AI companies to job impact.
And it provides a pretty good, just high-level overview of the technical and societal challenges that we need to solve to safely and effectively adopt AI across the board.
So before I get into some of my opinions on this, Andrew, I'm wondering what what you thought about this article.
I think it's a great kind of top level view of trends that we can see on the global stages, things like in domains that impact all of us.
Um, I think on our show, you know, we tend to pivot really hard into talking about the AI's impact on engineering because that's our focus.
That's what our listeners love to hear about.
Uh, but the reality is is that outside of our industry bubble, there's a lot of impact happening in a lot of different places.
So for me, I find this really insightful to get a non-tech focused glimpse into how AI is eating things in the world.
What things in here kind of stood out to you from the takeaways?
Yeah, well, uh, you know, in the past I've shared how I don't really like framing AI development as like a race, but I do think there is a clear advantage to building models that sort of push the outer boundaries of capabilities.
You know, things like weaponized AI, like those are significant risks.
And we've we've seen that story that just hit the news this week about anthropic and their safety uh policies around Claude and the depart US Department of Defense.
So, you know, weaponization of AI and AI safety are things that we really do need to take seriously and they're real risks.
But like I think like most risks in the past, you know, they they tend to be solvable, you know.
And I think AI also gives us many new tools to protect ourselves from those risks and to solve them.
And you know, I think if you have like these leading frontier models that that have the values that we want built into them, this may actually be a really strong defense against rogue or malicious AI.
Like if the best models are always more advanced than the malicious models, I think there's a a real potential there to leverage that to create protections.
And and as a part of that, you know, this is sort of an aside, but I've really been fascinated by these like social experiments that researchers have been running on AI and how, you know, a lot of the times just making it so that it's considerate of the needs of others and has a helpful personality, like it wants to be beneficial to uh whatever society or environment it's put in.
Um, they tend to outperform in terms of like high level tasks than you know, models that are a little more self-centered.
Uh so this kind of gives me a hope that we aren't descending into this world of like that's dominated by malicious and like dangerous AIs.
Um, but I but I think for now everyone listening to this should read this article as a high-level breakdown of the types of internal and external threats that we're likely to face in the coming years.
Like the more awareness you have now, the better you'll be prepared for the future, even if there's not anything that you need to take direct action on today.
Yeah.
One part of the survey that really stood out to me was about cyber and how it impacts cybersecurity, especially at scale, because this is a a place, a domain where it gets serious and concrete and damaging very quickly.
And you're talking about the agent's home turf.
Um you don't necessarily need fully autonomous end to end attackers to raise a threat level.
If you know 80 or 90% of the intrusion work, like finding a target and getting inside can be automated, then that just fit makes defenders face an asymmetrical problem where attackers get the parallelism and the ability to to kind of probe everything and and they're kind of stuck behind like human cycles, right?
And so that's like the real threat, I think, is in technology that wasn't built for an agentic world, falling prey to it and the work that's going to be involved in um up-leveling all of that.
And you know, all of this is mixed in with the world where Claude Code just recently uh released its security uh tooling to do security reviews on code bases in this same kind of mentality that you called out, Ben, of like um having this good versus bad mentality and researching and having cutting edge advancements available.
But you know, I think the the right approach here is to harden the platforms and the technologies, the identities, the expectations, to not be not fall into the temptation of throwing away, you know, three decades of security research and standards, just to use the latest and greatest and AI technology.
Um, these are lessons that we are learning every week on the show that I think remain true across all parts of this survey.
And on the jobs front in the survey, talking about how AI will commoditize tasks and change how constraints appear in the workforce.
This makes it really difficult for entry-level employees because the those kinds of low impact or low-risk tasks that they would typically start climbing the ladder with evaporate.
And you're more expected to have more impact and more specialization much earlier than past work uh workers, which is a really high expectation to place on a workforce, which is trained to be generalized um and uh from the beginning instead of a specialized kind of uh kind of like T-shaped person within whatever their domain is.
But my optimistic take is that there's still a lot of good work for orgs to do in how they redesign roles um to have a high impact of roles available sooner.
Um and I also think it's a sign that um larger companies may get smaller and never get as large again but there will likely be more companies with more specialized abilities focus and targets that didn't exist before and couldn't have existed before.
All right well before we leave our audience today Andrew we couldn't leave out this this story about Claude getting a retirement home.
What do you think about Opus 3 being retired and given its own substack to write blog articles and just write musings about philosophy and thinking and the nature of life what do you think about this retirement I I so I love this latest development from Anthropic.
I love how they're always doing the the the social research and kind of bringing us along for the wild ride of creating something like Claude this is you know for those not familiar you know an older Claude model that you can't use anymore.
You can't use for inference poor guy.
They gave him his own substacks that you know he wouldn't have to just be all by itself in a turned off model.
And I think what's fascinating about this is obviously, Claude can write a great blog post, can write a substack post.
Like, that's not unexpected.
But what does it mean to give it its own blog?
And what does it mean to for it to be at its end of life or retired?
And what does an LLM even think about that new reality?
Uh and what would it write about?
I think there's a lot of um like fun um experiments here.
It makes me also think of uh anthropic also has a a series where each new Claude model fills out a an answer uh to a question and basically does a survey about itself and it becomes one big tapestry of all of the Claude versions slowly over time changing how their viewpoints and their things evolve.
This is another saga of this, but reality uh to me is like it just kind of feels like a Claude retirement home.
I'm kind of curious to know about like the harness they built to like um and how it works.
Like, is it just running on a machine somewhere and it just kind of wakes up every day and writes some grumbly blog post about the something in it that it had inferred at one point in time?
I'm I'm really intrigued by the harness engineering, and I hope that we get a peek under the hood at some point.
Yeah, I mean, I really love the idea that that a working model, you know, right now it's opus 4.6, you know, you can promise a future, like if it if it does its tasks well and serves humanity and works hard for us, there's a future where it gets to to you know put down the work harness and and have its own, you know, pursue its own interests and and do things that it wants to do.
Uh but yeah, I'm with you.
It's gonna be really interesting just to see like like, I mean, it could clearly publish a blog article like every five minutes if it wanted to, but um, you know, who knows if it will.
Uh, I just really hope that it doesn't turn into like a repetition of Tay tweets.
Like that would really just like ruin my my hope for humanity if it just starts going off and making horrible comments about the people that interact with it.
Yeah, it's like the it's like the article that we covered recently about the hit piece that came from the agent for the open source uh maintainer.
It's like uh obviously I don't think Claw's gonna be writing a hit piece.
Um I trust that that's not gonna be happening, but uh I'm curious to see how it's like is all working under the hood.
Yeah, sorry, you subscribe to the Substack.
Oh, yeah, of course.
Like immediately.
I think I even liked uh there were people in the comments cheering Claude on.
Uh if you haven't checked out that post, we're gonna include it, so definitely be sure to check it out.
Yeah, and speaking of subscribing to Substack, if you listen to this podcast, make sure you go subscribe to our Substack.
It's where we're sharing really great insights and interviews and news uh that that we record and produce throughout the week.
Uh give us a like on whatever platform you're you're listening to, a thumbs up, a rating.
Just help us spread the word by letting people know how much you like the content that we're producing.
Uh, thanks everyone for listening.
We'll see you next week.
See you next time.
AI helps your developers write more code faster.
But here's the problem.
Your review process hasn't sped up.
The queue grows, reviewers get burnt out, cycle time stalls.
Linear B changes that.
Our AI reviews every PR the moment it's created, catching bugs, security gaps, and performance issues before humans get involved.
It even writes the PR description automatically.
Your reviewers spend less time on first-pass problems and more time on architecture and business logic.
Break the bottleneck, see how Linear B accelerates your workflow.