# AI-Driven Software Delivery: Merge Efficiency and Team Dynamics

**Podcast:** Engineering with AI
**Published:** 2026-08-31

## Transcript

So we actually came up with a new metric, which we call merge efficiency.
How many builds on a branch does it take before you can merge something to main?
And the really successful companies are seeing the same sort of merge rate or higher as they get better, right?
And everything is increasing because they're applying AI, they're creating more PRs, they're creating them faster, they're creating even bigger ones, yet still faster and still merging faster.
the sort of median company is creating a lot more builds on a branch because their work is broken and it fails and it fails and it fails and it fails and then they finally get it merged into main or whatever your default branch is that we assume is production.
And your final productivity, you're doing a lot more work.
You're using more tokens and more energy and whatever.
But then the final output is kind of the same.
And so that's a really interesting gap.
We've been trying to help people solve for that gap, right?
Like, how do you get more confidence earlier?
How do you steer your agent, assuming you're using an agent to build?
How do you steer your tools so that you're getting good outcomes earlier?
Rob Zuber is a veteran software leader with over two decades of startup experience, currently serving as the chief technology officer at CircleCI.
He joined the company back in 2014 following CircleCI's acquisition of Distiller.
a continuous integration and deployment platform that he co-founded.
Prior to that, Rob established a strong track record as a serial entrepreneur, co-founding technology companies such as Copias and YouHoot.
Currently, he oversees product innovation and leads CircleCI's globally distributed engineering team.
And today, Rob Zuber is our guest on the Engineering with AI podcast.
Cool.
Rob, thank you so much for coming on the show today.
We really appreciate having you on.
Yeah, thanks for having me, Kyle.
I'm excited to be here.
Yeah, awesome.
Well, I'm really keen to find out more about your specific position at CircleCI.
You know, I'm on teams that use it.
So I'm curious to know what you know about what all's going on.
But let's start with you.
You caught a little bit of a bio that I read back.
What would you add to that?
What do you think about when you think about your career?
What story would you tell?
Yeah.
It's a great question.
I mean, despite being forced to talk about it all the time, I don't think about it that much.
But I think, you know, when one of the things that happens to me a lot is people ask me, you know, what advice do you have for me as sort of someone entering software?
And there's two reasons I have no advice for them.
Or I could say, this is what I did, but, you know, I don't know if that's going to help you.
One, obviously, AI is changing everything.
And I'm just having this conversation with juniors all the time.
Like, I don't even know what you should be thinking about learning.
Like, let's learn together.
which is really interesting and different.
I can't just, you know, my sage advice from 25 years ago is totally useless at this point, right?
And the other is I came into software after graduating college, so I didn't study computer science.
I studied engineering physics for whatever that's worth.
So, I mean, I had a technical background, but sort of it was the late 90s.
It was the original dot-com boom.
I mean, anybody kind of that was interested in finding a job was moving into tech.
But then continued that pattern.
I spent some time doing product management, some time doing business development, some time teaching.
And so I think a lot of that exploration, you know, I always came back to software, but helped me build a really good understanding as an executive of what my partners and my colleagues need from me and a lot more empathy.
Like I think it's pretty classic as a software engineer to be like, what do these salespeople even do?
We build anything.
They just get paid for it.
You know, like this kind of nonsense.
Spend five minutes watching a salesperson do their job.
And you'll be like, I wouldn't have the slightest clue how to make this work.
Like I would be terrified of every single part of this.
And I remember a few moments both.
as an engineer supporting salespeople in the field and then as someone again doing business development and sort of sales related activities, just being like, no, I know what I'm good at and it's not this and I'm so thankful there are people who are good at this, right?
Because then when you go and build companies, like early you have to do all those jobs, which is another great way to get that rounded experience to allow you to become an executive that isn't just like, I know how to write software now, how do I buy a company sort of thing.
But you're so thankful when people show up with like tons of experience doing that job.
You know how it's actually done.
Yeah, exactly.
Like, you know, even at CircleCI where I joined, we were about, I think 14 people or something when three of us joined through that acquisition that you mentioned of Distiller, you know, we were doing sales.
for a long time.
And when the first, you know, proper salespeople came in, I was like, Oh, thank you for being here.
I'm so glad that I don't, you know, like I I'm okay.
I'm not afraid to talk to customers, but yeah, there's just, you know, the process, the steps you go through the repeatability, all of it is just not what, what you do.
Right.
But I do think dabbling, getting some experience, you know, playing a salesperson on TV, whatever, like it gives you enough again.
empathy and then understanding of what those folks need from you.
Because I think technical people really get into like, I explain my technical details and it's very logical.
So I don't understand why you don't understand.
Do you know what I mean?
And learning how other people think and what they're listening for and what they need out of a conversation or about your information.
I think it's really important.
Yeah, for sure.
For sure.
I love that you mentioned that.
I will never forget the day working as, this was when I was at Bindley, a little software as a service company.
We ended up getting acquired by Workday.
And I was sort of on the front line of a team that had, you know, involved somewhat in the sale and was part of the conversations about what we did and didn't have and what they did and didn't need.
And then later you get in there and you realize, oh, there was things you didn't know you needed to tell us.
You know, like they're thinking about the current solution and it pisses me off in the following two ways.
You better have it better than that.
But they don't tell you the hundred ways in which it's already fine, you know.
And so they don't necessarily think to check and you don't think to ask.
And yeah, it's all just like, we're all just sort of crashing into each other in terms of how all this works.
And we're all doing our best.
And I can remember being on the other side when you're not involved with that and you're just like.
Who said yes to this deal?
And you learn some empathy of like, okay, actually, it's not as easy as it sounds.
It feels like you ought to just be able to check all these details, but you kind of can't in the end.
Yeah, it's, I mean, the kind of asymmetry of hindsight, meaning one person looked at it without the hindsight and then another person comes along and looks at it after the fact.
And it's like, it's every engineer walking into someone else's code base.
You know, you don't even have to take it to like selling a deal or whatever.
You're like, why would anyone have ever done this this way?
But someone very rational, right?
Did.
Yeah.
Who would happily look at some of your stuff and be like, what on earth?
Right.
They had a different context.
They had half the information that you now have, all those things.
Right.
And yeah, I think there's a huge amount in there, even within our own domain, but then especially across domains to just say, wow, like there's so many things about this that I don't understand.
And I like how you mentioned AI changing everything.
I mean, that is partially what we're here to talk about.
And it is true, right?
Like this feels like one of the biggest changes in my career.
And it sounds like our careers are about as long.
So let's get into that.
Like when you're using these tools, when you're sitting down to do technical work, if you're keeping your skills sharp or organizing strategy or anything else, like what tools are you reaching for as part of your day-to-day practitioner work these days?
Yeah.
Just before I talk about specific tools, it's worth saying because you kind of said if this or that.
Like I am.
I'm thinking strategically we're a software developer tool company.
In order to do that, I can't not.
For the way my brain works and the way that I think about what our customer needs, I can't not use the tools.
Yeah.
If I'm just reading blog posts and saying this is probably the direction we should go as a company.
I feel disconnected from my people.
I just feel disconnected from my customers.
So I am a very hands-on learner.
And so I'd say the basics, right?
I mean, I spend a decent amount of time with both Claude and ChatGPT.
It kind of depends on the day.
I have funny stories about how they interact with me and their personas that some days I'm just not comfortable with one of them versus the other, but it doesn't matter.
But you're just like, how would I go about this?
You know, what would be, what tools are available to me to try to achieve this thing?
What does good look like in this case?
And I always end up in this weird situation where then, you know, if I'm using one of the more chat tools as opposed to like Codex or Cloudcode, they'll spit out some code and I go, what am I doing with this tool?
I should be, you know, in a terminal right now.
Like if you're going to give me code, let me go to, and then let's say I'm in like Cloud Chat.
Then I'll go to Claude Code and say, your friend Claude Chat told me and then like pastes stuff.
I don't know why.
I feel it's really important.
Like, I haven't been cheating on you, Claude Code.
Like, I'm seeing your brother Claude on the side or whatever.
Like, it's a very weird thing.
And I don't know why.
Like, I try so hard in a way not to personify or anthropomorphize like these tools.
But it's hard.
But they try really hard to.
have me do that.
And so there's like this weird interaction.
Anyway, so, so I, you know, codex and cloud code I use, I wouldn't say interchangeably, but sometimes I feel like I'm going down a weird rabbit hole or something.
I feel like I'm off track and I'll find opening up another one, um, mostly cause it's going to give me a different model.
Right.
And then of course I've dabbled in pie and open weight models and other stuff just to kind of see where the state of those.
Um, cause that matters to me in terms of like.
everybody's talking about token cost management and everybody's talking about like the evolution tools and whatever and optionality.
But I will switch between sometimes just for like a fresh perspective, I guess, right?
Like, hey, I got the, you know, this was built by something else.
What's your take on it?
Is there another look at or, you know, and sometimes I'll do this with the same tool.
Like, okay, cool.
We built that together.
One, we have some internal review tools that we've constructed.
I think everybody's built some review tools at some point, right?
But like now put your adversarial hat on, right?
Like if you were checking for this specifically, do you see, you know, do you see patterns that don't align with our coding standards?
Do you see issues with security?
Like we have a bunch of tooling to kind of force that cycle.
A lot of this is, you know, we're home growing.
and solving our problems in an effort to build things we think will solve our customers' problems.
And so we're, you know, we're trying to get stuff out there, but then we're also trying to push the limits internally and see what works and what doesn't.
But I think that like both fresh perspective, and also sometimes, you know, like the model providers are having so many issues with capacity that sometimes it's just like, who's going to answer my query right now is another driver.
So like we...
As an organization earlier this year, it's so funny, it feels like it was maybe 10 years ago at the rate that things are changing, right?
But earlier this year, we had been in a place where we were encouraging folks to go explore any tools, pick any of the AI tools off the shelf.
And we had subscriptions to a bunch of different ones, like we were doing that evaluation, are we comfortable with their terms of service, that kind of stuff.
But then saying within this pool, because we're a developer tool provider, we want people to have experience with all these tools.
And we found two things.
One, a number of our folks might be trying these things, but not really giving feedback.
So it wasn't actually valuable to us.
So it was better to constrain down to a particular tool set.
And two, while we like to believe that every engineer wants to futz about with all these different tools and is going to be really excited about them and whatever, a lot of people were just like, I have work to get done.
Either tell me how to use it and I'll use it or don't tell me and I won't try any of this stuff because you told me I need to get this thing done by next week or whatever, which is like...
I love this.
You're playing into my next set of questions, right?
We normally sort of open with like what tools you like versus like what are you seeing in the teams you observe?
And it's interesting, Rob.
I mean, you obviously as a tech leader, you've got your own team, right?
That you serve and you've got your own observations there.
But I wonder, like given your role at CircleCI, like you can kind of see what we're all doing.
What insight does that bring you?
Yeah.
And we do.
We have this report that we put out.
We used to put out every year.
I think we still do like the annual version of it, right?
It's called the State of Software Delivery Report where we started from, you know, pre-AI era.
I think we're in year seven or eight or something.
Pre-AI era, you know, Dora metrics.
What can we see about how, you know, PR throughput and like what's actually flowing through the system.
A lot of what's out there is.
survey-driven, self-reported, which is valuable for sure.
And developers' perception of what's happening is often more valuable than what's actually happening, right?
In some cases, you're trying to diagnose.
But people don't have the best memories, right?
I can't remember what I had for lunch.
So like, can I remember how many PRs I got through last week?
Like, not necessarily, but the systems will tell us, right?
So we do this large analysis across our entire customer base.
And we've actually gone to putting out quarterly updates now because the state of software delivery is changing so quickly that comparing year-over-year numbers is just like, wait, what was happening last year?
I don't even remember, right?
So one of the really interesting things that we saw earlier this year and we've been seeing continue, I mean, unsurprisingly, we see significant spreads, right?
Like the difference between median and P95 is shocking.
And in particular, something that shows up in CI numbers that's been quite telling is The difference between increased rate of branch builds, which I'll explain why this matters in a second, and the rate of main builds, right?
So we're seeing significant increase in CI consumption.
Leading companies, so we actually came up with a new metric, which we call merge efficiency.
How many builds on a branch does it take before you can merge something to main?
Okay.
Really successful companies are seeing the same sort of merge rate or higher as they get better, right?
And everything is increasing because they're applying AI, they're creating more PRs, they're creating them faster, they're creating even bigger ones yet still faster and still merging faster.
The sort of median company is creating a lot more builds on a branch.
Because their work is broken and it fails and it fails and it fails and it fails.
And then they finally get it merged into main or whatever your default branch is that we assume is to production.
And your final productivity, you're doing a lot more work.
You're using more tokens and more energy and whatever.
But then the final output is kind of the same.
And so that's a really interesting gap.
We've been trying to help people solve for that gap, right?
Like how do you get more confidence earlier?
How do you steer?
your agent, assuming you're using an agent to build or some version of that, right?
How do you steer your tools so that you're getting good outcomes earlier, less effort, less token spend, et cetera, so that your merge ratio goes back up, right?
It takes me one or two builds to know the thing is good to then merge it, right?
Versus like I'm creating all of this.
It's like all heat, but it's all friction.
So I've dissipated all of that energy.
This is where my physics background, I always like to use physics analogies.
But we had all this energy, and yes, energy was conserved, but it's been dissipated into the environment as heat instead of turning into actual light and output to our customers.
So it's super interesting to see.
And I would say the spreads are getting bigger, not smaller.
Right.
And so if you, if you imagine, and lots of people have used expressions like AI as an amplifier, right?
Like if you have brokenness in your organization, you just see the brokenness in more stark contrast than you did before.
If you made messes before, you're going to make them faster now.
Right.
Really big ones.
And you're still not going to get stuff out.
So like if you were really good, right, if we're all moving along the track and like some of us are slightly ahead or whatever, the really good folks.
are going way faster and the people who were struggling are still struggling.
So their velocity hasn't really changed if you're talking about like actual impact to customers, right?
Like we could debate story points all day long, but like the actual impact of customers that you're delivering isn't really shifting and the folks that really have it dialed or realized how they needed to get things dialed really quickly in order to be able to take advantage of the tools, those folks.
are breaking away from the pack.
So the spread is, is even bigger.
That's really interesting.
I, I wondered if you would say something like, oh, people are like starting stuff and starting stuff and starting stuff and it's never finishing.
That was my own, especially early on, you know, experience was like, you're so good at Greenfield.
I'm just going to Greenfield all the Greenfield.
And, and it's like, wait, wait, wait, I'm not shipping anything.
But, uh, but I, I love what you're talking about.
So, so to your point, um, the better organizations sort of have less time on branch then and fewer builds before it actually gets merged and actually gets shipped and deployed.
And one assumes released versus there's this other sort of set of organizations which, you know, they're spending more time on that feature branch.
And so were they doing that before and now they're spending even longer or did the other ones get faster or both?
Or help me with that part.
Yeah, I think we've seen an increase.
in the kind of not great branch build.
Although there's a secondary, like thinking of my own experience, there's a secondary factor there that I don't know if we've been able to isolate.
So what we're seeing more total builds, right?
Take that more total builds in a leading organization and more of those will be on main, trunk, whatever you happen to call that.
Take a struggling organization and it'll be more just piling up on branches.
Right.
Yeah.
One of the interesting, like, because I personally, uh, am working on, had been working on something, but I was like, you know, I'm, I'm in a bit of a tough spot.
I spend a lot of time in meetings and then I'll be like a big week might come up where I don't have a lot of time.
And so maybe I'm not a great example, but I know this is real for folks where I, I get something to complete.
And then the next day I'm like, oh yeah, maybe, maybe I'll actually get it put up in a PR today or whatever, but let me just rebase.
Because so much activity has happened and it's not just a get rebase anymore because you can do so much work.
It's like, cool, take this intent and reapply it to main so that it's not a massive diversion.
I'm looking for, you know, I'm looking for merge conflicts that aren't just this line was changed in two places, but this no longer makes sense together or another capability was implemented.
that we were leveraging so we have to adjust our implementation right and so sometimes my merge or my my branch build count is high because every day i just rebased in hopes that maybe that day i would get the thing done and actually be able to put it up or maybe i put it up in a pr and it sat there and the different thing got merged and so like that that kind of um but when you rebase you found more changes than you would have a year ago A year ago, there would have been like a couple of new files, but, you know, there wouldn't have been a lot of wholesale change.
But now it's like, nope, it moves.
This is the assumption I'm making.
Yeah, exactly.
Like, yeah.
And this is on, I won't call it Greenfield, but something we're building that's pretty new.
And so we might one morning, like every stand up every morning is like a pretty deep discussion about where's the market and what do we want to do today?
And what did we learn from yesterday?
Not like still working on this ticket.
Like no one's still working on that ticket.
You know what I mean?
It's like, I mean, other than me, because I got distracted by meetings or whatever, but like, it's like, you know, those were standups back in the day.
It's like, well, it's day four.
I'm still working on this ticket.
Maybe I don't know what I'm doing.
Could someone help me?
Right.
And now it's like, well, I did those seven tickets.
Right.
And like, you know, Tuesday we have a chat, we agree on something.
And by Wednesday, we fundamentally re-architected some part of the system.
Yeah.
Right.
And then that thing that we were stringing along, we're like, you know, you're like, which of these two major re-architectures can we land first?
And then we'll rebuild the other one on top of that.
Right.
And that would be so frustrating five years ago.
Yeah.
And now I'm like, hey, buddy, because, you know, I totally have turned my agents into friends now.
Hey, buddy, I hate to tell you, but, you know, Maine has totally changed.
Can you just reapply this?
And buddy, who might be Codex or Claude or whatever that day is like, no problem.
Yeah.
You pay me to do that.
So I'm happy to do it.
Right.
I'm serving the goal of.
whichever foundation shop is getting all the token money right now.
Yeah, I'll have at it.
And so like that, yes, exactly.
You're making like big thematic changes and then reapplying a different goal on top of those thematic changes as opposed to like the little angle brackets to the right and left that say, oh, someone changed this function call.
They added this parameter and you added this parameter.
You're like, cool, I'll put both parameters and find the two call sites.
And now I can merge it, right?
Because that's the amount that someone else could have done in the period where we were sort of conflicting.
And so I think that that whole area of merging.
There's like a hygiene now around you need to get it into Maine faster almost somehow.
So working even smaller, even more atomically is the solution that pops into my head.
I wonder what you think though.
Yeah, I think The reason we're driven is that we can drive bigger changes in a small amount of time, right?
Yeah, right.
If I can write a thousand useful lines of code, like good, productive, useful lines of code in an hour or two, do you want me to ship you 10 of those at a time?
Yeah.
Right?
Like, hey, Kyle, here's another PR.
And you're like, and here's one, and here's one, and here's one.
That wouldn't actually make sense, right?
Okay, I've had two guests say that they do that on purpose.
One said that he's basically always telling Claude, for him, that what we do in this is going to be 10 lines of code or fewer.
That'll be a constraint that he'll give.
And another guest said, for similar reasons, but it's a different approach, was like, okay, tell me what pull requests we're going to do.
So really thinking it ahead of time in terms of...
We're not just going to, this isn't just going to be this giant ball that we're going to plan this out.
But I'm with you.
I think one is sort of like trying to avoid and pretend it's not happening.
And the other is sort of like trying to, well, it is happening, but let's deal with it.
You know, let's plan for this.
But you're right though.
Like the idea that we're all going to be as atomic as we used to be.
Yeah, that was a naive point on my point, I think.
Yeah, sorry, go ahead.
I mean, we're all figuring it out.
I think a little bit depends on your, on your code base, the maturity of that code base, the maturity of your business.
Although I would argue that like, it's not just software development that's been threatened by AI or changed by AI.
Like everyone's business model is immature all of a sudden, right?
Because like, what is my business model in this world?
And so like rate of change, adaptability, all of those things are more important.
And that means, I mean, one of the things that that means to me is we have to not just be throwing stuff out like here's an enormous PR and here's an enormous PR and it seems to work, so let's just keep going because we do have to be capable of changing and adapting, right?
Which is one of the reasons we put a lot of energy into our design, into small incremental change, into making sure that other people understood it.
So then if you go kind of first principles, like those were our goals, right?
But a lot of those were aimed at the human developer.
Right.
Like a book that I read three years ago, four years ago, I can't remember called the art of readable code.
I think that's the name of it.
Right.
Super interesting.
I, I was super passionate about readable code for all of the reasons that I need humans to be able to make safe future changes.
Right.
But I think I'm not sure yet.
We're trying to figure it out.
Readable to a human and readable to an agent are different.
It's going to be a different thing.
Yep.
And so.
The goal was safe, continued safe future changes, therefore adaptability and velocity, right?
And if I assume that my tool is me prompting an agent, then the things that I looked at originally when I saw agents write code or LLMs or whatever, we weren't really like agentic or whatever.
I was like, that's terrible.
And I would delete all their comments.
You know, this doesn't look like the way that I want it, whatever.
And I'm like, but if that makes the agent.
Right.
Work better on the next pass and is going to like lead to higher chances of success.
Why do I care?
I'm not, I'm not the primary sort of like reader and writer of the, of the code at this point.
And I think we're going to continue to, to shift and sort of like an emergent property that we will discover of software is what is software that is highly maintainable.
Yeah.
If maintenance is the right tool, like, you know, can we componentize and then just throw things out and replace them?
You know, whatever that is.
What does it look like for long-term, ongoing, sustainable, high-quality development driven by agents?
And it'll be everything from, yeah, like design, language selection, tooling choices, documentation, repo structure.
Like, I'm sure we're going to have mono-repo, multi-repo debates again.
You know, like, yeah.
How do we enable these tools?
And I think we're still discovering that and we'll see based on who's most successful.
Okay, so Rob, I don't know that you'll have an answer to this, but it occurs to me that you might.
Do you have any sort of like economic indicators around like growth of hiring or shrieking of hiring?
My own anecdata, which is never much to go on, is that most of my guests have reported like, oh, we actually haven't hired anybody.
sense AI, but nobody has talked too much about having laid anybody off.
Now, of course, there are high profile CEOs of Megacorps and you can kind of assume that they would have lots of reasons to do that.
But you have non-anecdata.
You have actual data, maybe?
Can you see much there?
I would say what we look at is well, I mean, we have We have both and we don't always know like when we look at our data, we have to do some like, okay, but who's that company and what do we know about them sort of thing, right?
Like, so you have organizations that are raising enormous amounts of capital and, you know, growing like very investment driven growth and they are hiring.
And then we see a lot of our base staying the course, but growing their delivery velocity, right?
Like they're right.
Again, without judging whether they're getting customer outcomes, we're seeing more builds, more throughput, more creation.
And so a lot of what we think about is in the way that you calculate net dollar retention, I think it's mostly an engineering audience, but so maybe I'll explain it for a second.
But you take a cohort and say from that cohort in the future, what is our revenue based on the people that were here at this time a year ago, right?
It's like net dollar retention.
And so you can think of like net.
pipeline retention as a number almost?
Like, are we growing from the same number of, like, projects and the same number of users sort of thing, right?
So we can see some health in our business independent of the other things we're doing, like we're signing new customers and all that kind of stuff, right?
And we definitely see growth within a cohort despite not a ton of growth.
I mean, again, it's different company by company, but a lot of organizations are pretty stable.
Although we had a really interesting, you know, You can only trust your data so much.
What we see is who's committed.
Yeah.
Right?
So we see more GitHub IDs, let's call it, associated with a project over a period.
Right?
And so we're like, cool, they hired eight humans to do this.
And I was in a conversation recently where we said, oh, you know, we were talking to a customer.
We said, oh, we've done a lot of hiring.
Like, are you doing a lot of hiring?
And they were like, no, we haven't.
And we're like, that's odd.
And what we learned, was everyone in the organization is now committing.
So they're designers and they're PMs and everybody.
So they hadn't hired net new people, but they had fundamentally changed the way they work such that to us, we're like, there's net new contributors and our simple brains are like, there's more engineers in the organization.
But what they explained to us was actually happening.
They had hired some people.
And then they had more people contributing to the project.
That's amazing.
And you hear about that, you know, and you wonder, oh, well, you know, the Anthropic product manager goes on a podcast and says that's happening.
And you're like, cool.
But to your point, you're seeing actual proof of that beyond just, yeah.
Right.
And it's not like necessarily the designer showed up and launched like 50 PRs that week right next to the engineer who put up 50 PRs or whatever the numbers are like these days.
But, you know, we say, oh, we see people.
Of course, you know.
if you model it, you'd say, oh, this is a new engineer.
Therefore, they've only, you know, they made a small amount of contribution or whatever.
But it's like, at some point, we probably could have done this a while ago.
The designer is like, I mean, I don't know if you remember the days, probably, where they would like make a new Photoshop image, right?
And they want you to like get the eyedropper and sample the color and then change the color on the button.
I'm like, could you please just edit the CSS?
Like, why are we doing this?
Right?
Like this whole process is ridiculous.
Stop making me slice up PSDs.
Oh my God, exactly.
98 anymore.
Exactly.
Right?
I mean, like slice it up into little images and try to place them on the screen and whatever.
And so, you know, we absolutely have that internally as well.
Just like, just do it.
Right?
And yeah, I mean, we have the tools, we have the capacity.
So we're, I mean, that's a little off of the hiring.
We are seeing what you see on the news.
You know, huge organizations thinning out sometimes because they truly believe AI is shifting things, sometimes because they're trying to come up with a reason to justify what they're doing.
And then we're seeing, you know, over, sure.
I mean, yeah, all sorts of reasons.
Yeah.
Companies taking off and still hiring you.
Cool.
So, okay, that was really fun.
Normally what we shift to right after, like what tools you reach for is what you're seeing in teams that you observe.
And we kind of had that nicely.
You've got all sorts of stories about sort of like all of our teams.
But let's zoom in a little bit into your own observations a little bit more direct and less just out of the data.
How are you, what are you seeing in terms of how this, well, actually, honestly, you can answer from either position, but we'll go through sort of SDLC left to right.
So what do you think you're seeing change in terms of how we think about requirements, you know?
And in the old days, it might have been epics and stories.
For some organizations, it might have been BRDs, PRDs, whatever.
Are we just having the agents write them now and the agents read them?
Like, what are you seeing change in how all of that works here?
Yeah.
So, I see the whole spectrum.
Even internally, right?
Even internally at CircleCI, I think the jury is very much still out.
And part of that might be, you know, your background and the metaphor that you cling to.
And part of it might be like your particular form of success that you've had with your tools that you're trying.
Right.
And, um, and we see, you know, like spectra and development, that whole sort of umbrella term, right.
Which means different things to different people.
And to your point, like sitting down and hammering something out, maybe with, with Claude or whatever you happen to use.
Right.
can be really effective as a thought partner.
Like I think just saying, hey, I want to build a thing.
Can you write me a spec?
I might as well just say, hey, build me the thing.
Yeah.
Right.
But if I say, hey, OK, cool.
Let's go look at this company.
I think they're very successful.
They're one of our competitors.
Like what's different about what they're doing from what we're doing?
Right.
And what what is no one considered?
Right.
Like I find these tools are really helpful for giving me like.
my blind spots or like things I haven't thought of or whatever.
But I don't tend to find that sitting down and like writing a long spec and trying to line it up and then we're going to do this and then we're going to do this and then we're going to do this.
I mean, I'm like trying to waterfall my agent at that point, right?
Right, right.
That doesn't, that doesn't really work well for me.
I find that I can open up Cloud Code or Codex or like a coding agent, which is pretty much the chat bot plus writing code better.
And by the time I figured out what I want to build, I've built it.
Right.
I was like, oh, if we should do that, then why don't you go ahead and do that?
And I'll fire up another sub agent or whatever.
We'll have a chat about something else.
It's like, cool.
Okay.
Why don't you go do that?
Oh, let's fix it like this.
And so for me, like I'm often reasoning about the thing, if it's new, like if it's pretty novel, I'm reasoning about it as I'm building it.
Instead of reasoning about the document that I'm going to write about how to build it, which then I'm going to feed to some other agents.
So then I'm going to come back and say, oh, you all did it wrong, whatever.
And so I see people having success on both ends.
I lean towards the like, I think my whole career I've thought about software by opening Vim.
I know everyone's going to yell at me and say that I should use Emacs or like get into the 2000s and get an IDE or whatever.
But like opening and reading the code and being like, okay, how does this work?
what if I did it like this, like make little experiments and stuff, just the way that my brain works.
And so for me, I find there's less ceremony up front, more like, and we, um, we're a linear shop.
So we, you know, we use linear tickets and they're like half a sentence.
We got distracted while we were writing the ticket, but someone's like, yeah, I think I was working on something like that, you know?
And, and so because our space is moving also.
Like, and the part that I'm particularly working on with the team that I'm working with, it's moving so quickly that like, let's lay out this plan for the next month or whatever.
Like, it just doesn't make a lot of sense.
It's very, you know, we're thinking it through and we're trying things and whatever.
And so I find that the spec is the functioning code, you know, like, and I find myself quoting the Agile Manifesto a lot, like back in 2000 or 2001, I think it was 2001, right?
When they got written.
you know, there's a statement, I think it's functional software over documents or something like that, right?
Working software over documentation.
Right, right.
Read it more reasonably than I have or just have a better memory.
And so, like, we can do that now, right?
We could do that in minutes.
Like, oh, you want it to work like this here?
Let me just try that real quick.
And okay, let's try it.
Oh, that's weird.
It has this weird behavior.
Well, we hadn't considered that.
And to me, like that, evolutionary, like that process of evolving software is really addictive.
Yeah.
Right.
I work like you.
Oh, go ahead.
For sure.
I can relate to what you're talking about.
Yeah.
All the sort of spectrum and adherence are trying to convince me to give up on my immediacy.
But that's a whole other show.
Yeah.
I do want some things that will take a long time.
Yeah.
Because I have this weird, like someone showed me a chart of this that really resonated.
But basically, like the feedback isn't immediate.
Because the agent goes and does something.
But it's not far enough off that I can really get engaged with something else, right?
So I'm trying to find, okay, can I give you a little, and I have built harnesses that are like goal-seeking, like you'll know you're done when you achieve this.
That to me is, I mean, it's different from the like I write a markdown spec, but I will have an agent build up a harness, right?
Which might just be my test suite in a for loop.
Right.
Like it's a little test driven development.
I think we started calling it harness engineering because we didn't want to have any more debates about test driven development, but like saying, okay, like this is what complete would look like, right?
When you meet these, when this all passes or this all works properly or whatever.
And certainly I'll say things like, okay, what else haven't we thought of?
What would break?
And then Claude will say, oh, this would break and this would break.
I'm like, could you write tests that would prove that doesn't work?
Absolutely.
great.
Now can you make them work?
Like that cycle gets me longer windows.
And when I have longer windows, I open up another terminal and start another agent.
And start another one.
Yep.
100%.
Yep.
Yep.
Cool.
Cool.
Thank you.
So you've talked in there about specification and thinking about planning, and we talked about a little bit into the actual development.
You even touched on testing as well.
Any other changes in either, I mean, obviously the development itself is the thing which has changed the most.
So anything you want to say about that that isn't obvious or anything on the testing side?
Like what are you seeing there as far as change to our industry?
Yeah, so probably two things.
I would say on the development, like I just described designing and interacting with the customer problem as the process of building.
Like I might as well be built, right?
By the time I figured it out.
What's interesting to me about that, and I said this a little bit earlier, is our stand-ups with my team, which ranges from some other very senior engineers down to new grads.
We talk every day about the business, the state of the business, the state of the market, what matters to our customer, so that when those folks leave and spend a day working on something, they can make those decisions as well.
Right?
So we're sort of like...
probably a little weird because they're sitting with the CTO who like knows a lot about the business or whatever, but, but we're distributing the product thinking.
Yeah.
We have a product manager who's helping us as well.
Does a lot of the, like, I went and looked at the metrics and it looks like this is happening and whatever and brings that to our discussions.
Uh, but it's not like a product manager writing tickets and handing them out to people.
Like if you move to this model, which is like, we just, we build and we discover as we go.
the engineers need to really understand the problem we're trying to solve.
We happen to build developer tools, which probably makes it a little easier to have engineers that understand that.
But junior engineers don't necessarily understand the problems of large engineering systems, right?
So we really have to bring those folks along.
And I think it's a worthwhile investment.
I think that's how you build leaders in your organization anyway.
But I think we've had a split over the last, I always quote, 25 years because it's kind of like, I guess I've been doing it longer than that.
Oh my God.
Anyway.
over that time into like really product thinking engineers and then very sort of like ticket taking engineers right and i think the ticket taking is is going to be hard to figure out in this world so i think how we think about what we're going to build it's not just that we don't necessarily write out a long spec it's that that requires an investment in the people participating that they really do have a good understanding and are comfortable just saying like hey yo i actually don't understand Right.
Like I did this and it doesn't seem like it's working or I went and talked to this other engineer and they don't understand what I'm trying to tell them.
I must be missing something like just creating that sort of that free flow.
So then on the on the testing side, like we talked about this, you know, merge efficiency, like am I getting all these failed builds on my branch, et cetera.
It is actually part of what I've been working on with this team is like trying to shift that earlier, meaning putting sort of what I would call CI level.
validation into the hands of the agent.
So as the agent is working, we're saying, oh, your test suite would pass, your test suite would pass.
Actually, your test suite failed right now.
Like, here's some feedback about that.
And then trying to build some tools around that that increase or like check and then can help improve the quality of the testing.
Because I think one of the fears that we have is if my agent writes the code and my agent writes the test, like we've all been in this scenario where the agent's like, well, this test passed, or this test failed, so I deleted it.
And you're like, That does get us to a green build, but it's not necessarily the green build we wanted.
We needed to know that that was failing.
Let's fix the thing.
Also, there's this model from the, I think it was originally created in the 70s, all cool things in software called mutation testing, where you basically manipulate the implementation and then check to see whether your test would catch it.
Make sure it actually fails.
Right.
And it's super expensive.
And probably like these things were invented in the seventies and nobody did them because it was like so much compute power, but like we all have access to ridiculous amounts of compute right now.
Right.
So we were probably doing it by hand.
We're probably going in.
What if I put some paper over this hole in the punch card and I like punch out this hole instead.
So, so yeah, we're, you know, we built some tooling around that.
And trying to push all that to the left and think about what else could we do to improve the quality so that we have really high confidence and we can get that confidence as early as possible.
So really like on the testing front, trying to create, again, improve the quality of the testing and ensure that by the time we get to that place, we're going to run into the merge conflicts and wait for the build time and all that stuff, that it's in really good shape and we have really high confidence.
And now we're just worried about the merge.
As opposed to worried about like, does my thing even pass the local test suite?
Right.
And I think for people with large monoliths, which I think is a lot of people, right?
Even people who have microservices architectures, 90% of their code is still in a giant monolith somewhere.
Those things are really hard to test, right?
And they're really hard to test locally, usually.
And so how do you kind of like build a bunch of different tools to optimize testing?
So like, I guess the other thing we're talking about testing that we're seeing, we're feeling it.
Our customers are feeling it.
is like some fundamental queuing theory, right?
Like your arrival rate of change is escalating to the point where the time spent executing or validating that change, particularly if you have to serialize that, right?
As you get something out to deploy, you're going to reach a point where the arrival, like it's impossible to service the change faster than the arrival rate.
And as a result, you're going to infinitely queue.
Right.
So then you have a real like you have a real fundamental problem, the design of your system.
And so then it's like, OK, how fast can I get that?
Like, how tight can I make that little serialization point?
Right.
And how can I minimize the risk of us of something breaking at that point?
Right.
Like there's techniques that we have already and then other techniques that we're introducing to try to to try to make that work.
And then that's where, when we talked about architecture, like, yes, architecture for agents, for them to be able to reason about a system is important.
But then architecture of the system in terms of how do I get that throughput, right, at some cost optimized approach in a way that I'm not just going to, you know, queue all night and be waiting.
You know, I come in Tuesday morning, I'm still waiting for Monday night's builds to go out so that we can start working again.
Oh, I see.
I see.
Got it, got it, got it.
Because I was about to ask you, like, okay, we talked earlier about how changes tend to be bigger now, but you're talking about optimizing that loop and making sure that it doesn't, but to your point, that's sort of in the overnights then where it's going to the mutation extras and all these other things.
Well, I think it's all, I've met companies where they're building up so much backlog over the course of the day.
I'm going to say they're primarily not CircleCI customers.
They're building up so much backlog over the course of the day.
that they're hoping that when they come back in the morning, yesterday's stuff will be done.
And I'm just talking about like the builds that were piling up waiting to go through, right?
And so, I mean, the goal of CD is the immediacy, right?
Like, oh, we just broke production.
Awesome.
That's because I hit merge four minutes ago, right?
Not three o'clock in the morning, someone's getting paid because their 3 p.m.
thing finally made it to production, right?
So there is some like...
Just how do we, again, how do we optimize this and minimize it?
You're helping me flash back to client sites I got to play with and sort of 2017, 18, where, yeah, like QA wouldn't get something for, you know, like a month because they were so queued up.
And to your point, like all the jobs were waiting on the other jobs in front of them in the, in the, okay, fine.
Yeah, yeah, yeah, yeah, yeah.
Right.
Yes.
Fair points.
Okay, cool.
So yeah, speed matters a lot now.
Yeah.
Yeah, yeah, for sure.
Yeah.
Otherwise, that queue just grows even faster.
Hmm.
What do you think change in terms of infrastructure and how we think about how we ship and those kinds of things?
Yeah.
AI begets AI.
That's one thing I would say.
As we move faster with less, I'm going to say less comprehensive understanding of every detail, right?
We've got the same number of humans with the same brains processing way more change, right?
So the likelihood is higher that we put something into a production environment that we don't fully understand.
And we are building all the tools that I mean, we are building it for ourselves.
But like as a software industry, there's a ton of investment going into the tools to make that as safe as possible, right?
And in many cases, it's going to be safer than what we did before.
Like, you know, we could spend a sidebar on humans reviewing PRs.
Like, were you going to take attention while you're reviewing that?
I know.
So there's that, but then you, so you get something in production.
I mean, like telemetry, observability, like pick your favorite word is going to be critical, right?
Because, and what I will say, thinking about debugging production systems is I know people, I know engineers who are not great at debugging the thing they just wrote.
And I know engineers who are great at debugging a system they've never seen before.
we're going to be probably more in that latter category, right?
Like this notion that we can't put something into production that we don't understand.
Like I've watched people parachute into someone else's environment and be like, it's right there.
Yeah.
That's the problem.
And that takes good tooling, right?
Like either they work really hard and it's slow to get to that understanding or they have really good tooling.
And specifically, I'm thinking about like traces and understanding like how is stuff flowing through the system.
And here's the full stack trace when something goes wrong.
And here's the source that I can reference back to so I can go look at it.
And now when I said AI begets AI, like I'm going to use, first of all, to find that in the first place, right?
Debugging production systems, particularly distributed systems, is an exercise in pattern matching and correlation across enormous data sets.
Yeah.
You know what's really good at that?
Is AI.
Like, way better than humans.
Right?
Like, you may not have written it, but just saying, this all appears to be, this is related to that, and this is related to that.
Like, that was my sound of a computer anyway.
It was like my modem from when I was a kid.
Anyway, like, that connection exists, and this is likely this.
And I went through this code, right?
Like, I have access to all the repos, and I have access to all this data.
That will be a remarkable improvement.
I think people have nibbled at the edges of that so far, but I think we're going to see more of that.
And there's going to be a greater need, I think, for the humans to really be able to do that large-scale thinking.
It'll be tool-assisted, right?
But, you know, systems will be complex.
They'll be fresh.
You won't actually know what the structure of all of it is.
But, I mean, I've...
been doing this a lot.
I've been to a lot of incidents where I had nothing to do with the code that was acting badly.
And I still was able to figure out what was happening.
And I know lots of people like that.
Right.
So I think, I mean, well, we responded to a Linux CVE recently that we were trying to, how does this impact us?
And we got four people on a call reading the Linux kernel, very clawed assisted.
Right.
But, and then building, you know, tests to validate, you know, whether or not it impacts us kind of thing.
And like, that's, that's going to be the world that we're in.
Right.
And that's like, that was not the Linux kernel wasn't written by AI, but it also wasn't written by us.
Right.
Right.
And so like a very, that's, it's not a distributed system, but a very large scale application that's extremely complex where I can be guided to a point and then make rational decisions about how that impacts my system.
Right.
I think is, is going to be a real part of the job.
Yeah.
And so I think being really comfortable leveraging the tools that we have, But also, like, I don't, I'm sad to say I don't know how to get people to be great debuggers.
It's not something I've spent a lot of time on, but I think about it.
No, yeah, yeah.
As I, you know, I've spent time in incidents like anybody else that runs a distributed system.
And I was working with one group at some point and I was like, I want you all to spend the next week playing Wordle.
And then I want you to tell me at the end, like, how you thought about reducing the problem space.
Right?
Like, how do you take, because I've got the alphabet and I've got every word that's ever existed, every five-letter word that exists, right?
I think it's five letters.
I haven't played Wordle in a while.
It is.
It is five.
And I've got a fixed number of guesses, right?
I mean, anyone that's listening to this doesn't know how to play Wordle, just look it up.
Like, I don't know why I'm explaining it.
But like, I have this broad problem space and I use techniques to analyze, to probe the space, to try to determine likelihoods, right?
of where I'm going.
And of course the game is kind of designed to drive you through that.
Right.
But that was like part of the, and so I think like that kind of reasoning, there's also this other thing that I'm totally obsessed with.
I'm so sorry.
This is totally off the rails, but it's, oh my God, I'm going to forget the name of it.
I think it's called the Matson consensus protocol and it's used in wilderness search and rescue where, so someone is missing or an aircraft is, I think it came from aircraft originally.
And Someone is missing or an airplane is missing or whatever.
And you get together a bunch of experts and they all get points to allocate to search areas based on their expertise and understanding of likelihood.
And that's how you decide with a limited search party where you should go look.
Right.
And one of the things I love about software is we always have to invent everything for ourselves.
Like there are so many industries and disciplines where people have solved these things.
Right.
And so this notion that like we have a bunch of experts on a call or whatever, like talking about production debugging.
Sorry, this is all built from the assumption that AI is going to make a mistake and we're going to put something in production.
We're going to try like having tool and you could automate that too, right?
Like having tools and techniques to say, to drive towards the likely cause of the problem instead of, I don't know, AI built all of this.
Let's start at line one.
And read through to line 50 million and maybe somewhere in there we'll see something that's wrong.
And so I think we'll be forced to build better tooling, better approaches, better practices.
I assume this will all show up in software because that's how we solve things.
I like how you're talking about.
We talked earlier about how you were saying that we all need to have more of a product lead these days.
And lots of guests have talked about, yeah, we're all closer to the product now.
That shows up in my life.
I advise people, talk to the AI about the domain.
If you're talking to it about the curly braces and the semicolons, you're missing the point.
Tell it about the business problem you're trying to solve, right?
And it will be better at helping you solve that business problem.
So I think there was a whole category of software developers who really wanted to make sure that each line was perfectly idiomatic and that we...
I'm not sure that skill set is as important as it used to be.
But to your point, this I can debug something, skill set is still quite valuable, especially in these complex distributed systems that we've created.
So I think for a lot of us, like we're just going to ask God to do it, you know.
It's funny when I'm doing that with my own sort of smaller systems, this is part of my solo work, like I will...
It's not that I'm racing it, but I will get it kicked off and say, start looking at this production issue, please, right away.
And then I'll go do the same thing.
And it's not so much that I think I'm going to win, but that I might spot something that I can share with it, you know, to help it along.
And every now and again, I win, but it's pretty rare.
Usually, Claude gets there first.
But I think the point you're making is a different one, though, which is that, like, the...
If you've tapped out the agent's ability to figure out what's going on, then that other skill set of being the human who can jump on that, you know, severity one call and like spot something that might help also continues to be an important skill set, you know, that maybe the idiomatic code guy or developer is less useful than they used to be.
But being able to spot the issue in a large distributed system needle in a haystack is still quite valuable.
Yeah.
Yeah, I mean, I would describe it as systems thinking.
Yeah.
Right.
Like understanding this thing over here impacts this thing over here.
Right.
And you could allow your agent to build up that context and like we'll probably build more permanent memory and context around these systems so that it isn't, hey, could you read all this code right now?
Like you know nothing about our code base and start from scratch and here's the error kind of thing.
Like I think that's a thing we'll advance.
But I also think that the fun and terrifying thing about prompting.
is we pass on our human biases, right?
Like we had an incident recently where a couple of us were trying to use some production debugging agent tools.
And after the incident, in our review of like, what could we have done better?
We both kind of independently went and retried knowing what had happened, right?
Now we know, so it's a little bit of a cheat, but like, what could I have asked that would have got me there sooner?
And you could see the questions we initially asked.
We're so burdened with bias that we sent our agents off on wild goose chases instead of giving them the freedom to be like, yeah, I don't know.
I think it might be over here.
You know what I mean?
Because it was like, you know, what is causing X to happen?
But X was a symptom, not a cause sort of thing.
Right.
Right.
And and so I think it takes both.
Right.
I want I need to guide the agent in the right direction.
Or spin up a bunch.
I mean, in that whole sort of like search protocol idea.
It does.
The thing is, I kind of have unbounded agents if I want them, assuming my foundation model is actually up in servicing requests right now.
And so if it's their incident, they probably, I wonder if they use other people's models.
Let's not talk about that.
Okay, so.
It's turtles all the way down.
Yeah, yeah, exactly.
So they have like some, someone has a like.
open source model under their desk that they just point all that at.
So you have kind of this unlimited capacity and you can, but knowing how to seed not random, but multi-path investigation, like that's what a good IC does, right?
You go look at this, you go look at that.
In a less well-managed incident, 12 people are looking at the same thing.
When the real problem is somewhere else, right?
It's funny when you say that, what occurs to me is like, yeah, I feel like it goes better if I describe to Claude inevitably, in my world anyway, what I'm observing and not try to, you know, just raw observations.
Like even something as dumb as it's down can be really helpful.
I mean, because it'll just go start getting its own observations, you know, and it might be faster at it than I am.
So I will have had an observation that pulled me to it.
So maybe it's like an AWS CloudWatch alert or something.
I might copy the text of that and just paste it in and say, go.
And I don't bother to do much theorizing at that stage.
But I also want to, there's something you said about how, like, oh, Kyle, that's just systems thinking.
And I think you're right, but I have known people that I thought of as good systems thinkers forward in terms of designing a system, but weren't necessarily the person you would call when it all breaks, right?
I think there were other people that were really good at like gisting a situation very quickly.
And they might also have been good forward systems thinkers, who knows?
But sometimes people who I think of as very good forward systems thinkers were not good at the quick gist.
So yeah, I mean, you're right.
It is a kind of systems thinking, but I still think that there's like specializations on it, I guess, if that makes any sense.
Yeah.
Yeah.
I think, yeah, diagnosing is different.
than um than building a system um and i i think i don't know yeah the the other big thing that i think about there in there is is this pattern matching and maybe that's not quite systems thing and it's like yeah it's like thinking fast and slow we run into the like we've seen this before therefore it must be this that's the worst kind of pattern matching right right as opposed to that thing there feels like it might be related to this thing over here even though no one's seeing that and it's it's right That story about the firefighter who runs into the building and just says, get out.
And can't really explain why or how he knows, but just get out.
Yeah.
And that is useful, right?
And sometimes thinking fast is not what you need.
It's thinking slow that you need.
But yeah.
Right.
We are really solving the world's problems now, Rob.
Sorry.
Well, I mean, I do think someone is going to build a really good version of this, right?
Which is like, you know, I said, well, that's a good IC.
But do I, like, being a good IC, like, there are these things.
I always joke about this or at least joke about it recently, like in the late 90s.
I think it was the late 90s.
We were really into this concept of expert systems, which is like we're going to take all the world's knowledge and build it into computers and then we can just ask it and it'll solve anything.
And we obviously gave up like so many other things, that and the unified email inbox.
And like in 2026.
We're accidentally building those things, right?
We're taking all of our knowledge.
And every time we have a conversation, we're like, cool, write a markdown file, write a markdown file.
We just have these enormous markdown files.
But we can give useful guidance and direction to an LLM and have it execute on something based on our learned experience, right?
Like, here's how you review code.
We talked about everyone building their own code reviewers because we've done the same review over and over.
Like the way that we built ours was we just...
pulled from all of our existing repos.
We said, go read all the reviews and find the authors that comment the most and that people trust and then figure out what they say over and over and over and just say it yourself.
Right?
Because every senior engineer is like, why am I telling you the same thing again today?
We talked about it yesterday.
So let's not bother giving that broken thing to the senior engineer to say again, like, we talked about this, do it this way, right?
Like, we can encode all that stuff.
And like, when It became really easy and the return on investment for that engineer became really clear.
Like, wow, I can stop having this conversation because like, you know, cute little Claude told you not to do that.
And like, you listen to Claude, great.
Like I can move on and do higher value things.
And I think, you know, these things like being a great incident commander, I keep saying IC, but incident commander is what I mean, where you, where it's.
encoded knowledge that you've built through reps and then you try to train up a new person, right?
And like, I think, I think we could build the tooling with all of the knowledge that we have to guide you through that process.
And it would be some combination of like, I fed it, you know, pager duty or incident IOs, like how to be an incident commander training guide.
And then I fed it all of our historical incident reports.
And then I fed it access to Honeycomb, Datadog, Grafana, like whatever your tool is.
All the things.
And now it's like paging the right person at the right, oh, this looks like it's a problem over here.
Like, you know, who's the expert on that?
Because I read all of the Git history, right?
And I know that this is the person who knows that.
Yeah.
Let's call them, right?
And I don't know.
I mean, someone's probably saying that already, but like, I think that.
It's probably happening.
I'm going to get right now, in fact.
That might not be the first.
Yeah, whatever.
Vibe coding doesn't, I don't even know what that means anymore.
But like huge amounts of data to process, right?
Like this is such a fantastic application of this tooling.
And I think it will force us to have to build it and it will make it possible to build it.
And I think that's like a weird interplay of a lot of scenarios with AI.
Yeah.
Right?
Now it's important, but now it's possible.
And we can afford it.
Like you were talking about mutation testing earlier.
It's like we couldn't afford it before and we maybe didn't need it.
We do now.
Right.
Yep.
And suddenly we can afford it, which is great.
Okay.
One more question on what you're observing, and then we need to sort of start, you know, Wendy down the hill.
You've been very generous with your time so far, Rob.
Thank you.
But one more question.
We've talked a little bit about industry trends and what you're seeing and what you can grok about what's happening with hiring and economics and those kinds of things.
And you talked a little bit about how, you know, everybody's committing now.
really, really good, insightful stuff.
Anything else that you're seeing around, you know, how teams are built, right?
I think especially in the consultancy world that I've lived in for a lot of my career, we get really used to like, okay, you're going to need, you know, sort of call it six software developers and a project manager and a business analyst, you know, especially sort of pre.
Eric Ries, you know, lean thinking.
Like maybe after that it would have been a product manager and not so much a business analyst.
But you got used to certain shapes of teams, right?
And of those six developers, like you needed a couple of front-end and a couple of back-end.
They all really needed to be full-stack, but you still needed some specialists, you know.
And maybe depending on the domain, you needed something else and something else.
But, you know, there was sort of a shape that you could almost just stamp out, you know.
Are you seeing that change any?
Yeah.
Yes, but I don't know what the destination is.
I just know it doesn't feel right anymore.
We already talked about giving more product thinking to the engineers, and maybe that was a path we were on anyway, but this is more clear.
I think if you think about the cycle, the life cycle of let's call it a feature, doesn't really matter, pick big project, small project, and the time horizons to execute on a thing, call that unit of work, you know, X, X amount of work.
The handoffs between multiple parties within that X amount of work might account for 10%, right?
Maybe 20.
I mean, I'm being really generous.
It's probably like 80, but, and then take X and make it achievable in one 10th of the time.
Or like one one hundredth of the time.
If that what was 20% time spent on handoffs would now become 200%, right?
Like we could probably use real numbers and model this out, but it's not that important.
200% of your time spent handing things off is completely unacceptable, right?
So you want, in my mind, you want to take more of the roles and capabilities of the team and consolidate them.
into fewer actors.
Now that doesn't mean like all PMs don't have jobs now, right?
Like people bring strengths to teams in ways that others are not capable of picking up, right?
And sort of saying, well, everyone's just an engineer and they'll just do all the other stuff, right?
Like I've designed apps before and been like laughed out of the room.
So like someone needs to do some design thinking, right?
And so.
But how we structure that, like what is the ratio, right?
Like it's like we kind of need less because the engineers are doing a lot themselves, but we need more because they can get so much done.
Like I think that's the part that we can't quite figure out where it's going to be something that will evolve and we'll see emerge, right?
And different teams will be different.
If you are a startup of four people, like that was always just the engineers doing it and you accepted that their design was crap.
Or they just lifted it off some site.
I mean, now they're using Claude Design or whatever to make it happen.
And so, like, everyone has to do more yet less.
We can all get more done.
Everyone's tools are assisted, right?
Like, the PM can generate, you know, perspective.
They can analyze the data.
The designer can create designs.
They can do all these things also augmented and faster.
And so, I think then I do think overall team size.
is going to trend smaller.
That's my gut.
Yeah.
Purely based on just communication overhead.
Right.
Like, yeah, right.
Every, you know, whatever, every node I add is, I forget, it's like factorial or whatever.
Yeah, exactly.
So we want to reduce that.
Yeah.
Which then means, you know, principles like team topologies and how do I create independence between teams so they can execute, you know, on their own value stream or whatever terminology you want to use.
I think that becomes important, but then it's like, I can literally touch every system in five minutes.
I spent four weeks working on something in a programming language that I still don't understand because I know enough about programming that I could tell roughly if it's right, but I couldn't sit down and handwrite that because I've never taken the time to learn that particular language, but it doesn't.
And the last 30% doesn't matter as much as it used to because machines are maintaining it now, not us.
Yeah.
I mean, I know if it's well tested.
I know if it does what I wanted it to do.
I have security scanner.
I have all this other automation to make sure that I'm not making massive blunders.
I have other people.
We have all these review tools we built.
We have the machinery to make that happen in a way that then allows me to be...
you know, to move independently across a system as opposed to like, I'm tucked in my little corner and I just do this thing.
So it feels like the greatest non-answer, like it's absolutely changing, but like you can see factors that are pushing in both directions.
And maybe we end up after all of that push and pull with the exact same team that just functions really differently.
Yeah.
I don't know.
I really think though the, you know, I am the one person who does this part.
I think that is going to be less.
I think we're going to see less of that.
Yeah.
Right.
A hundred percent.
And if you're.
Are we all generalizing more again?
Interesting.
It's a re-generalization almost.
Yeah.
Go ahead.
Yeah.
I mean, well, one, we already talked about the fact that like knowing the intricacies of the curly braces, right?
Like I remember my one college course on programming, like.
I was like, why?
Why is there a semicolon on this line and not this line?
Like someone make that make sense to me, right?
Like no kid is worried about that anymore.
This is not a thing, right?
Like it just doesn't happen.
And so like that specific component is dialed back in terms of the role of an engineer, right?
So the engineer has a lot more capacity.
It might just be I'm going to build more, but it might be I'm going to think more strategically and I'm going to have like these questions about the customer.
I'm going to spend that time thinking about product.
Something else.
Which maybe up levels the PM or the PM can support more teams or the PM writes some code.
And a lot of that is going to be individual aptitudes and skill sets, right?
Rewards.
Yeah.
I think, you know, it's interesting because you talked about like stamping out the teams, right?
Well, we know it's exactly looks like this, but.
I've always been, I don't know about skeptical, but curious about that because on any given team, like sure, we could have two front end developers on this team and two on this team and two on this team.
But this one over here is basically a designer.
This one over here is great at getting the pixels that the designer told them to get into place.
So like they kind of look the same, but they don't.
And like, you know, pick your, I watch a lot of soccer in my family.
It's World Cup summer.
So maybe that's topical.
NFL football starting or maybe CFL is already underway.
I don't know if you're CFL.
Pick your favorite sport.
Okay, you got a basketball behind you.
But like, right?
Well, okay.
So I live in the Bay Area.
I guess we kind of invented small ball, right?
Like everybody had a giant center.
And then one day we're just like, you know what?
What if we didn't?
Right?
Like what if the structure of our team, right, was such that that isn't actually the best way to play.
Right.
Because we, we, these are individual humans with individual interactions, the way they interact with other humans, the skills that they bring to the table, you know, whatever that might be.
And so like that, like, how do we take full advantage of that?
Right.
And then that becomes a thing.
And suddenly everyone in the NBA is thrown threes, but like that.
is someone right in their particular place, stopping to say like, what are the strengths that I have at the table right here?
And so I think particularly as we go through this evolution, I'm just like, I don't know, it could be anything.
Like we have multiple different teams at CircleCI and they're all shaped differently.
Differently, for sure.
No, because I don't mean to overstate that, but it does feel like that, like you said, that whatever we were doing before is not quite right.
So maybe my preamble there was a bit of an oversimplification.
But the basis of the question I think is still interesting.
It's super interesting because I think what we love in engineering is cookie cutter patterns.
We're like, this is the job of a PM.
And you're like, okay, but that PM is not very good at that part of the job.
This team member who's like a junior engineer is actually really great at that thing.
Could they do that instead of the PM doing it?
And I think it's, it takes a, it's taken me a long time.
I would say in my, in my career to get to the point where I'm like, yeah, I'm like, I'm a CTO and other people are CTOs and they do the job away and I do the job differently.
A different way.
And that's not because one of us is a bad CTO, but like at some point you, you realize you're a member of a team and ultimately the goal is the success of the team, not your success.
Right.
And so, Other people on my executive team bring strengths to the table that I don't have and vice versa.
And my role might look a little bit different than the way someone else does it, right?
As a CTO, like a very classic distinction would be someone who can't shut up, right?
Versus someone who like never wants to talk in front of other humans.
Yeah.
Those are two totally valid and useful and contributing CTOs, right?
And then you just ask the question like, okay, in our organization, who fills in this role and who fills in this role?
So I think...
you know, if anyone were to take anything about out of this ramble about teams, like now is a great time to really assess who on your team brings what to the table.
And that may be across multiple teams in your organization.
And how can you build a team that's going to be successful given the constraints of this environment?
Right.
I don't take my like, it's going to be two of these and one of these and one of these.
Like, don't know.
Yeah.
So we actually have three questions left, believe it or not.
And we've come to the end of our time also.
So I'm going to, I'll call an audible and just ask one of them.
What's something that's giving you joy right now?
That's giving me joy.
Okay.
This is, I know you're from Toronto, so hopefully you'll appreciate this.
This is totally random.
But Rush, the band from Toronto, right?
Is on tour.
I have to confess I have not gone to see them, but I've seen like every YouTube video.
And for anyone who doesn't know the band, they have this very famous drummer, Neil Peart, who like changed drumming, right?
For the better.
And he passed away in, I think, 2016, 2016, 2017, somewhere there.
So they're back on tour.
Actually, it was later, my bad.
But whatever, you can look it up.
They hadn't toured since 2016.
And then they found a new drummer, Anika Niles, I think is how you pronounce her name, and went out on tour.
And the fan, it's like the hardest job in the world is to fill in the seat of this man, Neil.
And she is absolutely crushing it.
And the Rush fan base absolutely loves her.
Oh, nice.
And it is like, when I first started seeing the videos rolling, because it shows up in my YouTube feed or whatever, I was like, finally something just wholesome and happy in the world right now.
There's so much angst out there.
And like, there's like.
on the internet, in like YouTube comments, like it's so easy to be a bad person.
And be dark.
And the amount of joy generated by something that could have easily been a very, very bad outcome, right?
There's all kinds of sort of skepticism and whatever.
Sure.
And it is like, they're obviously having a huge amount of fun.
The fan base is having a lot of fun.
And, you know, when you ask, the way that you ask the question just landed, like it's lately I've been saying, there's like, The one thing that's giving me great joy right now is just that people are so happy about this situation.
So yeah, it's awesome.
I love it.
Okay, one more thing.
You mentioned earlier the name of the newsletter.
So let's hear it one more time and I'll make sure it's in the show notes.
Oh, the State of Software Delivery Report is a thing that we publish and then we have a newsletter actually called The Confident Commit and a podcast called The Confident Commit.
Confident Commit is like an umbrella for some of the other material that we put out.
So you can go for that too.
Cool.
Rob, thank you so much for coming on the show today.
Really appreciate you.
Yeah, you bet, Kyle.
It's been really fun.