# Indeed Scales AI Adoption to 97% via Structured Enablement

**Podcast:** Engineering Enablement by DX
**Published:** 2026-06-29

## Transcript

Welcome back to the Engineering Enablement Podcast.
I'm your host, Justin Riok.
This episode was recorded live at DX Annual, where we were joined by Jeff Davis, VP of Engineering, and Michael Redding, Principal Product Manager for Developer Platforms at Indeed, where they're leading AI adoption and developer enablement initiatives across thousands of engineers.
In this session, they break down how Indeed increased AI coding tool adoption from 25% to 97% through structured enablement, training, and community-driven learning.
They also share how their teams reduced coding time by more than 35%, what they learned scaling AI adoption across a global engineering organization, and why enablement, not mandates, became the key to long-term behavior change.
Let's get into it.
Hi, everybody.
Thanks for joining us today.
As Brittany mentioned, my name is Jeff Davis.
I'm a VP of engineering at Indeed.
And today we're going to talk to you about how we used structured in-house training.
to drive adoption and more effective use of AI coding tools among software engineers.
In case you haven't heard of us, at Indeed we help people get jobs, and so if you need a better job or are looking for great candidates for a job, come and see us.
This is us, me and Michael.
Michael is on my team.
He's a principal product manager working on developer platforms for Indeed.
So a quick overview of the agenda.
We're going to talk about kind of where we started, the goals we set, the strategy that we used to get there, and then fill in some detail along the way and then briefly talk about what's coming next.
Okay, so first I want to take you way back in time to the beginning of last year, which is a long time ago in AI terms.
At that time...
end of 2024, January 2025, our DX survey told us that only about half of developer time was going toward new features, innovation, creating new value for the business.
And 48% of developers' time was going toward activities that we would consider to be overhead.
So this is maintenance, upgrades, incident response, administrative overhead.
things that are not the reason they got into the job in the first place probably.
And at that time, end of 2024, beginning of 2025, we had been actively encouraging adoption of AI coding tools among engineers.
We had coding co-pilot, we had unlimited budgets, we had training, we had lots of leadership talk about using AI more.
we were still hovering at around 25% of developers actively using AI tools on a weekly basis.
And so we laid out our AI-powered developer productivity strategy for the next couple of years to try to correct the problems that we saw in the DX survey, to create more engineering throughput, more product velocity.
And so we said, we're going to do two things.
First of all, we're going to use AI to shrink the amount of time that developers spend doing overhead, not innovating, not creating new value.
We're going to automate.
We're going to reduce the workload there.
We're going to try to make the yellow part of the pie.
Yes.
Oh, there we go.
So we're going to try to reduce the size of that yellow piece of the pie.
Here's the real trick.
OK.
And then second, with the fraction of the time the developers have for innovating, creating new features, we think that we can use AI tools to make them more effective.
We think that with AI, they'll be able to generate more output per unit of time.
And so not only are we going to close the Pac-Man's mouth, but we're going to actually make the Pac-Man bigger and thereby double overall engineering product productivity over the next few years.
This presented us with a big challenge.
As I mentioned a minute ago, at that time, we had been pushing AI coding tools and adoption of AI engineering practices.
For almost a year, we were hovering at a frustratingly low...
25-ish percent of people actually using those tools on a weekly basis.
And so the question that we faced was, how can we get everybody to use the tools and to use them well?
So to spoil the ending, we did it.
We got there.
And this is a graph of late 2024 until almost now, the weekly active users for our coding tools.
And we did this through a variety of efforts.
Michael's going to talk more about some of the details around how we did training and enablement and the measurable results that we saw.
And I think it's worth calling out here that most of the adoption that we saw happened before the best models came out.
You know, around Thanksgiving, Opus 4.5, Codex 5.1, those really started to change the game.
But by then we were, you know, above 90% or, you know, somewhere in that range.
And so now I'm going to turn it over to Michael and he's going to talk more about how we got there.
Awesome.
Yeah.
Thanks, Jeff.
So in January of 2025, which feels somehow like it was a decade ago, we were basically here.
Right.
We only had, like Jeff was saying, we only had Kodi and Copilot available for us.
And then we kind of committed, we started hearing from our developers actually that they were really enjoying this new tool called Cursor.
We found people paying for Cursor licenses out of their own pocket, which maybe you shouldn't do in an enterprise organization.
But that was really the vanguard of some of the more agentic.
what we might call second generation tools.
And so when we were looking at this, it's like, OK, our current tool lineup is not working.
We need to add sort of a second layer here that's definitely more agentic in nature.
And so this is sort of our timeline of rolling things out.
And then we actually have phased out Kodi and Copilot.
So sort of that first generation of tools we've actually already fully retired.
Our usage when we actually did retire them was down to about 3% of people.
So our users actually abandoned them before we did.
Mostly where we are right now is in the sort of agentic IDE and agentic CLI tool environment.
Like Jeff said, in 2024, we were at 25% usage.
That was basically entirely codey.
And then now, with just our Gen 2 tools, we're all the way up to around 97%.
And if you're interested, our breakdown, we're mostly cloud code.
a little bit cursor, some windsurf, and then down to the bottom amp.
If you actually add up these percentages, it's definitely more than 100%.
So we also see that our devs are using multiple tools at the same time.
And even more interestingly, within teams themselves, over half of our teams, the makeup of tools being used across people on their teams is actually three or more tools when you look at it at the team level.
So that's some interesting little wrinkles that we weren't necessarily expecting.
But yeah, we're like at 97%.
And you might be asking, how did we get there?
Which is a question we asked ourselves a lot too.
And so what we started with is a program called AI Coding Ambassadors.
Our original thought here was like, let's just do a train-the-trainer model because we have 2,000 some odd engineers at Indeed.
At the time that we were committing to the strategy, we had three people working in our AI tooling space.
So three to 2,000 is...
quite the ratio to try to do really, really in-depth training.
And so, yeah, we were like, let's do a train the trainers.
We'll find a few engineers per team.
We'll work with engineering leads to say, hey, like who on your team is actually the most excited, the most interested in AI?
We were looking at usage patterns already.
We identified folks.
And we basically came up with a cohort of around 60 engineers all across our organization and all over the place in a whole bunch of different locales.
And we basically, did a bunch of hands-on training.
We did webinars, we did workshops, we had peer sharing sessions, we had demos, we had them do some hacking projects and share it back to us.
And, you know, it was pretty, pretty, they were very invested.
They, you know, anybody that was participating in this program, I think we had something like a 90% satisfaction rate with the program.
And so we were like, great, this is, this is, this is a really good model.
When we look at the data from the AI Coding Ambassadors program, we rolled this out from January through February of 2025.
For the blue bar here, that's the coding usage, or what we would call power users, which is a now-retired metric that we don't use anymore.
But at the time, their usage was so much higher than the people who were non-participants, right?
And even within teams, right?
So now the peak bar here is the members of their team, right?
Not them, but the people who are on the team with the people who are the coding ambassadors.
At first, we were like, wow, this train the trainers model really seems to be working.
You know, in that first month, we were seeing near the same usage in our power usage metric.
And then in February, it started dipping, but the top line was dipping too.
So we're like, okay, well, you know, that's fine.
And then in March, it started going.
down again, the wrong direction.
And then we started talking to folks.
And what we realized was while they were going through the training, they were bringing stuff back to their product teams and they were like having those conversations.
But as the program itself was ending, you know, those types of conversations stopped happening on the teams.
But the people who got the training directly, right, they were still at the high level, but everybody else on their team, the rest of their cohort.
they were kind of declining.
And so when we were talking to folks, when we were interviewing, what we realized was the key difference was actually having the hands-on training yourself, right?
Not having somebody tell you, hey, here's this cool training that I took and here's the stuff I learned, but actually having people take that training.
And so we committed to a different program, which we call AI Coding Essentials, because I think for all of us that are practitioners, there's about five vocabulary words that we can use.
Essentials, ambassadors, showcase another fun one that we use a lot uh and basically this one was how do we take the content from the coding ambassadors program that was really really effective but actually structured in such a way that all of our engineers could take it um like i said we have around 2 000 engineers um most of our structured training uh tended to be compliance in nature right so don't click on phishing emails make sure to you know follow our privacy policy.
And so the idea of having sort of an actual like learning, you know, like in our actual like learning environment, our structured learning environment, have a training for engineers, I'm pretty sure this is the first and only one that we've done so far at scale that wasn't something that was being mandated by some compliance regime somewhere.
But we did.
We designed an entire course.
And then we also worked with our...
engineering leads across the organization to make sure that it was prioritized, right?
When you sit down and do the math, 2,000 engineers, eight hours of training over, you know, four to eight hours over four to five weeks, you know, that's about a three to $4 million investment, right?
Just in this program that's not, you know, a JIRA ticket, that's not.
making a code change, you know, it's actual real structured time and making sure that that was like actually prioritized.
And then really focusing on sort of the engagement pattern of, you know, this isn't just a one and done, like let's build the community.
Let's like have people talk about what they're doing.
You know, we're definitely not the experts here.
You know, there are other people that are out there in our engineering orgs that really are pushing the envelope even further than we are.
How do we...
make space to like elevate some of those stories as well right and so i'll pause here i'm not going to read all these words but if you do want to grab the qr code this is our actual playbook uh that we followed uh and this is really i i think the key things here is that you know this was like an all hands-on on deck effort we actually partnered with uh learning architects in our learning uh and development organization uh we hadn't historically worked with them before uh but they were great they were they actually really helped us figure out how to you know, structure our learning, what are the actual things that we're trying to get across, what are our outcomes, what are our, you know, what do we want people to actually take away with?
And they actually helped our engineers that were developing the content, structure the content in such a way that it was actually really effective, right?
So it wasn't just, here's a whole bunch of blog posts out there on the internet to read, here's maybe a couple of YouTube videos, but it really was, okay, so for this piece of content, for this type of message, like it's a hands-on exercise.
So let's...
come up with some way to have a hands-on exercise that you can do.
For this other, you know, for using MCP, which was just coming out as we were developing this, it was like, okay, actually this probably is sort of some blog posts maybe from Anthropix website and maybe some YouTube videos that other people have filmed about MCP and really kind of structuring that multi-modal form of content delivery.
And I do want to zoom in a little bit here on the organizational targets and reporting, right?
You know, we never set a mandate that said you must use AI, right?
We did, however, set a very suggested, you should complete this target for all of our engineers for the training itself.
Because the idea that we had was, hey, we can't force you to use these tools.
But if you do start using them through this training, and you find that you really like them, then you will use them.
So sort of the mandate and the target that we set was really around complete the training, not use the tools.
And so that I think we really do believe that that one little degree of separation, right, of an organizational mandate was a key part of this.
And then I think the other bits to really focus on, too, is like the engagement.
We really didn't want it to be a one and done.
about a month after the actual training program wrapped up we had our fall hackathon the theme for that hackathon was i forget the specific wording but it was basically ai all the things right and so the amount of engagement that we had coming off of this training program into that hackathon having people share what they were building what they were learning and kind of keep that conversation going has been really great I was looking the other day and in our main Slack channel where people share all their tips and tricks, we've had over 100 in March, we had over 100 unique community posts about something that somebody had been experimenting with, right?
So 100 out of 2000, that's a pretty high engagement from like a community management point of view.
So it's really even now, here we are, you know, six months after this training, really encouraging the continual sharing.
During the training, we did see an improvement in our coding time metric.
So when we think about software, like the actual SDLC, some folks might have seen the talk earlier this morning about coding time is actually only about 14, 15% of the actual SDLC.
So for this training, we were basically just talking, how do you get more effective at coding time?
So that was the primary metric that we want to look at for this training.
And so...
This top bar, these are the people who did not complete the training.
So even though it was mandated, we only had about a 70% completion rate.
So about 30% of our devs never finished the training.
Their coding time basically hasn't changed over time.
It's around 20, somewhere between 20 to 25 hours.
For the people who did complete the training, we saw about a 35%, 36% reduction in their coding time.
Right.
And, and because we had around 70% complete, like our overall reduction in coding time was about a 20% decrease.
Right.
And so we were like, wow, this is, you know, this is actually a really good result.
We're really excited about this result, but can we make it sticky?
The short answer is yes.
We did make it sticky because this is those same cohorts today.
Right.
So again, top bar.
the pink bar these are the people who never completed training right the bottom bar these are the people who completed training um at 97 uh ai usage basically everybody is using ai but this top bar these are the people who didn't actually have the hands-on training to get that effective use of training right so they're this cohort here you know they're using ai tools like we know they're using cloud code we know they're using cursor we know they're using windsurf just not as effectively as the cohort that actually did the training, right?
And that's a five-hour coding time, right?
I don't think we were even thinking we could get that.
And as we see this going through, like this is where Opus 4.5, this is where some of the more frontier models are coming in play, where it's like taking the skills around context engineering, around effective context engineering, around effective MCP usage.
combine that with really, really powerful frontier models that came out a couple months after the training completed, and we were able to see that drop.
So I think that was the thing that was really the most surprising for us and kind of fortuitous because we weren't necessarily expecting the stepwise change in model capacity over between October when our training wrapped up and...
basically the first of the year as people were coming back from the holidays.
And yeah, how have we made sure to keep it continuous?
Like I said, community engagement, I think is still really, really important.
Our main AI for developer productivity, Slack channel, has around 2,100 people in it.
Most interestingly, about 200 of those are not software engineers.
They're sort of other R&D roles like product, UX.
program managers.
There's even a few like customer support reps in there.
There's a couple of folks from Kinance in there.
So we're seeing sort of an interesting use of people who are using our channel where we're really sharing our tips and tricks.
And really the tactic there is really around just community management, right?
Almost like social media style community management, like facilitate the conversation.
If people get stuck.
you know, actually help them right then and there, don't make them file a ticket, have people who are working on things share things there publicly.
One thing that we also did in partnership with our main enablement team is we spun up an actual like recognition program.
We called it AI Showcase and it was really about to incentivize the actual experimentation, right?
So a lot of times, you know, it's like the let the thousand flowers bloom.
Right.
And so these were actually in some cases, you know, the winner of our showcase actually there was a monetary prize associated with that.
So the person who had that was judged to have like sort of the best innovative use of AI.
We're actually having an awards ceremony next month for internally.
And we're bringing people from all over the world to Austin to actually participate in that like award ceremony just as a way to kind of reinforce, hey, like this is a learning process that we want people to take.
From a support ecosystem, after the coding forum ended, from the surveys, our post-completion surveys, we had a surprisingly large number of people that didn't want us to stop, right?
Which I've never in my life heard people say, no, I'd like more training, please.
And so we started our monthly coding forum.
That's the fourth of the five words that we are allowed to use.
And these are just webinars on special topics, right?
So when Anthropic released skills.
the skill standard a couple months ago and marketplaces and plugins, that becomes our venue to kind of share more widely those newer patterns into our wider culture.
And we average about 200, 250 people a month tuning into those webinars.
And then we also hold office hours and workshops for individuals and teams that need help, right?
So if you do have a question, what we do see is some folks, I've actually heard this from more than one person, if they're a little too embarrassed to actually ask their question in a public forum, right?
Like having office hours, it's dedicated one-on-one time with an engineer.
Like there actually have been some really, really good discussions and really, really good learnings for us as a platform team coming out of that.
And I've had more than one person tell me that they wouldn't have even asked that question because they didn't want to seem dumb in public.
And in at least one of those cases, it actually was not a dumb in public situation at all.
It was a...
oh no, this is actually a problem with our platform that we fundamentally need to rethink how we're doing something, right?
So it's just like having these multiple touch points that aren't just, here's a training, here's a webinar, but like sort of the individual, making space for that individual one-on-one time.
And then like all of us as new tools, as new techniques come out, make sure that we're constantly bringing in the tooling improvement.
One thing that I forgot to mention earlier, but the training itself, we made sure that it was tool agnostic, right?
So we...
In our agentic space, we use Cloud Code, Cursor, AMP, and Windsurf.
And we made sure that any content that we created, whether it was a video or text or anything, we would always have examples of it in each of those, at least two of the actual tools that were available.
So that way it didn't turn into, here's how you use Cursor, right?
Here's how you use Windsurf.
But it was really about those practices that become transferable across.
I will say the one downside risk of that is when you work with these vendors and they want to offer enablement, they're offering, here's how you use Cursor, here's how you use AMP, here's how you use Windsurf.
And so we have actually not really gone down that path too deeply because we want to make sure that we keep that multi-tool mix going.
And with that, I will turn it back over to Jeff for where we're going next.
Okay, thanks.
Yeah, so just to wrap it up, we have a few follow-on plans for this year.
First of all, we're going to do it again.
We recognize that, you know, you saw the data.
It was, we feel like that program was very successful.
It had a lot of measurable impact on engineering output.
And also, AI changes very quickly.
The landscape has changed a lot in six months since we, you know, put together the curriculum last time.
And so now we're going to do another round of this to help bring the big middle of the bell curve along into full agentic coding and then drive adoption of orchestration as well.
We're also exploring opportunities to use AI more interactively in the training process and actually to have AI as a coach directly in coding sessions with engineers.
When we can spot opportunities where they're doing something that's not optimal, where there are opportunities for them to use the tools more effectively.
And then also exploring different content delivery mechanisms, new kinds of hands-on workshops.
Yeah, yeah.
Just iterating on this delivery mechanism, using our engineers to train the rest of the engineers.
And I think that's it.
I think we're on to Q&A.
the batteries something has failed here but we'll go to q a perfect okay well there's a lot of questions in the chat so that was incredible excited to see where this is going to head so maybe one of the the clarifying questions the audience had is how did you all calculate coding time in the first place was that self-reported data was that telemetry based in some way yeah so so the way that we measure coding time we use jira and so we actually do have a fairly I should say fairly opinionated.
We do have a couple of opinionated workflows in JIRA.
And so what we calculate that as is from when a ticket is picked up by a developer to when the diff is opened in GitLab.
Amazing.
Really creative.
I feel like that's a really creative way.
It's not something that we typically hear.
So that's amazing.
Tell us a little bit more about...
How you're leveraging these AI coding playbooks today?
Are they now a part of onboarding when you guys hire new people or when people switch roles?
Are you using them in more creative ways past just the structured enablement for people who are already in seat?
I will say one of the things that has been really interesting is, you know, last year, I think the theme at Indeed was really around how do we enable and upskill.
developers in the SDLC.
I think our big focus this year is really around the entirety of the product development lifecycle.
So how can we have our product managers and our UX folks get some of the same benefits?
And I think a lot of these principles we're taking into that space as well for our designers, our UX researchers, our program managers, and kind of taking this playbook and kind of trying to adapt it and apply it to what is...
Actually, it's a harder workflow to map because it's not just one workflow in the same way that, you know, pick up a ticket, write a diff, ship it is.
But using the same principles around enablement and figuring out what works and taking it there.
Yeah, I know the audience is looking forward to seeing that QR code for all of those ones too.
So sweet.
Okay, so, you know, outside of all of these things, some of the questions that are coming through.
are tell us more about why you ultimately determined to make some of this training and enablement more asynchronous rather than in person or even mandated.
Some people are wondering if you have a mainly remote culture or, you know, maybe explain a little bit more of your thinking about why these specific elements were the right approach for you.
Yeah, we do have a remote first culture.
We have engineers all over the world and lots in the U.S., but also in Asia and Europe.
So time zones are always a challenge.
Getting together physically in person is not possible, really.
And so, yeah, meeting, you know, kind of meeting people where they are, making the content available for them to consume at a time that works well for them, their team, their workflow, their lives outside of work.
I think that's a lot of how we think about it.
Yeah.
And I think the thing that about the program in particular, we ran it over five weeks.
from late August to the end of September of last year.
And we did make sure that within those time slots a week, we did have sort of open forum type Zoom based, right?
But open forum sessions that were synchronous.
And what we were seeing is that as the weeks went on, the attendance and the synchronous sessions were actually dropping.
And so I actually do think that part of having the sort of hybrid approach of sort of asynchronous content and then synchronous place to ask questions actually did work really well.
for people to kind of feel engaged and to kind of get the understanding of like, oh, this is actually something that Indeed is making the space for us to do, right?
And so I think it's not an either or.
I think it's a yes and.
Yeah, that makes sense.
So something that I know from working with Indeed is how much you all prioritize psychological safety of your team members and making sure that they feel really safe and comfortable about the initiatives that you're rolling out and the metrics that you're tracking and all of these details.
How much did that play into the types of enablement you ran or?
your overall strategy with this in general?
I feel like you kind of spoke to that before.
Like we, we carefully walked this line between actually like mandating tool use and just strongly encouraging, uh, and it ended up settling on, you know, very strongly encouraging people to complete the training, getting a lot of support from engineering leaders in every part of the business for, you know, echoing that message over and over again.
but never actually going so far as to say, like, you're going to start using the tools.
And yeah, I feel like it worked out pretty well.
Yeah, one thing that is interesting in the data, and I don't have the breakout for what it means for coding time, but at least in terms of completion, we did have one engineering organization that took, I want to be careful when I say this, they didn't want to say this is a mandate.
Like, they were very, very lax with like, hey, here's a thing that's being made available.
You should like...
prioritize it maybe.
And that cohort was the cohort that had the lowest completion rate.
So even just the language that you use of sort of where does it fall in that spectrum did make a difference.
So one of our engineering orgs that actually did have sort of the thou must language to complete, I think they have like an 80% completion rate.
Part of this that I kind of glided over is actually having sort of the tracking of completion available.
in a dashboard for managers so that way they could have those one-on-one conversations because like it's part of psychological safety, right?
It's like, how do you make sure that you're creating the space for people to have those conversations in a forum that isn't, you know, public, that isn't, you know, that dashboard, I think we've locked it down so only managers could see it.
You know, we didn't want sort of public shaming, you know, those types of, you know, maybe more negative approaches.
And so I think there's just a variety of tactics to take there, but really making it more about conversation and less top-down you must.
And another thing I think is really important, we continue to talk to engineering leaders across the business about continuing to repeat this message about the importance of upskilling, of adopting new practices.
Even in our most recent DX survey, like a month ago, we were seeing verbatims from engineers saying they just don't have time.
to pick up the new skills, new tools.
And so we find that we have to keep saying it over and over again.
Like, no, you totally do have the time.
We need to carve out the time.
This should be a very high priority.
And in spite of all the efforts that we went through through 2024 and 2025, we're still seeing that reluctance from some people feeling like maybe not entirely safe to invest in their own skill development.
Hello?
Oh, there we go.
Super fascinating stuff about what you guys were talking about with seeing the differences in the data for how teams approach this and the language that they used.
On that note, one of the slides that you showed showed a jump from 25% to 50% adoption rates in the first week of January of 2025.
Any idea of what led to that massive of a jump?
Yeah, we...
It was less visible, but we also saw a little bit of a jump right after the holidays from 2025 to 2026.
Less noticeable because we were already at a high rate.
And our best theory of the case is as people go off for the holidays, they're just trying stuff out on their own time, right?
You know, they've got some downtime.
It's a holiday time.
It's vacation time.
racking up their bill with Anthropic or Google or whoever their provider is for their personal projects.
And then they come back and they're, wow, this is really cool.
We should do this.
And so we saw that in 24 to 25, that jump from 25 into 26.
We saw another jump with our usage around cloud code in particular, which we think is similar effect of people using it on their holiday break.
And so that's our best theory of the case.
I do want to run a study.
just to try to really tease that out because we're also seeing some interesting things from our non-software engineering cohort where people six months ago who were like, I could never use the terminal, you know, no terminal bad, right?
And now some of them are our most prolific cloud code users outside of the software engineering cohort.
So there's more that I would love to learn in that specific, like for that specific tool, but that's our best guess.
Yeah, that makes sense.
We talked about how you saw coding time improve.
Well, really, you saw a reduction in coding time, which we're saying is a success.
How did you think about corresponding metrics to that?
Did you look at metrics around quality or any other signal to validate that the full impact across the SDLC was actually positive?
Yeah.
So we used the core four.
I mean, this is the DX conference.
Got to plug that.
You know, we set a benchmark target for our change failure rate, a do no harm target of about 4%.
We haven't gone over our do no harm target.
When we look at our throughput, our disparate engineer, we've seen that go up as well, right?
So I didn't have that graph here because we can't completely correlate it to just this program, but to all of our enablement efforts.
At the start of 2025, our disparate engineer, I think we jumped 40% year over year from 2025 to 2026.
So we know that was definitely not all of this program, but we...
we're kind of keeping a track of those.
And then even just in terms of like sentiment, which is really hard to measure, but just, you know, one of the things that I'm probably the most proud of, and I probably should have put it myself last self-eval, is like our most curmudgeonly engineer in the entire organization that in the eight years I've ever worked, that I've worked at Indeed, he's never had a single good thing to say about any of our internal tools.
Like wrote a blog post.
It's like, wow, I love Cloud Code.
This is amazing.
This is like, my job is fun again.
And it's like, We did it.
We did it.
We can go home.
We solved it.
And I think it's just that kind of mix of qualitative and quantitative understanding that we really want to see.
Jeff, it sounds like you have to redo a performance evaluation.
Yeah, he already got a pretty good rating, but we'll see.
And then also delivery lead time.
That was another metric that we track and that we've seen a pretty significant.
decline as well over the past year.
Nice.
What about any insights into where this time saved is going?
What are people reallocating their time towards?
I mean, I think first and foremost, we're seeing a 40 to 50% year over year increase in diffs per engineer.
So like they're just doing more, there's just more output.
I don't know.
I don't have a theory about where other time is going to you.
Yeah, that is probably the literal million dollar question.
I do think one of the things that we are starting to see some concerns around is a bottleneck at the actual merger.
So we use GitLab, so MergeRequest, that part of the pipeline.
So we're starting to see, you know, for a very, very long time, we had a very, very tight correlation between MergeRequest opens and MergeRequest resolutions, right?
You know, tracks.
very close to each other.
And I think in the last two to three months, we're starting to see those lines diverge a little bit.
And so we're definitely opening more than we're resolving.
And so that tells us that we're maybe starting to see a bottleneck at a code review that hasn't completely manifested itself in our like wall clock data yet.
But qualitatively, we're starting to hear that more.
And we're starting to get some signals.
So I would not be surprised if that started really turning up in our hard data where folks are spending more time in code review more than anything else right now.
Okay.
Well, you all mentioned, you hinted that what's next on the roadmap for you is redoing an enablement program, but focused on agent-first work.
Tell us about your plans to also go back after the cohort who hasn't yet adopted.
Do you have any initiatives there?
Do you have reasons why you think maybe they still have yet to adopt or what are you thinking there?
Yeah.
So I think the thing that we're the most interested in is that AI is coach, right?
So at, you know, 100%, roughly 100% adoption of these tools, right?
They are using these tools, but maybe not as effectively.
So if we're able to have like the session data that exists and, you know, we are actively exploring DX is new.
a local demon that can get some of those transcripts as well.
We should be able to start nudging those people who are using the tools but didn't complete the training to kind of nudge them in the direction that they would have been nudged by taking the training, right?
And so that's probably the biggest thing.
I think there's probably some other programs that we should do, you know, some research that we probably need to do to understand, well, why didn't you complete?
Some folks say, oh, yeah, we didn't have the time.
But other folks say, oh, I didn't think I needed to learn this.
And so it's kind of teasing that out.
But I think the most concrete thing on our roadmap is that AI is coach.
Nice.
Amazing.
OK, well, any final thoughts, any final advice for the audience here?
It'll be interesting to see what happens over the next year.
You know, I think a lot of what we've a lot of what we have planned or a lot of what we did in 2025 was really reactionary based off of.
We were kind of behind the curve a little bit in terms of where the agentic part of AI was going.
We caught up and then we exceeded it.
I think where frontier models are going, some of the things that we think are bottlenecks today, if mythos is as good as everybody says it will be, some of those bottlenecks may not exist.
And so I think some of our plans around training are basically right now what we know right now today, this is what we're thinking.
Already this morning, Anthropic released Opus 4.7, right?
It's like, okay, well, let's go rerun our benchmark and see what that means for our roadmap.
And that's just that continual need to constantly go back and kind of just see where the actual frontier models are.
Yeah, and another thing I'm thinking about a lot now is I think we're going to move the bottlenecks outside of the SDLC.
And I think we're going to have to shift or increase our focus on training the rest of our R&D functions in the broader product development lifecycle, all the way from customer understanding through an industry kind of analysis through product ideation and vetting and prototyping, and then out to analysis and production.
I think, yeah, we're kind of thinking about the big outer loop as well and how we can apply some of these same techniques across the whole thing.
So many roadmap items that I know everyone was able to gain from that.
So thank you both so much for being here and for all of the lessons shared.
Super excited to have you here with us today.
Thank you so much.
For everyone else in the room, we'll take about a five minute break.