# Operationalizing AI: From Pilot to Production

**Podcast:** Dev Interrupted
**Published:** 2026-06-30

## Transcript

Today's guest is Nick Sudan, Engineering Operations Lead at Kraken.
And in this session of Linear B's AI enablement interview series, Nick's going to share his insights and structural lessons about scaling AI maturity across engineering teams, including their hands-on experience deploying tools like the Linear B MCP server.
So Nick, thanks so much for joining us today.
Thanks for having me.
It's been really awesome to be here.
Awesome.
Well, I want to go ahead and dive in about...
Something that's really top of mind for a lot of folks that are working with AI tools is they...
start with this phase of just like adoption and then experimentation.
And then ultimately there's a mandate or a need to operationalize it and reach production.
It can be really hard to ship AI powered ideas from the pilot to production in order to have like long-term value.
So like in your experiences at Kraken, like what were the things that broke down during like transitioning from pilots to production and how did y'all overcome being able to operationalize with AI?
Yeah, great question.
So I love shipping stuff really quickly.
And, you know, in this day and age, I love proof of concepts as well.
And everyone at Kraken loves it.
Even before this whole AI renaissance or slopper get in, however you want to phrase it, right?
But we have always been moving really fast, right?
We always value that kind of startup vibe, try and keep teams lean, you know, stuff like that.
And every day we look for ways to execute faster.
And AI has made it extremely easy for everybody, right?
non-engineers which is where this can definitely have a greater effect than how it was before right we don't want to block anyone from building out kind of pilots proof of concepts right so transitioning from pilot proof of concept to production has always been quite tricky and i think today it's even easier now and it breaks down in a few predictable ways so i want to call out some of the problems because i think knowing the problems will help mitigate it but then i'll also give some examples of what we're trying to do to avoid it as well.
So number one, spending too long on your proof of concept, on your pilot, right?
It should be a quick and dirty couple of projects, like quick and dirty thing.
Take you a few days or a week maximum, right?
Get something out the door.
That's why it's called pilot, a proof of concept.
You know, you try to get the point across.
So don't overthink architecture or design.
You know, engineers will spend a lot of time on architecture.
You know, designers, more visual people do design.
Don't, don't, just, just for what is the kind of core essence of what you're trying to get across?
You know, like, you know, we're showing the BlackBerry proof of concept, you know, a dumb phone just to show investors, like that's what you're trying to do, right?
And just because we can build stuff in software doesn't mean that we should, you know, build something that's five times as good.
Still, just keep it simple.
Spending weeks polishing a proof of concept and you, I think you've lost the plot.
So secondly, treat your proof of concept as the foundation for your real thing.
Don't do that.
You know, don't treat it as the foundation.
If you're building a scalable product, you will need proper architecture.
Proof of concept doesn't need any of it, right?
Just goals are completely different.
Promote proof of concept straight to production.
And you're probably going to get a lot of pain later on, right?
So don't be afraid to work or throw away.
code projects yeah totally yeah it's about like showcasing what could be the end result more than about trying to lay every brick of the path to get there 100 yeah and again i think a lot of engineers try to think about this and stuff and i guess the mistake is promoting that to your production product right um and this is where i guess the third thing i would like to bring up which i say is i would say is more of a newer trap so you know now that everybody has ai everyone could build something fast you know you go on x you know someone someone's shooting their startup idea and then a week later they tweet oh sorry they post oh this broke my secret sleep to production so i guess i won't be committing them it's like yeah for us it's like no shit of course but but um you know with ai and building something fast as well there's a temptation to think it you know the speed just the high output scales it consistently all the way to production, right?
So you can iterate quickly, get things in front of people quickly, but making something that's scalable production ready system, you're still going to need to take time with it.
AI can help and will help you make it faster.
But you take out all of the thinking, you get agents building it, it's going to break.
Pressing one button telling, turning on auto modem Claude isn't going to ship you a moneymaker.
ship you something that looks like it does but it doesn't right um so i guess some examples of what we're doing at kraken as well to prevent this as well so testing specifically automated testing as well um and agent-based testing right um and also you don't try to test it just internal testing with with your users the bar for something being done has to be deliberately higher than you know this worked in the demo right especially with ai now as a as a engineer myself i spend less time coding properly now i will prompt stuff i'll have a whole bunch of agents doing stuff and i'll verify it works and then you know send it over but that's you know my my trust in that has dropped significantly so relying on testing and especially you know you run a company with hundreds thousands of engineers you're not going to keep on top of it.
So investing in testing works out a lot there.
And AI, having AI in the loop now, you know, where you have more non-deterministic behavior, right, means passing a unit test isn't the same as like a system, right?
So I think incorporating, you know, a genetic testing that works out well.
Another thing, isolating your proof of concept from production.
Um, we build proof concepts in separate environments to resist that temptation to kind of, right, let's actually turn it on.
So throw away repos, you know, very light scaffolding for deployments and stuff like that.
So people, you know, maybe relaxed kind of repository kind of push rules and stuff.
So people can just iterate quickly without dragging productions, concerns and stuff.
Um, our designers, so, um, our designers are a lot more hands on now with coding and such.
And they're working on kind of building up their vision for certain projects, but they're working in a separate repository.
We have all our mobile app, our web app repositories, but we've given them an environment where they can quickly iterate, get something.
It's removed from the production app.
And then when it's time to productionized, they just hand over the repository, the files, the links engineers.
And that's actually a lot easier to work with than a Figma file.
or a pdf or an image right and and that engineers then okay this is how it looks let's productionize it integrate it properly you know they know how the design system works they know how to build things effectively cache things effectively you know design is a good design so let them work focus on the visual part in that separate environment and i think um lastly as well be explicit about that pilot is right um Because I think many stakeholders, many reviewers aren't clear that this is the proof of concept.
You show it to them and they're going, oh, so can we expect it in production next week?
It's like, no, no, no, no.
This is something different, right?
You need to make it very clear that this is not your final production product.
Value in there is to prove the point, not to ship.
You know, show a working product instead of writing long AI generated briefs, running endless meetings, right?
Showing something is worth a thousand words, especially for people who are really busy all the time, right?
Prove that you can build the thing that they want, the thing that they envision in their heads.
Grants them a lot more confidence and they'll give you really, really good feedback there and then.
Because if you give them documents, if you talk to them endlessly, they'll be like, oh, I'm not really listening.
This sounds all right.
But you show them something, they'll give you feedback immediately, right?
So this is, and just making that very clear for them, right?
You know, I think everyone who's built products, you know, it's like a week before shipping to production, some stakeholder comes in and it goes, actually, this isn't right.
And it's like everyone panics like, oh God, oh no, right?
Like who let them see it before we shipped it?
Yeah, yeah.
But it's like, well, why didn't they see it earlier?
And it's like, yeah, because they've been busy.
And this is where the value of it comes in, right?
Get the feedback there.
Having feedback at the end is expensive, right?
It takes time.
It causes regressions as well because if you're changing massive stuff last minute, the chance of it not working in production, right?
And especially with, you know, with Linear B, you can measure the kind of rework rate and stuff like that.
You know, you want to keep that kind of stable, right?
And to kind of maintain that rework rate, you need to make sure that you kind of get things right at the start as soon as you can, right?
Right.
So just...
kind of making that kind of clear line, this is proving the idea, and then this is how we make it real, right?
That's kind of the ethos around it.
Right.
Yeah, that's kind of my take on that.
So what I hear about some of the shape of how y'all think about AI, here's what I hear.
It's almost like this permeable barrier.
Inside of it is all of this experimentation and rapid iteration.
And inside of this world, you're not creating production-level foundational code for what's going to be what drives the front of the sleigh for the next year.
Instead, you're creating the visions, the interfaces, the hooks, the things that you can show to non-technical stakeholders, to the technical leaders, and to your fellow engineers to all coalesce around what are we trying to ship.
And so you get this rapid...
bubble.
I call it a permeable barrier because ultimately from that, something has to exit and now start to enter, you know, the SDLC, the agentic DLC, whatever you want to label it as for your particular org.
And so, you know, you mentioned, for example, looking at, you know, When engineers are working, keeping the rework rate stable and tracking that through Linear B, for example, helps you keep a handle on overall code and organizational health as things cross out of that barrier.
So that's kind of what I want to zoom in on next and try to understand is when y'all are, as engineers who have built the way to go from that vision, we're all on the same page, and now we're going to lay the foundation brick by brick and get there in a really stable way and ship it.
You know, how do you start to partner with like your tooling to not only all get aligned, but then also like to understand, like, is this working?
And are the practices that are leaving the barrier and starting to enter our ADLC, are they effective?
Like, what are the kinds of ways that y'all look to that and make sure that your engineering pipeline stays healthy?
Everyone at Kraken is using AI right now.
It's whether we like it or not.
It has been a complete game changer when it comes to software engineering.
I guess the myth of like it replacing, you know, software engineering isn't really coming about.
And to be honest, it shouldn't because it is a partner.
It is a tool, right?
Kind of, you know, going from assembly language to kind of typed code to frameworks and all that stuff.
AI is just kind of another level of that on top, but rather than it being very deterministic, it's agentic, right?
And this is kind of how we interface with the modern world right now.
And, you know, everyone at, not just at Kraken, but I think everywhere, right?
They're jumping on AI to win on kind of throughput, on output, 10x their impact.
But the spend is, I think, one of the biggest things here, right?
to the point where it's close to or matching the salary of a developer.
So how, so the question here is, you know, if, if it gets more expensive, um, you know, when, you know, with like, you know, Fable coming out now and, you know, people are being like, oh, I asked it one question and there goes all of my tokens, you know?
stuff like that.
Yeah, I'm currently waiting for a token refresh.
I totally understand.
I have not switched over to Fable yet.
I saw the pop-up and I'm like, I'm going to stay with Opus for now, you know?
You're missing out, Nick.
All right.
Yeah, yeah.
I don't need it to do that for many things, actually.
I guess that's one thing as well, which I'll get into a bit, but just effectively, like, what's the best model for your work as well?
But yeah, cost, whether we like it or not, is the biggest thing about AI impact now and whether...
We get a value out of it because at the end of the day, that's like time is money, right?
Right.
It's like the value of an engineer is like the cost of the tokens that they're consuming.
And so I even think AI is going to get even more expensive over time.
And businesses who aren't thinking about that regarding ROI will be starting to think about it now, this week in particular, right?
Rather than assuming that, you know, today's costs are, this is how it is going to be in the business, you know?
What my take is in regarding how to measure the value of using AI with your SDLC and such.
So I think most leaders actually make one of these two mistakes.
So the first is, you know, AI usage is, uh, or spend, you know, is proof of value, right?
You know, so the assumption is that, you know, cost is high for an individual, for a team, we're getting our money's worth, right?
And then measuring adoption, you know, like how many seats are being filled, percentage of engineers using AI.
adoption dashboards tracking all that stuff they look great but they don't tell you anything about whether the work gets better right um ai usage isn't a metric that is great for effectiveness it's one signal across many but by itself it doesn't prove anything regarding sdlc uh the value of ai so what you would actually want here and what my recommendation is and what we're starting to do at kraken is You have to balance a series of metrics across throughput, quality, stability, right?
So with throughput, merge quest output, frequency, and the maturity and the size of those merge quest, all that you can see within linear B, right?
Quality.
So like, you know, what's the rework rates of the changes?
What about bug escape rates, you know?
How deep are the reviews that actually happen?
And then when it comes to stability, you know, is the thing that people ship to production actually holding up, right?
any incidents, any problems, regressions.
And all of it has to be contextualized as well.
You know, what kind of projects are people working on?
What are they contributing to?
What's the real impact of these projects when it comes to the company and the products, right?
I think for me, the most useful measure is, you know, cost per contribution.
AI spend divided by contribution.
You know, it's a very simple one.
You could compute it as spend per merged, merge request or per engineer, per sprint, or something like that, right?
As people use AI more, their output should increase.
But for generally valuable work, where the cost per contribution stays stable, that's what you're looking for, right?
And then this only works if you weigh by impact as well, right?
If you measure it raw, you're going to reward whoever ships the most trivial load changes, right?
I guess the best way to describe this was in our example is this.
You take two engineers who have the same output, and the same impact.
One spends thousands of dollars and the other hundreds.
Who's the most effective engineer here?
Maybe the one who's burning thousands of tokens has more raw output, right?
But the quality and impact of their changes is poor, then the output is worthless, right?
What we want to have is people using it effectively, you know, using tokens well, knowing which model to reason for, as we were talking about, caching, optimizing the usage with context, knowing when to use deterministic.
kind of thinking versus agentic.
AI isn't a replacement for this.
It is an enhancer.
So we need to make sure that we measure it like that.
And yeah, I don't think people out there are really thinking about this too much.
So that's my take on that.
Andrew and I, we just wrapped up this really great session on token maxing.
And we had a lot of fun running through it.
We got a lot of feedback, a lot of questions from the audience.
And it's very clear that this topic is really hitting a nerve right now.
And I think it really has to do with the fact that executive conversations all over the place right now around AI has really shifted from this, let's get everyone using it to now everyone's wondering how much are we spending?
And is it actually worth what we're spending?
If you missed the live stream, we have the full replay on demand over at LinearBee.io.
And, you know, the reality is that your CFO, they aren't looking at adoption rates or token counts anymore.
They want to see what all of that generated code is actually delivering for the business.
So in this session, we map out exactly where AI is shifting bottlenecks in your pipeline and how the LinearBee's Apex framework helps you measure what is really valuable to your business.
So we'll share a link in the show notes, but you can also head over to LinearBee.
to check out the full session.
And you get like, everyone's over-rotated and over-focused on like, how do we get in the code layer and get in the IDE and help make sure we're steered and we're all aligned on like, we have configs and, you know, an agents.md and best practices, whatever.
Like all of that stuff is super helpful.
But what you've just described is how there's a higher order problem to solve to be effective.
It's your entire engineering process.
need to harness.
In your case, it's like a linear B is that harness because it gives you this through this like view into all of these different health signals and things like quality and throughput across all of the code you're shipping so that when you are, you know, asked or presented and you're looking at the numbers of like your token consumption and are we getting value of this, you can actually.
you have a contextualized narrative for like why you are getting value out of this.
And that kind of like bridges into the next thing that I want to talk about, which is connecting like data silos and actually using that to get really good velocity as a team.
Because it sounds like from what you were describing, you have a lot of signals that come into one place.
And then it helps you understand how effective our coding processes, how effectively are we shipping, but then what's the health of it downstream?
which is a really important problem to always have in perspective.
So like one example is, you know, when we first were chatting about this, you mentioned that like Kraken was using MCP server from Linear B to...
kind of contextualize the information, but also combine it with these other sources and tools to create a really effective things that drove decision-making internally.
So like, how are y'all thinking about using MCP, for example, to distribute this engineering knowledge to folks?
So taking a couple of steps back, this boils down, all boils down, MCP's AI boils down to this one thing, which is the thing that AI is really good at right now.
which is data analysis, right?
So data analysis isn't new.
People are in that field for many years now.
But I think the difference now and the relevancy to the question right now is AI lets anyone be an effective data analyst, right?
It's the fact that, you know, now that everyone has a camera in their pocket, anyone can be a photographer.
You don't need crazy equipment.
Anyone can be a data analyst now, right?
Good data, anyone can analyze it.
they need to be asking the right questions.
You know, just because you have a camera in your pocket doesn't mean a really blurry photo of like, you know, a crooked angle and stuff.
It doesn't make you, it makes you a photographer, but it makes you a horrible one.
Oh yeah.
So it's the same with data analysis as well.
It makes you a photographer, sure.
Yeah, yeah.
So quite a lot, lots of air quotes there for sure.
So step one, at least with all these data silos, the MTP is to...
get the data in front of you and as rich and as sensibly as can be not over enriched because then you're going to be spending more and more with context and stuff as we talked about before but with it at a point where every data point you think might come in handy is there so timestamps you know names of people who reviewed it comments how many comments were left on merge requests you know what files were touched anything that might explain kind of skew or size of the change right So linear B is great at providing the high level stuff here.
And then when you, so when you're using your kind of Git provider here, so GitHub, GitLab, Big Bucket, you know, that's where you can dive in deeper, right?
And this is where the bridge comes in there.
So example here, we were asking probably one of the most common questions that people get.
Why does review time take so long, right?
And tools that linear B help surface it.
And you have to do a bit of digging in to find out why.
But it's not a simple answer when you have, you know, thousands of engineers contributing tons per day across different types of services, types of programming languages, types of like, there's so many factors, right?
And if you just get present, like this needs to improve, you need the data, right?
So review time, in my view, is...
arguably the most important part of cycle time.
And it is the current bottleneck for us and probably the bottleneck for many companies right now.
And the one thing that AI can help out with a lot, because it's a lot more like agentic with the way of thinking there are many different possibilities.
It's not as simple as like, oh, if the deploy time is broke, that means we need to speed up the pipelines or a number of runners available or something like that, right?
We'll reduce the number of dependencies, but review time that it's, it's a whole black box of life.
It could be this, it could be that, right?
A people and process problem rather than a tooling one, right?
Yeah.
So with the structure bit, right, as we were talking about before.
So yeah, use Linear B to aggregate all of that at a pure data level.
So right now you're not using the AI to aggregate or kind of make decisions, but the MCP is really good to get that data out.
The MCP can help answer questions, but...
one data source alone, especially as the NAB is focused more on the high-level stuff, that's the entry point, right?
High-level themes, trends.
And it shouldn't pull in the granular themes at this point because it's going to get expensive.
It's going to get unnecessary.
It gives you the opening path to kind of make that decision, kind of go like, right, this could be an avenue, this could be an avenue, right?
If you dumped every low-level detail in it, it would just muddy the signal as well.
high level trend is very important to first to identify so in this case review time is slow we find out the relevant merge requests the repositories the teams linear b is really good at that right because we like we just thousands and thousands every week at that point then once you've got kind of the high level data then we use our um repository uh our git provider mcp so then and or and also maybe not even mcp at this point because If you plug in a CLI, which I actually prefer to MTPs because they're not as flaky, their authentications are long-lived, and sometimes they even have more capabilities.
But whatever you use here in the agentic interface, this is where you can join, right?
And the two-layer approach works really well.
Linear B high, you know, then your repository provider, the low, and then AI is...
the part that moves fluidly between each other and enriches it, right?
So there's no human stitching it and making the connections itself.
And that's the biggest time set for right now.
And it also pays to be scientific in this rather than just trusting the hunch.
Too many people trust AI and AI is a yes person.
It's never good.
And it's generic as hell.
So you need to be thinking critically.
You need to be challenging it and you need to have hunches, you know?
Our hunch was wrong actually, right?
So first of all, we assumed that big complex merge requests were what was slowing the review time down.
That's what most people say.
You know, linear B gives those signals as well.
You know, review time bad, merge request large, size must be the cause.
But we joined up the data and we looked at it and size and complexity accounted for maybe 5% to 10% to slow down, right?
80, 70 to 80%, something like that came from time zone differences.
A hunch that we didn't even think about too much.
Team silos, reviewer availability, engineers in one region opening a merger post at a time that didn't line up with the code owners for the areas.
And it was really hard to analyze all that kind of stuff.
And who were largely in lovers so that the work just sits waiting for a reviewer who wasn't online, regardless of size, right?
And that's the hidden bottleneck that these silos are masking.
Now that we could prove it, it stopped being a hunch, you know, became a real data backed problem that leadership became aware of.
And the fix we're working on right now is to expand kind of the pool of code owners, you know, so reviews aren't gated to a single time zone.
And, you know, it's, it's a complicated thing, you know, people in process problem, but having the data to prove the hypothesis, like, you know, being scientific about it, that's, that's the thing.
That's the structural shift, you know.
It moves leaders from the anecdote-driven, the evidence-driven, right?
Instead of acting on the feeling and maybe some high-level metrics, or I think, you know, when you're having one-on-ones, if you're like an engineering manager, you might get someone going, oh yeah, no, I've had some slowdowns working with this team or that.
And you might just be like, I'll pencil it in, I'll put it for later, but maybe that's the hunch to start digging a bit deeper, right?
Find out why that's happening, right?
form the hypothesis, test the data actually supports this and be open if it doesn't, right?
So the one thing the AI doesn't do for you is it doesn't decide what you should ask.
That has to come from you.
It presents, it helps with the analysis, but the quality of what you get out of that assessment is bounded by the quality and the context that you give it as well, right?
So that's a little bit of how I use it and how I recommend kind of people use multiple sources as well, you know.
Use Jira, you know, that's really good for knowing the context as well.
If you're using Teams or Slack, you know, get the conversations in as well, because sometimes it's not as simple as, you know, at the end of the day, in engineering, everything comes kind of code change, but there are so many factors as well, right?
This is just a very simple example of that, right?
You know, plug-in schedules, calendars, all sorts of little nuances, right?
There's so many different connections now that are supported with MCPs that you'd be crazy not to use them.
Right.
It's about contextualizing all of the data that's available to you so that you can...
You can take your high-level questions that you yourself as the expert in the domain in which you serve values to your end users can pose these really complex questions that before were to take in a team of data scientists and maybe a few weeks, the crunch for you, that now you can do really quickly by assembling these tools.
And I totally know what you mean about like sometimes MCP can feel like it's in the way.
And for certain workflows, it certainly is.
And for yours, you know, you might benefit more from like a CLI-based approach for a lot of the stuff you do.
But one thing that...
that MCP solves really well, that other toolings and ways of sharing context with AI haven't, is distribution.
Because that MCP server can be really relatively simple to put into the hands of somebody who is less technical and equip them with abilities to take their high-level queries and actually get the crunch of the numbers.
But therein lies the trap that you've very, very smartly identified and stepped around.
One, the AI is a yes man.
And two, you have to have that.
that understanding of what you're doing to ask the right questions that ultimately extract the right stuff you want.
And if you drown it in data, you're not really going to get, you're going to get a facsimile, like a mirrored version of what you're asking instead of like the core truth, right?
So something I want to ask you about is for example, how you equip those stakeholders to be able to understand what's going on in the engineering world through tools like the Linear B MCP server, for example, because you can distribute that or otherwise make it accessible.
So like, For example, you've talked a lot about cycle time in this conversation, about why that's important to you.
How do you use these kinds of tools and distribution of context to get stakeholders that are non-technical in the conversation and really understanding what's going on?
So when it comes to contextualizing, I think these metrics, these kind of things, they shouldn't come top up.
They should come from the ground level and anything that's important.
for an engineering team should be important to that higher level audience.
I think most times this isn't the case because of that contextualization, that translation, you know, into the non-technical terms.
And, you know, this is why MCPs are really good right now.
So to give a bit of an example as to kind of like how we translate stuff at Kraken.
So, you know, we prefer P90 to average, for example, right?
because we want to find out the slowest 10%.
You know, we don't want an average because it flatters us, right?
And if a team is doing well overall, it quietly absorbs the areas that are actually struggling and you never see them.
And P90 surfaces that slow tail, which is exactly what us as an engineering want when we're trying to improve.
We hold a high bar for quality and technical ability.
And P90 is where we want to go and hold everyone rather than hiding behind a comfortable mean.
And so by doing that, it's...
to stakeholders who are higher up, they might go, oh, this is really slow, but it's like, no, no, no, this is what we want to do.
Because a metric shouldn't, like, you know, when a metric starts becoming a target, it's not a target, you know, a good heart's lure in this case, right?
So a metric should be helping you inform.
So the aggregation is one thing.
But then also, it's not just about, you know, a single goal or a company-wide P90 for us.
Different teams work in different ways.
on different services and different repositories with different repositories and sorry, different dependencies.
Right.
And some are slower for entirely legitimate reasons.
Um, so comparing all of these across one global number, ultimately meaningless.
Right.
Um, and this is where kind of this, like, you know, Hey, here's an average for the whole company.
Like that's not how, you know, maybe for a really small team, but if you're.
But once you start to productionize things, once you start to start being a company with hundreds of thousands of engineers, you need to understand the different domain contexts as well, right?
So identify the right development groups up front, isolate them in linear B, you know, sometimes at the team level, sometimes subgroup, like a front end or a back end or mobile, which scope to relevant repositories.
And then treat the industry average as sort of like the default benchmark, you know?
And if a team that comes up red against it, um, we don't just go, oh, so that seems rubbish.
Like look into it because sometimes it might be explained like, Hey, our way of working just, it doesn't, it doesn't work with this stuff.
And that's a fair outcome.
Right.
And that's the context.
And that's what you need to identify for, or the, before you kind of move over to stakeholders, non-technical stakeholders, like you have to do this groundwork to contextualize it.
um you know what is their green line set that goal right so that's the contextualization preamble but then you have to translate it right and this is where you can't just give that to non-technical stakeholders and such um you can't just throw metrics at people and expect them to understand them you know i think many people especially like at c levels and stuff director level get many dashboards which are just what like a bunch of numbers red and green like you know If they don't understand it, they don't trust it, you know, and that's, it's completely worthless.
And if they don't trust it, they won't act on it.
Right.
So, you know, what is the whole point by doing that?
So just for me, describe everything in plain language, cycle time.
I won't assume people know what it is, even if they're an engineer or not.
Right.
Cause cycle time actually, depending on what metrics aggregation do, it could be different.
Some people go to merge, some people to deploy, you know, what, what does it mean?
um describe it in plain language i want uh so i'll say basically look how long it takes us to ship something in front of our clients and link it to things that we care about you know like cycle to a higher cycle at what a low cycle time equals faster project execution speed and it's like okay right right so like you know there's there's the technical term but then there's like the non-technical term and also the non-technical term helps out engineers as well because and translate anything to basic language and help it be contextualized then it you know it all becomes clear right so what does this mean in practice how do you do this actually so you have to make it concrete product design operation leaders this number is high for this team which means this project is going slower like get actually sit maybe in a call or in them with them just walk through them no stupid questions bring it to that table many people might dismiss it as like we don't need to run through these engineering things, but you know, this project is moving slower.
They don't want to hear that.
You need to make sure you translate it into that terms.
And then you say, why as well?
And don't just stop with the diagnosis, right?
Call out specific areas that may need improvement so you can give actions and insights rather than it being red light, right?
And doing this, kind of helping with these leaders, bringing to them, being that translation, kind of Google translate for them, if you will.
unlocks prioritization of like engineering driven initiatives and getting non-technical stakeholders to understand the reasons behind the metrics will actually allow you to get time to work on tech debt, right?
And engineering initiatives, get them onto a rover that's already full of, you know, product initiatives, or we need to build this, we need to build that, you know, every company knows how that is.
So that's how you do that.
If leadership understands what it means and why, you can make the case of carving out that time, right?
And without that transition, They're not going to give, they'll be like, oh, they don't need to work on that.
We need to ship that.
That's going to make money.
But if you can conceptualize it, translate your thing, but this is money, you know, that's it.
That's why product have all their KPIs and stuff.
I know as engineers, we don't, we hate hearing, you know, KPI, OKR, all these acronyms, but you know what?
If we want to get them on board, we have to speak their language, right?
And then underneath all of it, you know, trust and adoption, you know?
Don't compare teams to each other.
Gamifying it.
That's the wrong move.
And that's the first move that many people make, right?
Evaluate each team against their own benchmark.
Not a leaderboard of like best versus worse.
The message to each team is like, hey, this is for you.
These metrics are for you.
You know, if you're green, great.
Nothing to worry about.
And if you're not, it's a signal to kind of step up.
And the right use of the data is like, what's worth learning from it, right?
Is it the internal team?
working under the same constraints which beats the generic industry fix every time like you know just trying to you know make it contextual for each team um and i think this matters most with leadership who are often quick to look at you know who's performing worst and then you know cut them you know that's wrong and you need to make sure that's translated to them as well right because the metric is a metric but then the rationale is to what's happening a team at the bottom is usually one suffering from friction right or they need engineering investment you know and it needs non-technical leadership to give it that time that backing to fix it not the chopping block because if you chop it you're just going to make the problem worse right so it highlights the opportunity because by chopping off the end of it down now there's a new end of the problem because you haven't addressed the core of why that that might be read right for you and like i really it's really brilliant nick how you think about bringing everybody into the conversation, not only like translating all the technical jargon to the engineers, which is like what we're really, or to like the non-technical stakeholders, which is like a really big driving function of Linear B, right?
But also to using it to contextualize those higher level decisions and things like KPIs and OKRs to the engineers and help them map the work that they're doing to that direct impact, to what we're shipping and to why that matters for our organization.
ultimately like by using a platform like Linear B to have an informed, shared conversation, you can build trust because now, like you said, those leaders can trust and understand their dashboard.
They might not be as fluent as you in talking about the healths and the metrics of their conversation, but you all now have like a pigeon language you can operate in.
You have like a way to get each other's point across and then also effectively get the work done.
And for a lot of engineering teams, this is a completely missing ingredient.
It was before, it is now, and now we're trying to go even faster.
And as you've rightly called out, most of the problems we're bumping into are people and process problems.
You need to have everybody equipped with the same language and ability to operate on the same kind of data, right?
So I really love all of these call-outs.
I think it's a really smart way of understanding problems, even just like...
things like using this MCP data layer to curate your own like P90 benchmark layer, right?
So you can understand these specific points because that was a priority for you.
And the beauty of this is it can be arranged to be a microscope onto any kind of problem that you might think exists in your org.
And if you are fluent in the data, then you're not going to have this whole, you know, can I trust the hypothesis situation because it's like grounded in the truth of your engineering org.
It's all really great.
kind of a, I think, like a playbook for how to use Linear B to ship things at scale, but then also have conversations where everybody in that Zoom meeting or whatever, they all understand what we're talking about.
Because for the first time, we're all now fluent in the health of our org.
Really great kind of way of...
putting that all together.
I do want to say, you know, we've covered a lot of ground today.
And just before we wrap up, are there any, you know, last parting thoughts or words that you have for folks that maybe were listening to this conversation and want to have a bite of that success as well?
One word, four letters, data.
You know, it's everything.
If you're not measuring AI effectiveness, adoption, cost, and relating to that as well, you know, the quantitative engineering data, like code throughput, DORA.
then it's all worthless, 100% worthless.
And this has to happen on day one.
Because if you're not doing it already, like, you know, it's going to be incredibly costly later on.
Start it now.
Like, prioritize this.
Like, after listening to this, just do it.
You need to be able to prove your point of data.
And adopting kind of AI, adopting all these tools is not as simple as like, hey, we're going to procure a new code editor or license for some framework.
It's a whole new way of working, right?
And you have to...
You have to measure it like a new way, right?
But it's not just AI specific, right?
It's a chain like data, context, insight, action, right?
That's the layer.
Each layer kind of gets built on it before and most teams actually stop early.
So, you know, raw metrics, step one, data and GIS station.
Good.
You've got it.
You pull most of it where code lives and tools like LinearV help support this, right?
Core metrics, throughput, cycle time, review time, et cetera, you know.
This is part both teams are doing right now.
But the context part, this is the second part derived from the foundation.
You're bringing the context sources that enrich that quantitative data.
Could it be qualitative?
What are people saying?
What projects?
What are the actual projects?
The company's priorities.
What's your expected outcome of all of this?
Financial impact.
Adoption impact.
Metrics will tell you how fast your engine is moving, but context tells you, are we going in the right direction?
And then with that data, and context gives you insight, right?
Someone using AI more doesn't mean they're great.
You know, as we said before, anything heavy usage is a signal to go and look what they're doing, right?
You know, are they actually doing it?
Is everyone using it well and saying it's brilliant, but you don't see a business impact.
Something's going wrong, right?
People hacking away their own little projects to improve their own little processes is fine, but it needs to show results, right?
You know, it's so tempting to work on like...
I'm going to revolutionize how the company works and roll it out to one person, which is yourself.
No, it's not.
You know, if you're not doing that, rethink it 100%.
And then this is where kind of the intentionality matters, right?
Someone using AI models, I mean, they're great.
And I think with that, keep it tight as well.
Leave room for experimentation, but make sure generally, it's generally, you know, pushing your engineering organization forward for finding out what is working.
and what doesn't and back up all of that data, all of that quantitative data with the quantitative data.
Surveys like space surveys are pretty good for this as well.
That's what I recommend.
And there are many other frameworks out there like, you know, DORA is something that's more quantitative and then plenty of reading materials to look at.
The numbers tell you what's happening, but then the people tell you why, right?
So, you know, if I was advising someone, the real kind of final idea, measurement in place day one.
Make sure it spans both the raw metrics and the context that enriches it.
Keep it all intentional and build towards that insight and action rather than just looking at the numbers and stopping at the numbers, right?
The tools then will thrive with all of that.
Well, I think there's been a really great view into how y'all operationalize with AI as well as how you think about working with data.
You know, data is really the king here.
It's going to allow you to operate and soar at this level.
So thanks for breaking it all down for us, Nick.
And we're, you know, it was really great chatting with you.
I hope to have another conversation with you in the future.
And thanks again for joining us today.
Thank you very much.