# Mercari's AI-Native Transformation: Measurement, Platform, and Culture

**Podcast:** Engineering Enablement by DX
**Published:** 2026-06-15

## Transcript

Welcome back to the Engineering Enablement Podcast.
I'm your host, Justin Riach.
This episode was recorded live at DX Annual, where we were joined by Michael Galloway, Head of Platform Engineering, and Snehal Shin, Head of Cost and Performance Engineering at Mercari, Japan's largest marketplace platform that serves more than 23 million monthly active users.
Michael and Snehal will share Mercari's journey towards becoming an AI-native company, sharing what happened after leadership mandated 100% AI adoption across the organization.
They break down how their teams build visibility into AI usage and engineering workflows, where AI actually improved developer productivity, and where it exposed deeper organizational bottlenecks across planning, review, CI, and deployment systems.
Let's get into it.
So I'm Michael Galloway.
I run the platform engineering organization at Mercari.
Hello, I'm Sneha Shinde, and I run cost and performance engineering at Mercari.
All right, Sneha, you want to go ahead and kick it off?
Yes.
All right.
So AI adoption is happening everywhere, every single field.
But adoption alone isn't the finish line.
So in this room, how many people have struggled to draw a straight line between AI adoption and ROI?
Quick raise of hands.
Nice.
Almost everybody.
We believe that the challenge isn't just about building AI or AI adoption.
It's trying to find the value in a measurable, repeatable, and visible way.
All right.
So as we are in the middle of a major shift that is happening in industry where we are augmenting software development in a very rapid way, and it's starting to reshape the role even much more faster.
So how many think that the error of just being a software engineer will go away?
Quick raise of hand.
Wow.
That's interesting.
So what's interesting is not the answer.
I feel currently the real struggle is that the role is evolving much more faster than the organizations defining what a good software engineer will look like.
So a quick introduction about Mercari.
So Mercari is one of the biggest, largest C2C marketplace in Japan.
So we have around 23 million monthly active users and growing at a very rapid pace.
And more importantly, one in five people in Japan uses Mercuri.
So at this scale, customer experience and infrastructure matters a lot to us.
However, visibility is very, very important to maintain something like that.
So to build that kind of visibility, data has been like in our DNA at Mercuri.
So whether it's for customers, we do a lot of APE experiments to check like what is working for the user and what is not.
And similarly, we do a lot of things around engineering to track KPIs and build DX dashboards because we want to measure and we want to learn from the failure and we want to improve our product and also the DX experience at Mercuri.
So a quick credibility break.
The Japan CTO Association runs an annual developer experience one, and McCartney was first in 2023.
It was first in 2024.
It was second in 2025.
So when we say that we have been working on DX, that is, you know, building, you know, sorry, measuring what matters.
building the right platforms and defining the clear, what do you say, golden paths.
We are very serious about it.
And we feel pretty good about all this stuff that we did in the past.
But here is an interesting path.
Even with all that, when AI hit us, none of the dashboard could answer the question we needed them to answer.
So about 10 months ago, Our CEO made it very clear.
Mercurial is going to be AI native.
This means we have to push for 100% AI adoption and no exception in any of the roles.
And that was the mandate.
You know, Japan works in a very different way.
Because I know, like some of the earlier speakers said that, they are encouraging AI in their company, but Japan is very different in terms of culture.
We had to move fast for this.
So what we did is we pulled together like 40 engineers and formed a dedicated AI task force under our CTO.
And the focus was just about like building, integrating, and unblocking.
At the same time, something is wrong with the slideshow.
At the same time, we identified like roughly 100 people.
within the company, and we call them AI champions.
And these people came from every single department within Mercury.
Legal, finance, CS, engineering product.
And the goal was very simple, but very ambitious.
The goal was human decide what matters and AI handles the execution for it.
But...
We didn't just want to add AI on top of our existing process.
We wanted to rethink even our workflows on how to make it much more better and make it much more faster.
So we did all this stuff and we run many, many steps, but we underestimated the effort required to do something like this.
Because at Mercurial, We have roughly 2,000 employees, and everybody had a different technical skill or their understanding of technology.
Like the person from engineering might be able to really install cloud code in five minutes, but can a CS person do that?
Or can a finance person do that?
Maybe not.
They are very comfortable with Gemini or GPT.
For us, it wasn't about like tool change, like enterprise software that you can roll out to everybody and they know what to do with it.
For us, it required a fundamental shift in culture and mindset across the entire company.
So to make it easier for us to pivot in the right direction, the leadership set a new mandate, which was rethink everything.
So what we build, are we even building the right thing for our customers?
Because with AI, now when you have 10 ideas, you can put those 10 ideas in one month.
But in the past, we were only able to roll out maybe one idea in that time duration.
How we build, like do the process really need to take weeks for future development?
Like you design something from design to ticket, from ticket to code, from code to review, from review to release.
Does it really need to take one week?
And even how people work, like not who is building it, but how well they are operating in their day-to-day life.
So now the challenge with saying rethink everything is it can be...
as so big that you end up doing nothing because you are chasing behind a perfect workflow or a perfect solution.
So what we did is we made one rule.
If you don't know how to measure, you don't start.
So within like few months of going all in, you know, something which we were observing was that our existing dashboards the ones that won so many awards, like every single year you were in top three, could not answer a single question that leadership kept on asking us, which was, is AI actually working and improving our productivity?
So the things that we saw, which was happening during that time, was local optimization wasn't translating to system optimization.
Engineers said they are fast.
Teams said they are fast.
But when we look at the company's end-to-end cycle, nothing has moved.
Our metrics were per team, they were per repo, maybe per pipeline, and they were not stitched together across the value chain.
So we couldn't see where we are getting stuck in the entire pipeline.
And we had no visibility around our AI tools.
So none of our AI tools that we use, like Corsair, DevIn, Cloud Code, none of the data was flowing into our DX dashboard.
And, you know, the code that is sent there was like a wakening call for us because we realized like we didn't know.
how to identify our gaps.
And basically, this was our first big dip.
So within a few months, my team built what we just call like DX visibility right now.
So at the bottom layer, we had data sources like cursor, dev in cloud code, which we were getting from AI tools.
And then we had the key SDLC system like GitHub, Datadog, and Jira.
Most companies job.
when they see, oh, the number of lines of code generated by AI is increasing, this means we are more productive.
But as you know, like Mercari love data, we wanted to go beyond that.
So at the middle layer, what we did is we built external system connector and we stitched together like certain metrics like PR cycle time, cross-platform PRs and ticket closure rates.
And this was the hard part.
But This part made a lot of things for us possible because on the top layer, we were able to build dashboards which were focusing on AI adoption, like usage by team, by role.
We were surprised with some of these numbers, which Michael will share later.
Flow and throughput, like PR throughput, deployment frequency, lead time, the Dora-style stuff.
and quality and risk like incident, MTTR.
The one thing that we did was we also measured Gen AI code acceptance rate.
Like when the code was generating, like how many people were clicking accept and pushing it to the, like pushing it into the code base.
So why these?
Because the question AI is working is very ambiguous and it's very hard to answer.
So you require multiple facets to find that information so you can answer those questions much more honestly.
So once we could see a little bit clearly, some pattern kept up showing.
The first one was awareness gap, like the biggest single finding that we realized.
So we were giving people like, you have access to GPT, you have access to cloud code, but they...
didn't know how to use some of them.
They were maybe using them for menial tasks, but they were not leveraging enough that can provide them with the productivity gain.
And some of the pocket of knowledge that were not getting shared enough within the company.
The second one was team boundaries.
AI didn't magically dissolve our charts.
It ran into the exact same problem.
Like when you're working in a bigger organization, You have a PR that you raise, and you have to make changes in three different code bases.
You require approvals.
And AI did the same.
When it was inside a service, it was really, really awesome.
But once it started making those changes, it ran into the same problems that humans do.
Third was the system complexity.
Because AI gave us more entry points, more ways to generate code.
It also had more ways to break things.
The blast radius of like an LLM did it for me was much more bigger than an intern did it for me.
The quotes at the bottom are like real engineer feedback that we got during this adoption phase.
We didn't sanitize them because Michael promised you to.
The one that really haunts me is like the last one.
Debugging AI outputs cost more time than we save.
And it was really, really true till last year, until they did the model upgrades.
So I'm thankful for that.
So that means like AI is costing more than we are benefiting from it at certain stage.
And we really cannot fix that because if you say like this is not working, you cannot say him like take a new license for another Gen AI tool and make it possible.
So this is the complexity.
OK, I'm going to go through big numbers here because Michael promised truth.
So we are going to drill down into complexity because it's really important.
I want to show you the baseline where we started.
This is what the dashboards were telling us in late 2025, before pre-AI DLC, P75 timings, and a 90-day window.
Plan, we didn't measure it.
We were not measuring how much time we took for planning a feature.
Code was within weeks.
One of the interesting facts was our local dev experience was our lowest rank.
in the engineering survey, like we didn't have a good, what do you say, local dev experience review.
I think this was our first hotspot.
So some number here, P75, time to fist approval, was five hours.
The mean was 20 hours.
On the big repos, like the legacy monolith, I think people who are working in big corps, they know when we say legacy monolith.
It's the code base that nobody wanted to touch.
So here, our PR cycle time median was 3.7 days, and P75 was 7.1 days.
That means half of our legacy monoliths PR took a work week to just get deployed in-depth.
CI was our second hotspot, and here the problem wasn't runtime.
It was the failure rate.
iOS on-pull requests was 14 minutes, 55 seconds with a 35% failure rate.
Android was 16 minutes with 34% failure rate.
And web took six minutes with 26% failure rate.
Our backend CI was an average of six minutes, P95, and had 12% failure rate.
In every one of these failure is a contact switch.
cost for an engineer.
Deploy infrastructure changes took an average of 8 minutes 49 seconds with 19% failure rates.
Monitoring our aggregated SLO for our legacy monolith was sitting at 19%.
And there were invisible bottlenecks.
And the biggest one was our review queue.
This was the biggest drag on the company.
It was not coding.
It was not CI.
It was waiting for another human to look at your code, look at the diff and approve it.
CI fragility.
It's not the slowness of the CI.
It's the flakiness.
And flakiness was worse than slowness because the frequent contact switching and a single threaded behavior was really, really dangerous for our engineer.
Support load, like these are real members, okay guys?
So platform absorbed like 766 requests in a quarter.
And every single of this request was stopping someone from delivering something meaningful in any of our environments.
So When we get to the second dip later, Michael is going to share some amazing stuff because he's going to thread together that AI didn't create these problems.
It didn't create these problems.
AI just made it impossible to ignore.
All right.
And this was a brutal reality.
AI as is doesn't fix a broken system.
If your foundation is brittle, Your pipelines are fragile and there are gates everywhere, security gate, compliance gate, legal gate, something that you added, you don't know why did you add it, but it's there and it's like a tribal knowledge and it's sitting there.
You have some of these unowned services.
What AI does, it will just go ahead and break it faster.
This leverage cuts both ways.
Now I'll pass it to Michael.
So he can share about how we fixed some of these things.
Some of them.
All right.
Thank you, Sneha.
So first, let's talk about the awareness gap that Sneha brought up, because I think that this is a topic that was discussed earlier on the main stage.
And I think it's a really key one to focus on first.
So what we did to go after this awareness gap was we set up several different parallel efforts.
The first one.
was really focused around culture.
We accepted that this was a community problem.
It wasn't a tooling problem.
So we kicked off AI Jamborees, which were essentially company-wide demo days where we had 200 people per session and anyone could show something that they built, really showcasing what's possible.
We set up pizza demo days.
And if anybody wants a piece of advice that hasn't done a pizza demo day, you should do pizza demo days.
People show up when you give them free food.
So it's a very good idea.
It was low stakes lunch and learns.
And it was driven by ICs and for ICs.
So it wasn't something really formal.
We did AI open doors where there was drop-in office hours with our AI task force.
So you could just bring a problem and ask for some feedback and questions.
And then we did, and this is actually something that Tim mentioned.
And I want to highlight because I was very proud that we did it.
We did these role specific workshops.
So we focused on different kinds of roles in the company that were non-engineering roles.
So finance, legal, design, ops.
This is actually something that we'll talk about in a few minutes, but it was a very big win for us.
The second thing we did is we went after friction.
Most of the time we found, at least within Mercari, we made it hard for people or very difficult for them to understand.
how to even get access to the tools or what the process was, or they had to go through security reviews or get manager approvals, then you're going to shut down the whole program.
And so we actually focused on that directly, and we made an AI bot in Slack that you could basically request and say, I want cursor.
It would ask you a few questions, and you'd confirm that, yes.
And not only did it provision cursor for you, but it also wired up all the security and other configuration we needed so that it was ready to go for you.
So that was actually a tiny little investment that produced an outsized outcome.
And one of the reasons why it did that is it set that understanding.
It communicated implicitly that AI was important to us and that we wanted everybody to try this.
The task force also, and it's marked as an early win, but I want to just highlight a few things around the efforts here.
Task Force went through and they mapped 33 domains inside of Mercari in 100 days.
And that was that what we mean by domains is these are functional areas where we could either automate AI or where we couldn't and what the leverage points were.
This is really important because it helped us understand where the low hanging fruit opportunities were.
And it was far beyond engineering.
As a result of this, other parts of the organization that those role-based workshops started kicking off efforts.
And so our help desk, our internal help desk around support built support bots effectively and started to pull the information in and it resulted in a 60% reduction in help desk workloads.
Actually in platform engineering, we also started experimenting with this and was able to get about a 25% decrease in our support burden for our individual contributors.
inside a platform.
So this is a very effective and early way to start leveraging AI and expose people to the benefits of it.
We also saw this one here was particularly interesting.
It's 45.7% drop in accounting tasks.
So our finance and accounting organization, again, built support tooling and everything like that so that engineers or other people in the company could ask accounting questions, usually things like month-end close questions, looking up policies.
And that kind of material, I mean, on the policies alone, again, as Sneha highlighted, we're a Japanese company, so you can imagine how many policies there are.
And they are in both, in varying different languages and detail.
But this was a really big win for us as well.
This was another thing that was not only just a toil reduction for our business, but it was also really creating that reinforcing.
mechanism where it really exhibits, hey, we're all trying to leverage this new capability in different ways.
And so it added to that kind of culture.
And then this last one here is we built an internal, yeah, it doesn't really tell you very much.
It's a BI agent.
We gather a lot of data, as you can imagine, and that data goes into BigQuery and querying on that data can be very complicated.
And then even making sense of that information that you query can be very difficult.
So internally, we built an agent that sits on top of that called Socrates.
And there's a lot more to it than I'm going to just detail right now.
But effectively, what this enabled people to do is ask Socrates for information as it related to our internal customer usage, or you could poke at different dimensions.
of information.
And what this did is, is it actually helped a lot of people that had business intelligent related questions.
But I think what was more compelling was, is we saw a lot of people that never thought about asking those questions, now starting to ask those questions because we had suddenly made it much easier and democratized that information.
This was a big win for us because it also caused that movement towards another term that I'm sure everybody is familiar with.
that product engineer kind of mindset, right?
If you want people to actually invest in building and involving your product and not just solving technical problems, you need to give them access to the information that will help them ask interesting questions.
And this was one of the things that we did that actually unlocked that.
So we actually have 500 plus weekly users.
It's actually growing quite a lot.
We're not a gigantic company, so this is actually quite a lot of usage for us.
It's about one in every four people in the company are actively using this.
Okay, so where's the time going now?
So over the last two quarters, we've invested a lot in the entire traditional SDLC, and I'll talk a little bit more about that.
We weren't measuring planning before, as Sneha pointed out.
We actually did an internal survey and started to really dig into our planning process and how long it takes.
It's marked as plus visibility because now we understand it better, but we haven't done enough, I think, yet to improve on that, although we are moving towards some ideas around that.
For the code, I mean, this is no surprise.
Everybody knows the coding time has shrunk quite a lot.
So it's moved from weeks to days, and we'll talk a little bit more about that.
I think I want to highlight something here that everybody else is probably very aware of as well.
This is largely gains you see from Greenfield situations, not as much Brownfield.
And I think that's something we'll talk a little bit more about.
Greenfield has skyrocketed.
Everybody's very happy.
They show their demos of something that's nonconsequential, and everybody gets excited.
Oh, my gosh, how fast it goes.
Review.
So we've implemented code reviews with agents.
It's saying in flight because there's varying levels of quality with code reviews with agents.
I've been taking a look at a lot of statistics to try to reconcile some of the things that I've been trying to make sense of about code reviewing.
Generally, what I saw is there's more reviews happening, but I didn't see any kind of change really in the time to approval, which was interesting on its own.
So we're still trying to kind of...
dig into that.
I have a lot of theories on why that is.
One real quick side note, there was a friend of mine from another company.
Can I mention the name?
Is it okay to mention the name?
Luis.
Oh, no, no, no, no, not Luis.
I was going to mention, okay, Airbnb.
Okay, they're here, so I'll just mention Airbnb.
So apparently their way of handling some of the reviews.
Is anybody from Airbnb here?
All right.
OK, so on some of the code reviews, the way that they have sped things up is they'll put a comment because they require several approvals on the code review.
And so people will put a comment, TBR.
Any guess what TBR stands for?
Yeah, OK, you guys do it too.
And that's their way of bypassing the time it will take to review.
And they just move the code forward.
So I think that they might be seeing improvements in their review time.
So on the CI time, we did talk about flakiness.
We actually have improved the flakiness quite a lot.
So we're down from a significant level of, it was 17% in our Android time.
We've spent a lot of time trying to improve our flaky tests.
CI, we've been able to bring down for iOS from 16 minutes to 8 minutes.
And CI, I mean, I don't need to tell you all this, is a critical stage of feedback.
So the time saved here improves the overall experience and it allows teams, most importantly, to shift their behavior.
And this is something I really want to highlight.
You want to shift the behavior to when you have shorter time to shorter and smaller releases, smaller changes, right?
And so the more we can shrink the CI time, the more we started to see that we would see smaller changes going through, which was a really good outcome for us.
Deployment time, which is a big one for us.
So our P95 went from 48 minutes to 22 minutes, which is about a 54% reduction.
And so getting services into the development environment, everybody knows, is a really important goal for us.
However, I will say a deployment time that's roughly the length of time of an episode of Rick and Morty is still not really acceptable to me.
So I want us to...
push a little harder.
One of the guys in my organization is sitting in the front rows.
This is the part he owns, so I'm talking to him.
We can do better.
But I'm really proud of the work that they've been able to do there.
It is a good win.
And then finally, on the monitoring, how many of you have established, well, first of all, how many of you are in platform engineering?
Awesome.
These are the people.
All right.
So how many of you have actually established formal SLOs in your platform?
Yeah.
Okay, good.
That's good.
That's good.
We did not until recently.
It was very, you know, of course, we were aware of the availability of our systems and everything like that.
But the amount of error budget was not always formally recorded.
And the SLO itself, more than just being a tracking mechanism for us, was a mechanism for us to communicate our intention to the rest of the business.
And that was a really important step for us.
We started to establish air budgets, or sorry, SLOs for availability as well as performance.
And we're adding to those more and more.
So that was really important also to communicate our intention in that area and to build some discipline into what we prioritize as a group.
We also, down here, and I'm not going to go through all of this.
Yeah, the formatting is a little funky, huh?
It didn't look that way before.
All right.
So one thing I will highlight.
So how many of you have spent time trying to right size your clusters?
Good.
Keep doing that because it's going to get real expensive real fast.
This is one of the things that we spent some time on.
So we actually built something in-house and then we later decided to switch to a vendor that has worked out pretty well for us.
And we've been able to optimize our resource utilization and our development environment.
We're rolling it out to production.
43% is a massive savings for us.
It's massive savings for anybody.
This is critical because we're expecting more and more workloads.
And I want to highlight this because all of the points that we've been talking about with agents making lots of changes, it's going to bump across this very heavily.
Man, I'm running out of time.
I've got to move forward.
High point here is that this is actually just KTLO work and general performance investments, but every bump that you have in your road.
Agents are going to hit it 1,000 times more than your developers.
And so it's really important to invest in those.
And it's even more so now your license to explain to the business why you must invest in those things.
Because if you don't, you're going to slow things down.
All right.
So some of the wins that we have.
100% of Mercari employees are using AI tooling.
This is up from 30% 12 months ago.
It is helpful that it was a mandate.
70% of code generated is being used with AI.
I want to highlight this doesn't mean 70% of our lines of code.
We don't care about that.
We do care about are we utilizing the tooling and learning and benefiting from it.
64% year-over-year increase in the output of engineering measured by some of our internal proxies.
Now, output is really, it's a rough metric, right?
These are things like, okay, we've been able to produce more.
PR sizes have gone up.
But I won't say that we have seen 64% of value necessarily being increased in delivering out to production.
And that's for another reason.
We found code went faster, but release fastly did not keep up.
PR spiked, but merge throughput lagged.
Q at the code review in the CI grew really big.
We realized that code wasn't really the bottleneck.
It was actually more of the issue was the idea.
to the code that was where we saw some of the challenges.
And as a very real example, oh, and this one was nasty, our maintenance PRs.
So all the things where, you know, yeah, we got some changes out.
We thought it was really great.
We saw that we had to keep making more changes than we were expecting.
And a lot of that has to do with the fact that code quality was impacted without a doubt because it's not as easy for people to review something that they did not write.
when we started seeing maintenance PRs go up.
One good story around the throughput.
So we had this project called NACO.
It's an internal project we had before we were looking at utilizing AI.
The estimate was that it would take six months to build it.
A team of two engineers were able to build it using some of the AI tooling, including making architectural changes, and they built that in a month and a half.
This is Greenfield again, right?
So they saw a massive improvement in time.
However, they got stuck, not because of the code, but because they were able to move so fast that the PMs and designers did not have new things for them to work on.
And as a result, things slowed down.
And so this led us to this realization.
We need to invest in further earlier in the stage.
So we started to invest in what we called agent spec-driven development.
So we brought agents into the front part of the design of the solutions where agent would read in this first stage, they would read the code, they would go through the docs, they would read Slack history, and it would build up the agent's context early on.
And then we would get into the spec generation stage, or you can think of a little bit now the term is intent again.
But this would be the new artifact where a human would work with the agent on.
figuring out what it is that we wanted to build.
And then you go into the implementation stage where, in this case, humans or agents themselves would start executing on the work, and then it would go through that validation or eval suite to regression tests, smoke tests, and whatnot.
So the reason this matters is because the second dip wasn't because AI couldn't write code.
It was because the upstream was the problem.
And so...
This was our attempt to move AI to the front of the lifecycle, not the middle.
So where are we investing now?
Brownfield modernization.
So most of the AI hype is around greenfield, as you all know.
Brownfield is actually where the money is.
And so we're starting to use agents to refactor, document, test, and modernize services.
Critical observability.
We're not just observing the traditional parts of the system.
We're also looking at instrumenting the entire.
PDLC so that we can understand basically everything from, like I mentioned, the spec generation all the way through to delivery into production and then even what the results are after production.
So that can help us start to identify other parts of the problem.
Guardrails for agents.
Obviously, you can't have agents writing payment code with extremely long.
or payment codes, we also do have a FinTech component of the company.
So we need to have strong guardrails.
And I have a lot to talk about on this one, but we won't have time today.
But risk management is critical here.
And I actually want to challenge you all to think that the strategy should be that we change the way we think about risk mitigation.
Heterogeneous small teams, we're forming these small pods for our new projects and new initiatives, the kind of two to three members that are mixed group.
with PM, EM, and maybe an engineer, or PM engineer, and maybe a designer to kind of run after these small projects.
And then we're rethinking what the PDLC should look like, the product development lifecycle.
So we're looking at more of the human on the loop as opposed to in the loop and starting to take a look at setting constraints, humans for setting constraints and direction, and agents do the execution.
And then of course, better dashboards.
DX probably likes to hear that.
But the better dashboards are for us, not for the agents.
And it's for us to better understand where the other parts of the problem are.
If you think about it as a road, every one of those little potholes you might have ignored because you were humans and you could get around them.
The agents don't know that and they slam into those.
And you don't know that until they slam into them.
And so getting those dashboards up has helped us identify where are they hitting the bumps.
So this is what we are working towards is a multi-agent.
Well, it's a multi-loop with basically a swarm of agents.
So we're working on this now, and it's evolving the way we think about a development lifecycle and thinking about it, like I said, as a series of loops.
So improvements made in the very, very inner loops obviously have an outsized impact on the rest of the delivery.
Basically, until you get through all of these loops, you have not delivered value for the business.
So I'll quickly just mention the inner loop is what you might think of here.
It's intent, cogen, and validation.
So the faster and closer you can make these things to each other, the better you're going to be.
So we're trying to move as much validation as we can right next to the generation of the agent.
So if it's running in a sandbox somewhere, we want an ephemeral environment.
Actually, even more than an ephemeral environment, we want local testing, just like you can imagine somebody with their laptop doing that.
on their laptop, you want to bring it right next to the agent so that it doesn't have much distance.
That means cutting things out, like not having to check in stuff into Git necessarily, or not having to go through a PR and a code review cycle for it to validate what it built.
So the closer you can get to that inner loop stage, the faster gains you're going to get, the better gains you're going to get.
The ephemeral loop, we are making, we already have preview environments.
We're spinning up.
We're still investing in this to make it faster.
The preview environments take minutes to come up.
They go through a PR cycle.
It's a bit of GitOps flow.
We think we can do better, and we're trying to make this a little bit faster so that it's basically a namespace isolated, an isolated space in our cluster so that the agent is able to validate something within an active running cluster, but that's not exposed or deployed out to the environment yet.
Then finally, once you're satisfied with where it is, or rather the agent's satisfied with where it is, it'll go into a development or it'll deploy out to our dev environment where it can run more robust testing, look at the integration situation, and validate that what it actually built is working.
Now, my personal goal, and actually the goal of Sneha and Raphael, who's in the front here, I'm going to mention you, we want all these steps, everything out to the purple to not require human approval.
That's actually what we're trying to go for because we believe that if we can get this all the way through to the purple stage, actually past the purple stage, so development, deployment, we will be able to unlock a better, faster iteration for the agents, but more importantly, we'll be able to focus on the right problems which are around guardrails and how to basically make sure that the code that's going through our systems meets our standards without requiring humans to determine that.
That's a really key goal that we're trying to get to.
The outer loop does still require, would still require humans for the foreseeable future.
But this still can be done with canary deployments and progressive rollouts that can be automated and rolled back.
And then the measure piece there is, of course, our way of knowing did we build the right thing.
So this is essentially what we're focused on.
Some hard lessons.
Just as a quick summary, gates slow you down.
You have more than you realize.
Go through and audit your system.
I've gone through this multiple times.
What does it take to build a service on our platform?
What does it take to get a change to go through?
What does it take to add new infrastructure?
Ask all of those questions and manually do it yourselves.
When you go through the process from an experience standpoint, you suddenly discover all these little points of tribal knowledge, all these little gates and approval steps that need to happen.
all of those are going to slow down your agents.
Behavioral change is by far the hardest thing.
I mentioned about eliminating human approval all the way through to the dev deployment to another individual earlier today or yesterday.
And the first reaction they had is, oh, engineers are going to hate that.
And I was like, why?
Because they don't want accountability.
Makes sense.
They don't want the agent to make decisions, roll something out, break something, and then they're in trouble.
And that's a really good point.
That's a thing we have to combat is we're going to have to figure out how to change our views around these things.
Right.
So behavioral change is by far harder.
You can change agents.
Changing humans is a little bit more difficult.
The W curve is real.
So expect lots and lots of these dips.
Every time you hit a dip, you're probably going to have another dip.
We don't we've we hit two of them.
I expect we're going to hit a third one.
So it looks something like this.
Our first dip was due to the fact that we didn't have enough visibility.
Our second dip was due to the fact that we had upstream and downstream, that coding was fast, but there was other issues between upstream and downstream.
And basically what makes these deeper and not shallow has more to do with the fragility of your platform components, heterogeneous architecture that makes it very difficult for agents to understand because they lack the context to know what to do.
And then, of course, legacy systems, stuff that nobody wants to touch.
Agents will touch it confidently and will break it.
So this is kind of the sum up slide.
And we'll borrow a frame from Jim Collins, the Stockdale Paradox.
So Stockdale was a prisoner of war.
He survived because he held two ideas at once.
Total faith that he would prevail in the end and brutal honesty about how bad the current reality was.
The people who didn't survive were the optimists who just set arbitrary deadlines and then got crushed when they passed.
So that's how we're approaching AI at Mercari.
We absolutely are going towards a multi-agent future.
We have total conviction about that destination, but we are also completely willing to look at the dashboards every week and see that dip and then pivot until we can improve.
So faith in the destination, brutal honesty about the road.
So every company has their own journey, and without the right insights, you're going to spend a lot of time on the wrong problems.
Thank you.