# Building Context-Centric Software Factories with AI Agents

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-09-02

## Transcript

We hit a peak of something like 850 PRs in a week and 85-90% of those were handled by agents entirely.
A lot of people when they think software factories are agents they're sort of expecting a speed-up or hoping for a speed-up.
So we've actually seen that now that we have all this extra capacity.
You don't turn all of it towards cranking out new features.
One of the members of my design team was saying, I just put up and landed 13 PRs that were all about fixing consistency and copy and brand voice and like removing borders that weren't supposed to be there.
Things that would have just lingered on a backlog can actually now get done.
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
Join your hosts, Guy Pagiani and me, Simon Maple, every week.
as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
This is the AI Native Dev.
Yesterday you were writing code.
Today you're prompting agents.
Tomorrow you'll be running a software factory.
Agentic coding is moving fast enough to give anyone whiplash.
This June AI DevCon London sold out.
Hundreds of developers, engineering leads and VPs packed in to hear speakers from Anthropic OpenAI, Netlify and GitHub cut through the hype and get to what actually works.
This November, we're bringing it back to New York.
Bigger, bolder and full of people actually building the agentic future.
Three days of real talks, real tools, real answers, whether you're directing the agents, building the loops themselves, leading the teams or setting the strategy.
Our early bird pricing ends on September 30th, so make sure you lock in your place before prices increase.
As a podcast listener, you can get 15% off your ticket using promo code POD15.
We have so much planned, and we can't wait to have you join us in New York.
Hello, everyone.
Welcome back to the AI Native Dev.
Thanks for tuning back in.
Today, myself and Drew here, our head product at TESOL.
We'll want to catch you up a little bit on what we've been up to at TESOL, what we've been building over the last sort of two or three months.
I think we had a bit of this episode talking about the TESOL agent, which we'll kind of mention.
A year and a half to get me on the podcast, back in three months.
So I'm trending.
What can I say?
Just sort of so, you know, irresistible sort of once you're getting on.
So really, there's a lot that we've been building here at TESL, but I think the key thing we're most excited by is this expansion of TESL from just focusing on helping you own your context and having a skill registry and build skills correctly and optimize them, which I still think is an enormous, important piece of this new development paradigm, to starting to build loops.
We talked about that a little bit with the TESL agent launch on it.
to now really increasingly thinking about TESL as a composable factory or as a platform that helps you build factory.
And we talk about, and we'll sort of mention this a lot today, we talk about how TESL helps you get to your factory from skills to loops to factory, which is sort of our view of the world of this sort of context-centric factory.
Yeah.
So before we dive into that, we actually released this super fun little mini documentary about Kikimora, the spirit, the Eastern European spirit.
It will clean your house at night.
I'm quoting Maria, so thank you, Maria, for letting me know this.
It cleans your house at night, but only if you're good.
If you are making a mess, if you are being disrespectful, it makes things worse, which we felt like was a perfect analogy for agents.
Like if you're keeping your house in order, agents give you superpowers.
If you're being irresponsible, they give you anti-superpowers, right?
They go off the rails.
They amplify whatever it is that you've sort of given them to amplify.
A little bit of some Christmas illusions there for the naughty list.
So that's the name of our sort of dark factory.
And we have a really fun mini documentary about this that was very well produced on it, you know, by a friend here, Tom, who's our producer.
And definitely maybe like 13 minutes or something like that, 14 minutes.
Definitely a fun watch recommended.
We're not going to force you to sort of repeat the whole thing over here, but maybe before we dive into the sort of factories and how we build it, maybe Drew, you can just remind us a little bit of...
Tesla's own kind of dark factory would be built inside.
So a few words about what is it that we have and what's the value we're seeing?
Maybe I'll start very, very briefly.
I feel like every one of these podcasts has to start with a bit of just like terms.
What do they mean these days?
You know, two weeks later.
So really quickly, I think a couple of things.
First, we've mentioned skills, loops, and factories.
So skills are, you know, obviously skills became quite popular at the beginning of the year.
They've sort of expanded.
Some people will call them plugins.
Think when we say skills as like a unit of work.
It's either a workflow that you want completed or it is a policy, a standard that you want agents to adhere to.
That includes skills.
It also includes things like hooks or MCP tools, like anything you could bundle in a plugin.
It's just that skills is the most popular term.
It's what everybody thinks of as the unit for agents.
So we sort of use that interchangeably.
So that's, I think, one.
The second is loops.
Think of these as effectively automated skills.
It's things that are running without immediate human intervention, at least to start them.
And then as they run, typically there's some kind of meta process where the loop is getting better.
As it runs, it is making things better for the next run.
And so you start to get compounding gains.
You start to see agents sort of inflecting your productivity upwards faster than just linear.
Yeah, and this ends the loop, right?
Because it goes from an agentic workflow to something that loops with some form of learning back into the beginning.
So hopefully the second time around or the hundredth time around is better than the first one.
Yeah, and loops are where you really start to see your first glimmers of that sort of inflection point or your trajectory really taking off.
And then a factory is, in many ways, it's a way of working.
It's where you've moved almost all of your development.
to the creation and maintenance and monitoring of loops.
And the loops are now what produce most of your software or the product.
One way of thinking of this is that your factory will have some code.
That code will largely be loops.
And then your product that you ship to users is also some code.
Where are your humans spending their time?
When you move to the factory, you're mostly spending your time maintaining the loops and thinking about how to make them better.
and in a ways it's like the entirety of your sdlc right if your loops are Kind of a kind of illusion of your pipelines, you know, of some automation that become now so better.
I'm trying to make factory development lifecycle a thing, FDLC.
It hasn't caught on yet, but we'll see.
If I get to do a term segment at the beginning of the next one, hopefully that'll be one.
Maybe the FDLC.
Yeah.
If you have a list, it might be the FDLC versus the FDLC.
So with that as groundwork, what is Kikimora?
What have we been building?
So Tesla's really been trying to go on this same journey from the start of the year.
Our development internally was very skills-driven to start.
We wrote down what good looks like, how we wanted code review to function, how we wanted, if you're building something in the CLI versus in the UI, just writing things down.
And then once we felt like we were in a good place there, and we'll talk about why we think that's important to start there, we then wanted to really get into the business of handing work over to agents.
And so we started thinking about how can we automate.
more and more of our tasks so that a human isn't having to click buttons or type text into a text box to kick it off.
And we started that and very quickly, like the minute you get your first couple loops in place, you start to realize this is going to get out of hand really quickly and you have to move to a different way of working that has a bit more of the solution baked into the problem itself.
And so that was where we made the decision, like, we're going to go full factory, we're going to build out a system where basically all of our work, all of our PRs are created by agents.
We focus more on shaping work, so we create issues, reviewing work, so we will look at PRs as they're put up.
Increasingly, we're trying to have agents review more and more of that code, and we only invest in the sort of like high leverage points.
And then where we're doing our software engineering, our observability, our monitoring is in the loops themselves.
And so oftentimes you're creating loops to monitor loops or you are creating dashboards to control plane your loops and things like that.
And so that's been the flow that we've moved to internally at TESL.
I think the last stat we saw, we hit a peak of something like 850 PRs in a week and about...
85-90% of those were handled by agents entirely.
So that's been our journey.
Again, we won't make you watch the whole documentary again here, but if you're interested, go check it out.
It's a lot of fun.
Lots of great nuggets for folks to think about.
And the pace, I guess maybe what I would add is before we kind of leaned into the factory, what it felt was...
that we had a bunch of high automation environments, but they were very kind of disparate environments.
Different developers, like some members of the team, first of all, were sort of more, I guess, into automation and automated their kind of work to a large degree, while others were a bit further behind, just sort of using it in a slightly more sort of standard way.
But then also the ones that were automating were automating their work and iterating, and it was quite siloed.
So I think the factory is also a bit of that.
path to go from single player to multiplayer, right?
And go into that collaboration.
And I think from a security lens perspective, there's also the advantage that once you start running things in the cloud and in kind of these defined workflows, then you start kind of having visibility to it.
You know what's installed.
You can identify risks and contain them.
You can create more protected environment.
It's a transparent system as well.
You can observe everything.
Yeah, exactly.
And I've...
I'm one of those who thought the cloud IDEs will catch on a lot sooner.
It's like they're just...
they're just better.
You have a stronger machine behind there and you can work from any spot on it and you can do it.
And lo and behold, they didn't really.
They're used somewhat.
But there's all these advantages.
I think as you get into agentic workflows and really you're interacting through chats, I mean, developers' machines look more like the matrix and kind of the stereotypical hacker in movies than they ever did before, then I think it makes sense.
And we're seeing, I guess I will add, maybe with a management hat on it, is that we're seeing some incredible speed on it.
Again, there's a bunch of stats that we mentioned in a variety of spots and in the movie.
But we're seeing just sort of compounding paces.
And now increasingly, indeed, the org is leaning into it and more of the GTM is in it.
I think maybe if I could add one quick, the sort of increased feature velocity.
Great to see.
It's the one you sort of are targeting.
It'd be a lot of people when they think software factories or agents, they're sort of expecting a speed up or hoping for a speed up.
Two of the things that we've seen that were interesting, I may not have guessed up front, the big one was an increase in quality.
So we've actually seen that now that we have all this extra capacity.
you don't turn all of it towards cranking out new features.
Some of it, like one of the members of my design team was saying, I just put up and landed 13 PRs that were all about fixing consistency and copy and brand voice and like removing borders that weren't supposed to be there.
Things that would have just lingered on a backlog can actually now get done.
And then the other one is just better fungibility.
Like people, yeah, fungibility is maybe such a boring business way of saying it, but.
Everyone is able to do a little bit of everything.
Like I just mentioned, a designer putting up PRs that are getting merged, but also our GTM team is able to make changes to the marketing website without waiting on support from engineering.
Engineering can write their own specs.
You just have this ability for people to work on whatever's high leverage and not thinking about, like, who's ready to pick this up?
How can I hand it off?
All of that.
Right.
And I think, so factories are en vogue.
You hear a lot of companies talk about sort of their path to it.
And I think that's exciting.
This is the new development paradigm.
And at the same time, there's sort of this world of skepticism around being able to produce quality software in that process.
And is it ready?
a lot of sentiment of how you move from the bottleneck being the production of the code to being the review of the code and is the code that's being generated any good.
And at TESOL, we've long seen the world through this sort of lens of specs and context and skills, right?
Like our kind of roots are very much around, you know, start by defining a spec, right?
It sort of says, what is it that you want to do?
And we have this core conviction that...
To automate the thing, you first have to sit down and do this bit that is defining what correct is.
What is it that you want to do?
And I would use, I think...
since the beginning of TESOL, this example of saying, hey, what do you want for your birthday?
It's like, I don't know, you decide, you know, like vibe it on it.
You may or may not be happy with it.
You know, if I, you know, use that sort of approach to buy you every year, you know, that gift, you know, some of them will be disappointments.
And I think this is what we're seeing with a bunch of that software.
And so we think of context as the core, right?
As this sort of manual for operating your company, operating your workflows, you know, a bunch of those definitions.
And when we talk about building the factory, we have this perspective on how a factory should work, which is that it should be context-centric, right?
It should, and hence, kind of the sort of repeated terms of going from skills to loops to factory.
And it was interesting to sort of see that manifest in code.
A couple of months ago or so, we launched the TESL agent.
The TESL agent helps you build loops.
That was a core part of it.
You run it and it helps you find automations and improvements and optimizations in your system and starts creating a loop that would run that and make it go again and again.
We had someone on our team, Tom, build...
an automation that says, okay, let's build, I forget actually the specific workflow.
I think it was around test coverage or something in that domain.
And created that workflow with sort of plain vanilla Claude and with the Tesla agent and with another sort of factory related agent out there.
And what we saw was that these sort of two other ones, they actually built the solution, the cookie cutter solution.
a little bit faster because they just wrote a whole pile of code.
A lot of that output of that process was creating code that just created the pipeline to run a bunch of those tests and run the process around them.
While the approach that the Tesla agent takes and was almost unintentional, it's just our life view, our view on this domain, was first it created this chain of logic.
This is the definition of what correct is.
This is the specs.
it would work and then it built on top of it and so it was a much more context driven workflow which on one hand took a moment longer and was stronger on some things it was more resilient more malleable and could find more cases and make them work at the same time It, again, was a little bit more effort, but it was more resilient now to be able to evolve and to grow.
So this sort of context centricity was so ingrained in how we build.
So this is like a little bit of like, I like that story because it just sort of naturally emerged from how we think about the world.
And I think you had a similar view when we think about that context centricity when it comes to code review.
Maybe you can say a few words about that.
I think that code review is actually...
Both a good example of how context centricity is sort of crucial, but it's also hopefully one that hits close to home for folks.
So maybe to start with the way most people will adopt some form of agentic code review.
There's a bunch of great tools out there that make it really easy to get started.
You install a GitHub app or you click a few buttons here and there and it's boom.
You've now got agents reviewing your PRs, you're finding bugs, you're maybe merging faster, or you're at least catching more issues.
And it feels great.
It's kind of like this dopamine hit.
As is so often the case with AI, it's so easy to get to that first demo or that first wow moment.
How do you scale it from there?
And what a lot of people see is that...
amazing, I'm doing agenda code review, I'm living it, and then a couple weeks later they find, well, okay, we're finding a few more bugs, or maybe merging a bit faster, but nothing's changing after that.
We got a small boost, but it's not this exponential compounding gains, it's just a small improvement, and again, we're stuck again.
It still feels like code quality is a problem, or PR review is still an issue.
And yeah, the Tesla philosophy is because you're hitting this wall because you jumped past the, you know, you tried to get to the big muscles without working out, or you tried to get healthy without eating your vegetables.
There is a certain amount of, you have to write down what good looks like.
You have to know, like, what do I care about?
How does quality in the code base manifest?
And if you don't do that, if you skip past, you just have a tool that says like, don't worry, I'll handle it.
I'll review the code for you.
You will eventually hit a problem where, two big ones.
The first is that, well, now you'd like to use those standards elsewhere, right?
Like, code review is good for catching things at the end, but really what you'd like to do, especially to help with costs or to get features developed faster, is...
While building the feature, you'd like the agent to also see your standards and make sure that it is writing code in the correct way.
And, you know, not everything is going to get caught by code review.
So you'd also like to have regular automations that sweep your code base and say, is there anything that's not conforming to our standards that slipped through or somehow is accrued and we need to fix and pull back?
So you need all the, like, you want to use it in multiple places.
And if you haven't written it down, you can't do that.
You can't reuse it.
Or say you have a new project.
Somebody's creating a new app or testing a prototype.
It's in a different repo.
How do you translate your standards over to them so they can get started right away?
And so there is this, one, there's this enormous benefit in writing it down and being able to share it.
But also when you write it down, it becomes much easier to reason about and improve it and to even have.
specialized versions for different parts of the code base.
So you want to say, in this area, our standards are this.
For the front end, we care a lot about ARIA attributes and accessibility and having the right nesting of borders.
But when it's a back-end PR, none of that matters.
Don't waste time, don't waste tokens.
Focus instead on brittleness or reusing certain components.
And in fact, this specificity, you know, being able to tune exactly what you're reviewing becomes increasingly important as you move towards factory, because really the volume a factory is going to put out, you have to get humans out of the loop for code review.
And a general purpose code review is not going to get to 100%.
quality, right?
Like you have to be able to say for this component, here's all the things you need to check.
You have to get really specific.
And you just can't do that if you're not writing it down.
why they're there, why they're useful, the mistakes that we've made along the way, the sort of metrics that we focused on early that actually didn't end up helping us and the ones that actually did and all the different quirks and features that ended up building out the factory that we have today.
This is going to be a much more practical look into the factory that we have here at TESOL and some of the specifics on how it works and why we made some of the decisions that we did.
Go to tesl.io slash events and I hope to see you all there.
I think, so really kind of a good articulation of...
the view that you have to capture and define what correct is, what correct looks like.
And then you can do that at the granularity that you want.
Sometimes it is shared across different parts of the product stack.
You might want it in review and encoding.
Maybe you want it across repos.
And sometimes it's the other way around.
It kind of gets more granular as well.
In this subfolder, when Drew writes code, I want to make sure that you sort of check that it doesn't again do X or whatever it is.
So I think the sort of core is a really good story of it and an example of a loop that you can improve because indeed a factory has to sort of go to, you have to sort of drive towards higher autonomy and that has to sort of come with sort of definition of what's correct, hence kind of skill-centric.
So I think let's shift a little bit and talk about some of the capabilities that we're sort of excited by in it.
And because we talk about the journey from skills to loops to factory, maybe we'll start with a little bit over the last sort of two to three months on it.
What's been sort of interesting in the skills part of it, of the product?
What's been launching?
How do you capture what's good, what the workflows are, etc.
So I think last time we checked in, Tesla Agent was a big part of this.
We talked about how you could use it to create loops that themselves turn back and help you find new context and things that you want to write down.
Since then, we've added a big feature that we've been calling Skills Inventory, Context Inventory.
So what this is, is a...
command line as well as a GitHub app that you can install across your source estate, so across all your repos, and we will scan and find all instances of context that's being fed to agents.
So the most common form here is skills, but it could also be other agents MDs or plugin marketplaces, things like this.
And then we just help you reason about it.
So we give you a bunch of findings around, do you have out-of-date context?
We can detect, oh, you've installed this skill from Anthropics Marketplace.
It's now four versions behind.
Do you want to update to the most recent version?
We can do the same thing internally.
Like, hey, we have found this same skill in five different places.
It seems like this is actually the source where it's being maintained.
Do you want to link all of those downstream versions to this and then get them up to date?
But we can also find things like duplicate context where multiple people have been solving the same problem and not realizing it.
And we can help you standardize on one so that you can then start improving it for everyone.
We also look on stale things where maybe you wrote a skill about how to contribute to the backend and now the code has drifted away from the skill itself.
We can help you identify that and get it up to date.
And this is a capability we're going to keep investing in where We're just continually scanning your code estate.
We're finding all of the context that agents are using.
We're pulling it out and we're telling you how can it be improved?
How could it be reused, shared?
And so that's an area that we'll keep improving, but the sort of duplicate, stale, or in general, just finding things where there's opportunities to improve the overall performance and quality of your critical skills is a big improvement we've made to the platform.
Yeah, I think it's been interesting to see how how deep the inventory capability goes.
We thought it was a quick feature.
It started quick.
It's like, hey, I will just enumerate files in your repos and I'll find all the ones that are called X.
And then, okay, we'll run a review on all of those.
We already had the skill review, so we'll score them.
We'll say, are they any good doing it?
And then from there you get into Indeed, this like, well, these look like they do the same.
It's amazing how...
uh unimaginative we are when we think about sort of ideas uh for uh for skills on it like a lot a lot of duplicates which makes sense but then you know that creates problems, you know, it's not just wasteful because the knowledge doesn't accumulate in one spot.
It again creates kind of inconsistency and misalignment in the systems.
How do you drive that?
Once you created those, there's all this question about, well, when people use a skill, how do they discover the right one?
How do you do?
So we added the whole kind of recommended skills that nudges you to sort of use the right ones.
I guess security has been there for a while, you know, around sort of assessing whether those skills are sort of trustworthy in doing it.
And it just sort of goes on and on around action.
the dynamics between the platform team that has found, maybe done that broad scan, and the people that own the repos.
Inventory has kind of become...
an entry point to all of the tools that you know you've known love for tesla right like security policies governance being able to require hey these skills like our brand voice or our code review needs to be in every repo now we can help you find those skills and set up those policies so it kind of becomes the integral control plane for somebody who's thinking about how do we manage skills across our organization yeah and i think um I can't avoid the analogies, the security world I came from, which is at Snyk it was very much around, hey, we'll scan all your repos, we'll find all your dependencies.
And it was similarly like a very deep tunnel over there to go to say which ones are vulnerable, but you get that instant feasibility.
Cloud security, poster management, CSPM tools like Waze are known for that.
So I think it's similar.
And then I think we've also done a bunch of stuff on the quality aspect of the skill.
We've started building kind of capturing of logs and we built these sort of verifiers.
Do you want to maybe say a couple of words on that and then we'll kind of go to loops?
Yeah, and also I'd be remiss if I didn't also mention we've extended support beyond just engineering use cases.
So now we have the ability, once we've inventoried all your context, you can now...
version, distribute, deploy for your go-to-market function, for sales, for support, for anything like that.
So that's been a big boost.
We're using it internally as well.
I like the thing I'm kind of especially fond right now of the synchronization with the marketplace.
Because I feel people on one hand like the convenience of the cloud marketplace.
It just sort of pushes it to co-work or to others.
don't really want to be beholden to Anthropic.
What about the open AI, chat GPT marketplaces?
What about other places?
Managing the skills centrally on the TESOL registry and then synchronizing them to the right spots has been quite appealing.
All the benefits being model agnostic, having an actual versioned lifecycle so you can prevent regressions, all of that, now comes for your non-devs as well.
And in a nice, smooth experience where you don't need to teach your sales team Git workflows or Semver or anything like that.
So yeah, on top of that, we're starting to explore from our own factory.
A lot of what we build comes from first, what did we need ourselves to make context-driven factories work inside of TESL?
And one of the big things we found is that skills really are...
the codification of a workflow of what good looks like.
And that's great.
That's sort of like the interface where you want to write down, this is what's important.
This is how agents will look at it.
But everybody wants that little bit of extra security, the extra surety that like, yes, I wrote it down, but agents don't have to follow Markdown.
So did it, did it actually do that?
And so verifiers are a thing we've built at TESL.
You can think of it as a way of turning your skills into or creating off of them very focused, high accuracy, very fast, very cheap LMS judge tools where to take an example, let's say you have a skill that says all front end components must have ARIA attributes on them, right?
And we can generate from that a verifier.
This is a very simple case, but we generate a verifier that says for every TSX file or JSX file that's changed.
We want to scan all React components and say, do they have ARIA attributes on them?
That's like a very simple prompt.
Then LM judge can be very small, very cheap, and get it right effectively all the time.
So fast, so cheap that you can actually run it in CI if you'd like.
And in fact, that's what we do here at TESL.
And so as a result, it basically gives you the enforcement half of a skill.
So you write out what is good look like, what is success.
And then we generate these verifiers that you can then put up in CICD so that you know your skills are being followed.
And so that's a tool that we are increasingly relying on where I think we're working right now on the ability to say, as soon as you create a skill, we'll create verifiers for it so that you can enforce the skill.
in the way that feels deterministic.
It moves us back towards the world we're used to in software development.
I very much see this as the trust but verify.
You run the agent, you give them instructions, you trust them, you have to let go to some extent for them to do that, but then you build these verifiers.
Indeed, I like the speed.
In general, verifiers are still forming, but they can...
check the agent actions against the logs.
They can check the output of it, of what has it done.
They can be deterministic or they can be LLM as judge.
And I think there will be a core fundamental going alongside in a skill, the context of what you want it to do.
Verifiers are, you know.
the checks to say, has it actually done it?
And I think that is very powerful.
And I think maybe, you know, we do need to get to loops, but maybe say a few words, which is we've also seen a bit of a revived interest in evals.
Evals have kind of...
been a little bit more the purist domain, the people reviving it to the extent.
And I think what has happened is the world has matured enough and people have come across those cases where they have a challenge.
And I think it might feel a little bit scary to people as we talk about, hey, you have to sit down and kind of write down all of your paths.
But I think one of the key unlocks that we've embraced probably starting about three months ago, is this appreciation that a lot of that correctness can actually be extracted from historical behavior.
It just then needs to be codified and captured, and then you need to iterate on it.
So verifiers, evals are increasingly, even in the Tesla agent example, extracted out of historical activity.
So it's not that it's a real burden that you need to...
sit down and kind of write down painfully all of those instructions.
But if there's a thing that you are reviewing, if there's a thing that you own, it really should be these domains of your instructions.
And the more you review and trust that sort of instruction set and those verifiers, the more autonomy you can give the agents as they operate and the more value you get.
And yeah, I think it's just worth pointing out that that the Tesla agent exists to try and make it not feel like going to the gym or eating your vegetables.
You know, like to your point, you can point the Tesla agent at your repo, at your PR history, at your tickets, Slack, anything you want.
And it's, it's whole job is to be able to extract this context and capture it for you so that yes, it is a bit of a, you're not just clicking a button and getting code review, but you're not sitting down and spending three weeks getting your whole team to, it's like, look at our code review comments from the last.
three months, write a document that captures our code review standards, and then we'll just evolve it from there, right?
It's very easy to get going and then improve over time.
And that's where loops come in.
Hey, everyone.
Hope you're enjoying the episode so far.
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
but for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
So maybe let me say a few words about loops and we'll dive into specifically code review, which we launched as a dedicated loop, an example, one of multiple examples of how to build a good loop.
But maybe say a few words about that.
So what we've observed is we work with customers, we help them create very good context, but the reality is that companies kind of evolve their practices.
And if you're not...
creating loops if you're not creating a factory, you're not really benefiting from creating this well-crafted piece of context or these great skills because you might not be equipped to tap the benefit of them.
And so we've been building, and I think this was the inception to create the Tessel agent, so it's like, hey, let's help you create those loops as well.
And the loops will demonstrate that.
A code review is an example we'll dig into, define what correct is.
evolve that, improve that as a skill, get better code review.
But then, you know, many automations that happen within the system that really should be done recurringly.
And we have been building sort of two types of maintenance loops on it.
One is a maintenance loop for your software.
Can you find flaky tests and fix them?
Can you review architecture on a recurring basis?
Can you indeed get test coverage of it?
Can you just handle dependency upgrades?
And so all of those are example of loops that can be skill-driven.
that the Tesla agent can very easily set up for you and then you can go on to optimize and improve.
And it's not hard.
It's not a big lift for an organization to come in.
You don't need to be afraid of that sort of factory ward.
You can kind of increment your way.
They're always skill-based so you always kind of own your sort of heart and your core.
And in fact, can start manually at first.
Make sure it works.
run the skill a couple times, then turn it into a loop.
Yeah, precisely.
And you can pull it back out of it.
So you very much are kind of in the driver's seat.
And you kind of build out of those now this sort of optimization loop.
And so the Tesla agent will sort of help you set up the skill and run it.
But what you also need behind the scenes, which is a little bit...
invisible or might not be trivial, is that once you start running, you need a few of these fundamentals.
You need to be able to run the skill in a way that captures the logs of what has been executed.
You want a central place in which you can modify the model, for instance, and say, okay, now I want to run it with a different model.
So if I ran a bunch of evals and...
Now I know that I can run it with a cheap model or whatever it is.
I can run that.
I can modify that relatively easily.
And so all of those, the TESOL project, a bunch of that infrastructure supports creating those loops and running them.
And I want to stress that...
we still see the world as skill-centric.
Eventually what the loop does is it hones a skill.
It creates a very good skill, maybe a harness around it, like again, skill in the broad sense of the word.
But it really kind of hones a skill, and then you can use it in many places.
You don't just have to use it in the context of that specific loop.
So I think, feel free to correct me on a bunch of these things that I'm doing, but maybe we, again, this is a very...
big and kind of confusing world on it.
And so one of the things we're excited that we launched is around TESL Code Review.
Again, we have some dedicated videos around that, which we'll kind of link to in the notes.
But maybe you can say a few words about what TESL Code Review is and maybe using it as a representation of a loop.
So I think the core name of the game is that TESL's Code Review tool is effectively our stance on context-driven...
code review as one loop that you're going to use in a context-driven factory.
So we talked about, oh yeah, you get started with code review, it feels great, then you kind of hit a wall, and that happened to TESL.
We had a code review tool.
We still love it.
It's still a great tool, but we found that as we scaled further and further into factory world, we were just unable to compound and get to a place where we were spending less time reviewing code, which was what we wanted.
And so we took a step back and we thought about, what's missing.
We came to this idea that first we needed a way to be able to shift our standards left.
So we wanted the same things that are happening in code review to be happening at development time.
And also we needed to have faith that if things slip through the cracks, they will be caught.
And so we wanted automated loops that come up after code review time and search for things that got lost.
And so we said, we need all of the standards that go into code review to be documented and separate so that we can use them elsewhere.
So that's probably the core of Tesla's code review is that it is skill powered.
We call them lenses, just like code review lenses, where you can supply any number of skills.
It doesn't just have to be one that codify some standard of good in your code base.
You can say which files, which file patterns.
those lenses apply to.
And then just like any other code review tool, you'll set it up through a GitHub app.
And basically every time a PR goes up, The appropriate lenses will be triggered based on the file patterns and will review your code.
But now it's a set of skills.
And so you can also make those available to the agent while it develops or in sort of maintenance agents or automated sort of sweeps of your code base.
So that's a great piece, right?
Like it allows you to reuse it in many places.
It's also now a thing that you own.
It connects to the rest of Tessels.
context platform so we can help you evaluate your code review skill, optimize it, share it across repos for different teams.
And then importantly, it also starts to feed back into the context platform.
So that was another thing that we learned is that code review is a very rich source of what's going wrong.
Like what skills are we missing?
What skills aren't working?
And so as code review is running, it's not just making code review better, it's making the context of your code base better.
So it's looking for missing context or skills that maybe aren't being triggered or aren't having the effect that you'd like and suggesting ways to improve them so that they don't have to be caught at code review time later.
And that kind of gets to your point of, you really want these things to be integrated with one another.
You don't want your context for development and your code review standards to be separate.
You want them to feed back into each other to create compounding gains.
And so that's really what the TESOL code review philosophy is.
And I think a lot of the analogies to humans, right?
The developers on the team do a peer review.
So the knowledge that they have available to them when they review is the same knowledge that they have as they build.
And the loop is the example of someone having received a comment, having gotten some feedback in the review, and then internalize that and come back.
And say, why don't we make a standard for this or document?
Exactly.
Capture that so the next person coming along driving it.
For us, it's not the next person, it's the next agent that's coming along, has that knowledge.
I really like it.
I think reviewing is a critical part of it, but I guess I think of our code review in comparison to...
to the generic and powerful code review tools that are out there, is that I think the key difference is that you own it.
It's your...
skills, it's your tunings, you can therefore refine them to your specific needs, not just what it looks for, but also choices like depth of scanning and choice of how much do you spend on different models on it.
We've done some interesting research around that.
So you own it.
The second is that because of that sort of refinement and doing it, it's actually just better.
And we have a bunch of that data.
We've sort of seen it in our system.
And I don't think it's because We have some sort of brilliant algorithmic improvement over it, but this is probably just sort of the side effect of the fact that you own it is the result is better.
It just sort of finds more issues, it finds them faster, it creates less noise.
Specificity basically is the ruler of all quality.
If you can have a very specific task that the agent has clear instructions on what to do, it's going to be better than a generic, here's how to look for...
Code smells.
Exactly.
And then the third is that it is a path to your factory.
And increasingly, we say code review and we imagine a pull request.
But in practice, if you're performing, if you're kind of running an agentic workflow, right, or a factory that takes a ticket and implements it, you don't need to wait for it to have been sort of checked in.
And so you just sort of take that skill and run the review as part of your factory.
And you build that out.
So I guess that sort of leads us.
So we talked about skills and the skill centricity of it.
We talked about loops that indeed, you know, just sort of improve skills and use them with code review as an example.
And there's many others.
You can already do many today and more and more of these sort of cookie cutter use cases will come out.
And then maybe like let's conclude with the sort of factory piece.
And factory is admittedly like a more nascent piece for us, right?
And I think it's because we're kind of accompanying.
companies, right, as they sort of go through this journey.
And so their sort of more immediate needs were sort of the skills and then loops.
Now they're getting into that sort of broader factory of it.
And we've sort of learned through both kind of this work with customers and through our own sort of factory building around kind of the need for more componentry to be available to kind of make the factory run smooth.
Maybe you can say a few words about that.
Yeah, so the The core of it, again, we talked about if you have a context-centric factory, I'm going to sound like a broken record on that.
You want to start with writing down what the workflow is, what good looks like, what success is.
But basically what we found is that...
So that's important, but you can't stop there.
Having a really great skill that you still have to run manually or you have to go paste into all of the various places you want it to be used, you're not going to get the value that comes from having done the work to write it down.
And so we found with customers generally, you now need to provide the rails or the plumbing to deploy that context and to do more and more work on your behalf in an automated fashion.
And what we found is basically, once you start getting into this world, like factory building is really, it's more of a continuum.
It's not a final state.
You're not going to say, okay, I've made a few loops.
Now I will have a factory and in two months I'll be done.
It's sort of, you're going to keep working on loops until you have so many loops that most of your work is in maintaining loops and building loops to observe them.
And so it's kind of a continuum, whereas you get more and more, I guess, to use our term from last time, more and more loopy, right?
It's going to just, sort of transition seamlessly into factory work.
It'll be a sort of a blended approach.
And so the first thing that we've built to enable this is an automations platform, which effectively allows you to take the context that you've created and deploy it in a way that runs on schedule.
Coming soon will be things like web hooks and more automated triggers.
But the idea here is that you've done the work to make the workflow to define a standard, automations will now let you say, run this workflow every night or give it access to.
So a big part of the automations platform is controlling the environment where it runs.
So you can say this workflow needs Slack.
It needs Notion.
And then it needs to run every Friday.
You can put all that together.
It can run, but it's.
powered by all of the context tools we already have.
You can evaluate that workflow.
You can pick the cheapest model possible so that you're getting as much like the best quality, but without overpaying for it.
But then importantly, so, you know, that's context on the front half of making the automations actually work.
But on the back half, the automations platform integrates back into the context platform.
So all of those runs are capturing logs.
You can then create additional automations that review those logs and make improvements to the skill based on real-world usage.
And then, of course, it also integrates into the rest of our context platform where...
All of these automations are multiplayer by default.
The same way you can see all of the skills in your repo, you can see all of the automations that your team has created.
You can share them.
You can create permissions around them.
So that's a really exciting way that we're giving users more.
ability to use the skills they've created.
So that's a good kind of description of these different services that we build to it.
I guess the point I'd emphasize is we think every company should own its factory.
In fact, we think there will be a plurality of factories, right?
Like they will have factories for different repos or business units or sort of segments of it.
And we think a factory is something you should own, right?
Like it is your custom definition of what correct is.
It's the new software engineering.
And so that does not mean that you should build everything.
aspect of it.
And we think that our job as tool providers, as enablers of this agentic era development is to give you a bunch of these tools, to make it easy.
to make a bunch of these things simple.
And so a lot of these kind of components, they come out of portions that we think are reusable, whether they are around observability, whether they're around, of course, everything to do with context, the capturing of the context, the assessment of the context, et cetera, whether it's just sort of utilities like scheduling, like...
like access controls, like dashboards and visuals around what makes for an effective dashboard or an effective factory or not and what should you learn.
So our job is to really consolidate a lot of industry learnings and our own learnings and repeat work in various parts of the industry and capture it up.
And so I would say that our factory componentry is still...
relatively light.
There's a world that we still need to build over here.
And that is on purpose because, again, we sort of advance with our customers.
And many of them, they want to build their own factory.
Some of these things should really be custom-built components.
And we'll sort of help them build the componentry there.
But what we will emphasize maybe once more is that we think a factory should really be skills-centric, context-centric.
It should start from your definitions of what correct software development is.
And what we want to be is just like...
an SDLC platform that has a sequence of these different components to help you apply your development process.
This is what Tesla is to you to help you kind of build your factory and evolve.
And of course, as always, we're sort of there to kind of help you in the process itself, right?
Help you kind of learn and grow.
So a lot has happened.
We've been busy.
It's actually been sort of super exciting to have these sort of weekly reviews of what's coming this week of it.
It's like, okay, these teams, we have a very empowered team.
They've been building great components, working with customers on it.
And just like the next week, it's like, oh, yeah, that sort of big chunk over here that would have been maybe like a quarter's work or something like that.
It's done.
Okay, what do we do next?
It does feel like you can't blink, you know, which is sort of exciting and daunting at the same time.
So if you found all of this conversation interesting, we'd love for you to try the sort of the expanded TESOL of it.
Of course, you can always run still some TESOL skill review and the evals and a lot of these components that we've been building for a long while.
We're very excited.
And we're seeing a lot of uptake around the TESL agent and using it to build loops.
So we'd love that.
And then maybe try out thinking about using TESL as the platform to kind of build up a broader context-centric, if it wasn't, if we didn't say that enough, context-centric factory over it.
And, you know, as hopefully I think is evidence to anyone that has been tracking us, we're very, very community-oriented.
So we'd love...
if you try it out.
And we'd also love your perspective on the sort of philosophy that we're espousing here around our belief that you should build your factory gradually going from skills to loops to a factory.
And with that, thanks for tuning in and I hope you join us for the next one.
The AI Native Dev is not just a podcast, it's a community, and we host monthly meetups at the Tesla offices in central London.
Visit tesla.io forward slash community to learn more, and I hope to see you there.