# Feature Flags and Autonomous AI Release Management

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-11-10

## Transcript

I remember talking to a prospect and I asked them how often they released and they said once a quarter and I asked if they wanted to ship more and they said no.
Which now just seems absolutely jaw-dropping.
You know, Anthropic just came out with a blog and they talked about how they have moved from 2,500 changes to 30,000 changes in a day.
It's just a very different pace that needs a different sort of workflow for production.
If a deploy goes wrong at 2 a.m.
and it's rolled back within a second, did it really go wrong?
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
Join your hosts, Guy Pagiani, and me, Simon Maple, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
This is the AI Native Dev.
AI DevCon is coming back to London.
Our first London event sold out in June.
After our second New York event this November, we're heading back to the brewery Moorgate in 2027 with a bigger lineup and two full days of talks from the people actually building with agents.
You'll hear honest takes on where the industry's at, real stories from teams adopting AI, the wins as well as the messy bits, and where they think things are heading next.
It's on June 7th and 8th, and don't tell anyone, but we've got a really great deal for you.
Scan this QR code to grab an early bird ticket for just £100.
See you in London.
Hello, everyone.
Welcome back to the AI Native Dev.
Thanks for tuning in.
Today, we'll talk about, you know, what is even a release and why do you kind of manage these...
kind of insane pace of application deliveries that we have.
And we have a guest who's probably one of the best equipped people to deal with that, talking about sort of delivery and, you know, dynamic delivery and such of applications for quite a few years and also kind of a good friend, Edith Harbaugh, who is the founder and CEO of LaunchDarkly.
Edith, thanks for joining us here.
Yeah, it's great to be here in the virtual studio.
So I guess for starters, I'd love...
to kind of get your view a little bit of what does it mean to do this type of feature flag controls, release management, give us a little bit the right terminology at a pace that seemed or like makes the cloud era pace feel like a snail.
Yeah, so it's been a journey.
So when we started the company in 2014, it was when cloud was still kind of a new idea.
You know, a lot of people were actually still shipping on packet software.
I remember talking to a prospect and I asked them how often they released and they said once a quarter.
And I asked if they wanted to ship more and they said no.
Which now just seems absolutely jaw-dropping of that they thought once a quarter was more than sufficient.
Now, you know, Anthropic just came out with a blog and they talked about how they have moved from 2,500 changes to 30,000 changes in a day.
Like in a single day, which is inspiring and amazing, but it's just a very different pace that needs a different sort of workflow for production.
Yeah, yeah, no doubt.
But so how do you like, how do you think about sort of feature flag in this reality or about kind of managing behavior?
Because I feel today there's definitely an ethos of ship it.
You can monitor the reality of it.
So if there's a problem, you can roll back and you can iterate.
Clearly, all these things require the right infrastructure to be able to do as well.
But then when you think about this type of pace, oftentimes the decision of whether to roll back and all that takes longer than the time.
You already want to evolve your next feature, right?
You already want to build the next one.
Yeah, I mean, and that's really so...
That's really what LaunchDarkly and feature flags can help with.
And let me be clear, you don't need LaunchDarkly to use feature flags.
This is a technique.
We're a vendor.
I would like to think that we make it a lot easier.
But everything I'm talking about, you can do on your own.
So the concepts you just talked about are a rollback.
And to do that effectively, you do need feature flags.
So if you're looking at, even if you're not anthropic scale, but if you're looking at let's say 10 or 100 changes a day, you need a way where you can quickly detect if something is going right or wrong and move it back if it's going wrong.
And that's where, one, you could do it on your own if you had feature flags, but two, launch directly could really help you because you can have that as automated.
So if it's going bad in the field, you could within seconds say, hey, this isn't working and have it automatically rolled back.
And this is what AI is very good at because you don't need a human in that loop at all.
You don't need somebody to be sitting at a dashboard looking at it.
It could just be a very autonomous rollback.
And then you multiply that times 10, 100, 1,000.
And it's a very good failsafe if you're shipping a lot of code to production and you want not to have to babysit every single change.
Yeah.
So there's the technical aspect of figuring out how to sort of, you know, do the flag and sort of apply the change on it.
But, you know, hearing this, my kind of bigger question is, how do you decide that you need to roll back?
And if you do that agentically as well, what would you sort of say, we're sort of looking right now at sort of state of the art, maybe we'll talk a little bit about the journey, but if you're really good at it, how do you make those decisions?
So a lot of the work we're doing right now is to make that very automated.
so that you can send us whatever telemetry you have, whether it be error rate, whether it be elevated operational metrics, whether it be something as fuzzy as just an uptick in tickets.
It's a good question, though, but what is it that you should feature flag?
Because some things get you in trouble if you start getting a little bit too dependent on feature flags.
You want to roll them out gradually, but you end up creating inconsistent experiences, or when you decide to roll back, you kind of have...
I don't know, data structure problems, or you maybe are letting down some user who has now kind of formed some attachments, right, to the capability you just launched.
I guess what would you, sort of systems aside, like from the, you know, probably have more exposure to sort of correct and incorrect feature flags than almost anyone, like what's a, how do you define like a good feature flag criteria?
Well, so leaving aside launch directly as a vendor, I would say, A good one is always one that you understand.
If you have put in one and you don't understand what it does, that's where you get into a lot of trouble.
So I've seen people who have still flags, a flag that was once very important, but nobody understands what it does anymore.
And it actually accidentally get triggered or the worst case I heard is.
One company that had something to shed a lot of load and one feature flag, if you toggled it, it would actually prevent you from uploading any new files.
And it would actually get toggled and nobody knew what it was.
So the number one, I'd say, guideline is just make sure you understand that.
And then you can still have a human in the loop to make those judgment calls.
It's so interesting to think about judgment, right?
I feel...
when I look at how we build a TESOL and I look at the pace, alignment becomes this really kind of interesting challenge, right?
Because you have a bunch of people, they're individually empowered and just sort of so equipped, you know, like they have such powerful tools at their disposal so that each of them operates like, I don't know, a directorship, right, of sort of people.
But keeping them...
aligned as they make many of these decisions is hard because it takes the same amount of time to talk to their colleague and describe what it is that they're building.
And it's still a pain or a product discomfort if they're misaligned.
I'm trying to think about deploying capabilities with feature flags.
And so you add to that kind of matrix or to the already kind of existing risk of I'm not sure what my colleague has shipped, right, and we're trying to kind of reconcile this as we fly the plane, you add a potentially very long list of feature flags that further customize and sort of tweak behavior and you do that.
I guess if you...
When does this go wrong?
I don't know if there's magic.
I guess I'm sort of yearning for these sort of magic numbers of never have more than 100 feature flags.
I guess what are sort of guidelines to say, until this point, it's helpful, and from here on, it's kind of detrimental?
Again, I don't want to give too much of a vendor pitch, which means I'm about to give one.
I think the guideline is as many as you can safely manage in a system.
So a lot of small startups, I talk to them and they say, hey, we've rolled our own feature flagging system.
I say, wonderful.
That's great.
Like if you have like 10 to 20 or 40 and they're well understood within your org, you know, you don't really need a vendor.
It's saying when you get to a scope, when you have multiple teams who are perhaps distributed, you start to need more of a framework about managing them.
You might also have dependent flags, which is pretty common, where you have perhaps a flag that's handling five other subsystems where you want them to move in parallel.
And you also want to be really diligent about cleanup.
So where I've seen people get into trouble is if they've just been adding features for every release.
And there's always what I say, the boring part of doing the dishes and going back and really examining whether a flag can be used or not.
So I guess related to that a little bit, I think, I guess what I was also kind of wondering is whether the use of feature flags, and especially as you kind of grow them, depends, I mean, you sort of said like how much you understand them, but also about how well can you clarify the outcome.
And in that sense, I really sort of think a little bit about agents.
There's this...
comparable that I often run into, which is in DevOps, we were talking about all these great traits of you need to instrument your systems and you need to have good lineage and you need to manage your data and have understandable information architecture.
And then if you did all of those, agents can actually help you run faster.
And agents can also help you overcome a bunch of pains that you had and get your house in order.
But if you had a bunch of that infrastructure, those are actually now multiplied.
Like the agents really benefit from sort of having that data and that access.
And so I guess sort of similarly, I wonder if with the feature flags, you know, one of the questions is like, what's your judgment?
You know, what is the thing that you measure?
So I guess how much does that factor in?
Like we're in this world where a lot of things are ill-defined, you know, of what correct behavior is.
Like, does that get in the way?
I'm also kind of wondering about what you're sort of seeing in the wild.
there's kind of established businesses, but when you're building kind of newer capabilities, even within those established businesses, they don't necessarily know what good looks like, right?
Like what is it that the feature flag should aspire to achieve?
Well, that was actually, I think about four or five questions wrapped into one.
I've got a tendency of doing that.
I apologize.
Well, no, it was a really good question.
It's something that I've been thinking a lot about.
You know, so I originally got it.
I originally found it launched Darkly because I'd been, an engineer and then a product manager, and I had built things that just did not work.
You know, in terms of they worked at an engineering level, but then they got out into the field and customers were sometimes just completely apathetic to them.
You know, and I say apathetic deliberately because it wasn't they loved or hated a feature, it was just that they did not care.
They didn't care, yeah.
They didn't care at all.
I didn't need that.
And that's painful.
The other part of an engineer is that you put out something and it turns out that it breaks in the field, which is also painful.
You know, that you always either over-engineer or under-engineer, depending on where the minimum viable product spectrum you are.
So sometimes you think a lot of people use it and nobody uses it.
Sometimes you think you'll have very light workload and suddenly it's through the roof.
In this world of agents, it forces you to think about that.
You know, so I think about a year ago was the heyday of the one-shotter, you know, which we even tried internally at LaunchDarkly about, like, you know, the one-shot of refactor this part of our code base.
And that phrase works about as well as you think it might.
Which is...
Like to human, what seems very clear, which you might give to a team of humans of refactor the code base, they go away and refactor it to the agent.
They just have no idea what that means.
Refactor for what?
Like, what does that mean?
So what we found in our own development is that the dream of the one-shot agent that goes away and does the work is more like us trying to decompose work into smaller and smaller and smaller and smaller atomic units until we got to a point where we could point agents at different points, parts of the code base and say, okay, here are your tasks.
Like make sure that everything is commented out, make sure that an agent understands it, make sure that we're looking for bloat.
So it was being very specific about what we meant by refactor.
This is like defining what you actually want.
You need to express that.
What do you want and what good looks like?
So you give the right instruction and you're able to verify it as part of the software building in this case.
Yeah, and even add on to that, that agents do not know the concepts of good and bad.
These are abstract words.
No, what you think good looks like.
What is a legitimate outcome versus what is a failed outcome?
Yeah, so if you told an engineer, make sure your code looks good, there's usually a common understanding about what that word good means.
But an agent has no concept of it.
So all of this work that we were doing into defining work for agents is actually really good for a feature flag.
Because your original question was, well, how do you define what success looks like?
And it's like, well, you're defining it from the beginning for the agent.
Like you're saying, hey, this part of code needs to be performant to this standard.
It needs to load within this time.
And then those are very useful metrics that you can then push into a feature flag for when it's actually out into the world of, hey, all that stuff I told you up front about how I wanted this thing to behave, make sure it still behaves that way out of the real world.
Yeah, I think that makes a lot of sense.
And I feel...
Again, there's like a very common through line, I think, in everything that you deal with sort of delegating any type of work to agents, which is you have to do the hard part, which is describe what success looks like, what failure looks like.
guardrails that you want them to stay within, you know, like what are the lines, like what's this allowed, what's allowed.
And so you need to do, you know, what we oftentimes avoid and just saying like, we'll know it when we see it, you know, for humans, you know, now we need to actually codify that in the system.
How, I guess kind of pushing that a little bit to the extreme.
How far would you let agents evolve?
Like, you know, you're probably like the most advanced sort of user of feature flags or like I'm up there.
How much would you let them just experiment?
Like, hey, agents, like...
Like, look at all my telemetry and how people are using it.
Listen to all the customer calls that I've had, you know, like read all of my internal documentations.
Propose changes, roll them out if I'm being bold here.
Like, you know, build them, roll them out under feature flags, see what the result is, come up.
And, you know, I'll be surprised and amazed at the great enhancements that you've sort of built for my product.
I mean, that's the dream.
I'd say we are not quite there, but we're moving there.
We have a pretty strong internal blogging culture at LaunchDarkly.
So we have roughly 200 people in R&D, and I encourage them to write, and I try to read them.
And I was reading one this morning about how they built a fleet of about 20 plus agents to work on a part of the code base.
And then the work of the engineer actually turned out to be just reviewing all the PRs.
So in my mind, they had transitioned more to what used to be called the classic architect role.
I'm old enough where you used to have the architects that would just do code review.
So there was still a human in the loop.
There was still some judgment, but it just went much faster because somebody else was doing actual code writing.
And then when this stuff is ready to go out to production, it already has guardrails pre-built in because that's part of the processes of before you change the code, figure out what the guard rows are.
So it is much more autonomous.
We've also done projects where we pointed an agent at a backlog of bugs.
And I think this is just amazing, where you can point it at a backlog of bugs and it could produce PRs, you know, for sometimes months, if not years, of things that never bubbled to the top of fixing and just start releasing them with guard rows.
And I love this because I think, as I said, I started off in engineering, I remember just the tedium of sitting in a bug review.
Did you ever see a bug review or just like, hey, it doesn't work on this browser, there's this cosmetic flaw, is this worth fixing or not?
And you just spend hours on that tedium versus an agent just goes through all of them.
I don't know if you're seeing the same things in your own development.
We're definitely sort of, you know, definitely many, many, many things are building there.
But I guess I want to then sort of push a little bit on, like, why not sort of drive the autonomy?
So, like, what's getting in the way?
Because, like, I guess my view is that...
The feature flags and sort of this type of kind of runtime management should enable you actually to release stuff that is bad.
But it's okay because you're like gradually exposing people to them.
And to the extent that you have enough of a user base.
And yes, there's probably some constraints around like some areas in which your tolerance for a mistake is very low.
Like, why wouldn't you, like, why do you need the human in the loop if you have the sort of the feature flag capabilities with sort of, you know, tight management and are able to gauge?
Like, if you have these sort of two, you have these controls and you have, like, a pretty good understanding of outcomes.
Oh, I mean, we're, depending on the feature, we basically do that.
Like, if it's a bug that's coming in on UI, we can fix it within minutes.
It's very little human there.
For things with a bigger surface area, there is a lot more human judgment.
Yeah.
But I'm curious.
I guess what would make you unleash this?
Sort of say, hey, in this area of the product, autonomously improve my product.
So I like that you're pushing me.
I think the question of autonomously improve is pretty broad.
So let me give you a real example.
Like we have SOC compliance, we have HIPAA compliance.
So just say autonomously approve is so broad that I don't know if an agent would understand that right now.
Yeah.
Yeah.
It's interesting to...
So I think compliance is definitely a legit answer.
It's like hopefully a temporary answer, as in, you know, we should learn how to sort of build compliance systems that still have sufficient guardrails that they can evolve autonomously.
But I think today that is still a scary proposition.
You have to be very confident in the sort of guardrails that you have to be able to attest to an auditor that you're doing in that fashion.
I guess I'm sort of pushing on it just because I feel...
uh there's like different parts of it right there's the judgment of uh of what is a reasonable change i think agents are actually already pretty good at that especially if you're sort of scoping kind of the volume of change for a proposed change they don't need necessarily know that it's great but they'll have good kind of betting stats uh over here uh then the implementation okay they're getting pretty pretty good of that you know, like being able to like build the capability.
And then, you know, when you release it, there isn't a taste element, right?
Or like a curation element.
And, you know, one way I think about feature flags is just, can they replace taste?
You know, I don't know, like with ads, with sort of coupons to supermarkets, with doing it, like, I don't know, a lot of taste goes away in favor of data, right?
Oh, I see the same future.
I mean, I would like to get there.
I think also, though, people like predictability.
So a real example is, you know, we have customers who are in education or airlines, and they do not want all of their interfaces to change on a moment-by-moment basis.
They're like, you know, the airline clerk, which who knows, might go away.
Somebody who's sitting at an interface and is like, I want to push this button, know that the button is in the same place as it was yesterday.
They don't want that rate of change.
However, there are things on the back end that can be constantly iterating much faster just to improve that front end.
Something we've been talking about internally is LaunchDarkly, the UI is all API driven.
Making it even easier for people to just have more of a conversational interface with LaunchDarkly, where instead of using our, for lack of a better word, canned UIs, that you could be building UIs on the fly with an agent every time you want to talk with us, which I think is great.
But I think some people want the familiarity of, I know where everything is.
This part is like the slow change of the feature.
For some parts of the system, stability really is the feature.
So let me kind of shift gears a little bit.
I'm very excited by the notion of dynamic behavior and how do you control it.
I have all sorts of thoughts that probably will sort of suck us down the wrong rabbit hole over here around personalized software and the likes.
How do you think about...
this type of delivery versus or like as part of the software factory is it the same is it the same thing like you build and maybe i'll i'll stabilize a little bit to say like let's assume the core of the software factory is this like tickets to pr but then the sort of the breadth of it thinks about the sdlc as a whole like how do you think about feature flags as a whole but also maybe like launch darkly's place right in in that flow yeah so It's interesting the software factory metaphor has really taken off, I'd say, in the last nine months.
I personally like it, though I know there's a little bit of backlash against it.
I like it in that a factory seems more predictable.
It seems like there are gates and steps and repeatability.
I know that some engineers fear that it makes them obsolete or just more rote.
In my own vision, I think it frees them to be more creative, knowing that there is stuff that they can depend on.
So that knowing, for example, that you could go, to extend the factory metaphor, that you could go somewhere and heat treat your plastic so that it's usable is actually a good thing rather than having to create that heat treater every time.
So when I think about LaunchDarkly, we assume that there's a lot of stuff upstream from us about how you produce code.
Like we don't want to compete with a clod or, you know.
And then we also assume that you have some sort of code review facility.
We don't want to compete with a code rabbit, for example.
What we're really good at is that last step of you've built something, you want to release it to the world, and you want to have guardrails around it so that once it's out in the world, the things we talked about before, you can either break up who it goes to.
So, for example, if you want to do a phased release where it goes to people in London and then in Scotland and then to the rest of the world, you can do that.
And also, so if something is suboptimal, you can roll it back.
Also, if you're getting a lot of positive feedback, you can feed that very quickly, perhaps even to a coding agent to enhance a feature.
So, for example, if you built a simple workflow and it's getting a lot of use, Maybe in the future, an agent could say, hey, this is getting a lot of use.
Let's continue iterating on it autonomously without even a human in loop.
So that's kind of the vision.
What we're doing a lot at LaunchDarkly right now as a vendor, and what I said before is vendor independent, is building up agentic controls where you can tune your system about what does and does not need it to be flagged so that you can have a risk tolerance.
for this is what I want to control, this is what a change I could just go through, but it gets smarter over time, so it's just auto-magic.
And you can use that as, hey, if any of these measures takes a jump, start rolling back.
And you could do that in an automated fashion without a human ever knowing something went wrong.
We call it internally the two, if a deploy goes wrong at 2 a.m.
and it's rolled back within a second, did it really go wrong?
And then on top of that, what we're adding to LaunchDarkly as a vendor is a lot of automation around making it a lot easier to know when to feature flag.
So we've been using our own system for 10 plus years.
And there is, I would say, some amount of friction about like you have to wrap a flag, you have to send this telemetry.
But in an agentic world, the agent can do all that for you.
Like you could say, hey, instead of having to look at, hey, I want to wrap this flag.
Let me think about what signals to send to it.
You just say, agent, do this for me.
And it's very automated.
Hey, everyone.
Hope you're enjoying the episode so far.
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
Whether that's talking about the latest tools, the most efficient workflows or defining best practices.
But for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
So I feel I think there's a lot of kind of value, right, in sort of that portion of it.
But I do, I want to sort of...
geek out a little bit about sort of product scope in this era because I think that the thought about a software factory is just about like how all-encompassing it is, right?
And I had quite a few months ago now, I had Meryl from Graphite on the podcast and this was before the cursor acquisition and, you know, we talked about whether code review will be, you know, a part of the factory, right?
And I feel...
I guess I would claim today that the answer is yes, Scud Review is a part of the software factory.
In fact, if you don't have kind of increasing autonomy, you know, with sort of growing confidence and competence to be able to merge things more and more, you know, autonomously, then you're stuck, you know, like that becomes the bottleneck and you're just not going to be able to push through it.
And also in that conversation, we talked about how...
At the time they built their first features to say, hey, when you found a bug in the code review of it, then hey, you can actually sort of fix it.
over here in the path.
It's like, well, how are you different than the coding agent?
I don't know, by the way, if the cursor conversation was already happening in the background and that happened, but how are you fixing there?
And then can't I just run your review as part of that coding agent?
Why do I need to wait for the PR?
Can I run it early?
And so I guess I'm sort of saying all of this because how do you think about, does that not extend?
If you think about...
feature flags and this type of dynamic insights, you can think of it as a continuation of quality.
You can look at things statically, you can look at dynamic tests on it, and then you can look at actual real-world experience of this capability that you're releasing it.
Does it make sense eventually?
What's your vision or prediction around the world in five years' time in terms of division of...
core capability providers, right, you know, in this space.
I think there will still be the last mile issue.
Like, when you were talking about the factory metaphor, I was thinking about how there's always a bottleneck somewhere.
Yeah.
And as soon as you fix a bottleneck in one place, the bottlenecks move somewhere else.
The next thing comes along, yeah.
You know, so the bottleneck, I'd say, for most of my career has been just code.
You know, how many engineers can we hire?
How fast can they get up to speed?
And then there's all sorts of artificial metrics of productivity that were gamed, like lines of code.
I don't actually care about lines of code.
I care about productive code.
Now code is incredibly cheap.
Like you can ask Claude to make you code and it will make far more code than you care to.
And then the bottleneck was code review.
And I think there's a lot of very good vendors right now for code review.
And then the bottleneck right now for me is there's a lot of code that's been code reviewed and there's still this last mile of, is it actually having a productive impact on the end person?
And I say person, it could be a system.
It doesn't have to be a person, but that gauge of, is this doing something that I want?
And I think.
Is it kind of properly improving whatever is those business metrics that I've sort of defined as good?
Yeah, and I think a change from the world in the past, because I talked earlier about the age of package releases, is you used not to be able to get that telemetry.
Like if you were shipping, like, let's say, I'm old enough to remember Windows 95, like it came as this immense 32-disc blob.
Where it's impossible to say, hey, is this feature better than this feature?
You just couldn't know.
But now with the ability to break down changes into smaller and smaller bundles, you can actually have a really good gauge of, is this working?
But you also want to be able to say, hey, all these changes together, is this having a holistic good effect?
Yeah.
I think the management of it is quite complex, and I feel like there's a lot of...
industry learnings.
I guess when I think about software factories, I feel on one hand, I feel the software factory foundation will be all encompassing as in it will, it will like eventually you want to aspire to a thing that is the full loop.
that goes all the way.
And I will say that many years ago, already, various tech leaders that I spoke to spoke about their R&D organization as a software factory, which may or may not have been endearing to their teams.
But it was still about a repeatable process of software generation that goes end-to-end.
I do think within that, there will be sort of worlds of specialization for sort of difficult parts of it, for security, you know, for, you know, for various aspects of sort of quality.
And I think very much likely that sort of Feature Flags is its own domain.
But it's just sort of interesting to think about scope products kind of in that sort of destination.
Yeah.
And something I want to emphasize is that you don't want to lose the...
The forest for leafs, if that makes sense.
Like you can definitely over optimize for very narrow features and lose track of this isn't actually product anymore.
And we've over optimized.
An example I'll give is, you know, you can realize that you've backed yourself into a niche product, that you have a very passionate bunch of loyalties, but you're not one of the broader market.
So I worry that.
The worry is perhaps too strong, but I worry that with the rate of change that companies will back themselves into a quarter if they don't have some sort of level of judgment and product management about like, hey, we've over-optimized for 50 people in London who love this one feature, but the rest of the world is ambivalent.
Yeah, I love that.
So feature flags are amazing, but they don't replace your judgment.
They just sort of enable you to kind of apply it more wisely.
You still need to do that.
And I agree you'll kind of, to carry the risk of sort of hitting a local maxima, you know, for sort of optimizing for it and not really having, you have to have some vision.
And I guess that comes back to like accountability because if you're going to make a bold decision, even a medium bold decision, but one that is not sort of purely in the data, but rather your sort of analysis and sort of intuition on it.
there probably needs to be a human signing off on that type of change.
Maybe if you're very, very big and very, very confident and have a very, very fast learning cycle, whatever the TikTok algorithm can probably make more of these wild bets.
Yeah.
Yeah, I was thinking about it because I was a product manager at TripIt, which was at the time in the 2010s, the number one mobile app on the Apple iOS store.
And it was a self-fulfilling prophecy.
We were the number one app.
So people would go to the app store and download us.
We were the number one app, which was a great place to be.
But the founders of the company, I wasn't a founder, by the way, I was product, had made this really good decision that they focused on Apple early.
And one of their big competitors focused on BlackBerry.
And I don't know if that was something that an AI could ever decide.
Make the choice.
They doubled and tripled down on we want to be the dominant app on Apple.
And then Apple expanded versus their competitor who had this shrinking market because people at the time, there was a time when BlackBerry had more market share.
It wasn't sustainable.
I think it would actually be difficult to buy a BlackBerry right now.
I believe.
I think I agree.
I have this sort of blog post brewing in my head, which I'm probably never going to write, that the greatest advantage of humanity is irrationality.
It's sort of this desire or this willingness to say, no, no, this is right, against all the data.
And this is how all great companies, all great...
pieces of art have been created.
So it's not really about whether an LLM or an agent would be able to conceive of this idea.
They'd be able to conceive of all ideas.
It's about this sort of like irrational dedication.
which typically fails, but then for a subset of people succeeds, you know, a subset of cases succeeds because of that.
And sometimes like actually changes reality, right?
Like I think the work you've done with LaunchDarkly, I'd like to think the work we've done at Sneak, you know, from a DevSec perspective, you know, they don't just roll into the market.
It's the sort of persistence actually helps shape some aspect of the market.
Yeah, so I started LaunchDarkly in 2014 with my co-founder.
who'd been at Elasi and I'd been at Tripit.
And we both had this strong conviction that we did not want to have an on-prem version.
And it was because I had been the product manager and program manager for supporter platform Matrix at Vignette.
And I was like, this is hell.
Yeah, you had the scars.
You're sorting out databases, you're sorting out app servers.
I was like, don't want to do that again.
We're SaaS.
And my co-founder was the same.
And at the time, it was very contrarian.
Like, I remember we were trying to sell to customers, and A, we were trying to sell them on fast releases, which some of them just didn't want at all.
And then some of them were like, well, you know, we would never use a vendor for this that we cannot install.
But we held firm to that conviction that though we could have gotten more sales, it would have been a quagmire of pain to have an installed version.
Yeah, I wish we had that much conviction at Snyk.
We kind of went back and forth, back and forth on it.
But, you know, I'm not even sure what the status is, but I'm pretty sure today there's on-prem components, but there isn't an on-prem system.
Let me kind of shift us a little bit to add a degree of freedom, if you will, on it, which is runtime agents.
Up until now, we're talking about...
software that runs in runtime, yes, it is modifying faster.
And, you know, by using feature flags, you can try out capabilities within there and, you know, allow it to change and learn even faster.
Is there such a thing as feature flags for agents?
Like when you're, as a portion of your...
product becomes instead of software you deliver a bunch of that value in the form of agents but what is the equivalent of feature flags in that world so i feel like you're feeding me marketing lines so um we do have a product called agent control and it is where you can put guardrails around an agent and put in parameters so that if your models drift It can either shut off the agent completely or just dial it down so that you can have the ability to control an agent in production without a redeploy.
And this will be like, how is that different to just sort of the core management of the agent or like how people run their agent without LodgeDarkly, right?
So we give you a layer where you can easily switch models or even switch the prompts without having to do a redeploy.
So you can do it at runtime.
And it's again like one of these folks in London get one sort of prompt versus people else who are getting another and sort of seeing what happens.
Correct.
Or monitoring for errors.
So if you have, for example, a chatbot that suddenly is getting a lot of customer complaints.
you can dial it back or switch models.
Or if it's suddenly burning a lot of tokens, you can switch between models on the fly.
Who do you think is the primary user for the runtime control of the agent?
And is that different?
Is it still about managing product functionality that's available to people?
Or is it about actually developing the prompts and the choices of models?
Almost the more...
kind of implementation.
It's the same goal, but it's a different, it's a change of the instrumentation that makes it happen.
I think people are still figuring out what agents are.
So I think within LaunchDarkly, we have both very ephemeral agents and long-running agents.
So I'll use an ephemeral agent as an example of, you know, we're working on a project to redevelop part of the code base.
We spin up.
you know, 20, 30 agents to work on it, and they are specific to this specific task.
We have other, I'd say, kind of long-running agents.
Like, for example, we have an agent called Vega, which is constantly doing flag cleanup on everything that we produce.
We, like everybody else, didn't always practice what we preached, and we had some stale flags, you know, some dating back, you know, to when we were still a startup at Heavybit.
In this agentic world, we now have agents that just go around and examine every flag and say, is this still in use?
Is it still needed?
And if not, safely delete it.
And that's just really fun.
I think that's a real best practice that AI has made a lot easier.
So I think these are actually pretty different use cases.
And I would say, I think right now, I think the industry as a whole is figuring out what the right word, what the right use cases are.
To use some consumer use cases, it's funny because I come from a travel background because I was at TripIt.
Right now there's a lot of chatter about an agent that will book your next vacation.
Okay, that's a one-time use case.
If I'm trying to figure out how to go to Atlanta to see my aunt, that's a specifically defined project versus a long-running agent of, look at New York Times cooking recipes every week.
and buy the groceries so I can make the most top three popular ones.
Yeah, yeah.
Interesting.
Okay, so even as you think, so I guess in that context, when you do that type of runtime controls, you come back to, you need a person, like they need to figure out what is the question they're asking when they're making a modification, because the sort of the mental model of what is it trying to do, what is the context in which it runs, once again, what is success, not even for the product as a whole, but for this experiment.
that you're doing here, all of those things need to be pretty clear in your mind and then subsequently in the instructions.
Yeah, and to go back to the cooking agent, because I think it's a good example, if the agent went rogue and said, instead of the top three recipes, I'm going to choose the top 10 recipes, so that's better.
It's like, well, that's actually way too much food for me.
So I think we're sort of running out of time here.
I've got like five more questions on it, but I'll ask one.
Or maybe I'll ask one of those questions that is five in one, and we'll sort of, we'll find out.
So you've built, you know, you founded Non-Starkly, even before I founded Sneak, you know, like I hear before on it, you know, you've been, you've just sort of built in the cloud era.
You had some time away from the CEO seat, you know, for a couple of years, you came back to...
to kind of, I don't want to say reinvent, but like adapt the company to the AI era.
What would you say is different about running a company in, you know, a decade ago, you know, and like what is needed to make things successful versus today?
Oh, gosh.
I mean, I could talk about that for hours.
The company 10 years ago, I mean, we were a tiny startup.
You know, we were, you know, eight people at Heavybit.
the DevTool incubator and now we're 600 people worldwide.
And we have thousands of happy customers that I'm proud of.
I'd say the big change right now is that a lot of established software principles are up for grabs, but yet still the same.
So like some of the things we've talked about in terms of, you know, moving the bottleneck within the software development lifecycle is it's very different.
But I think the core belief in that our job as a company is to deliver value to our customers and to empower our employees hasn't changed at all.
Do you think, is there something different you look for in a team, in a higher, maybe don't sort of look quite at the sort of the 14 percent, but try to sort of imagine a 600 person company a decade ago or like maybe even somewhere along the way versus now?
Like, are there different aptitudes that you look for in a developer on the team now versus before?
So we just republished our values, and a core one that's been constant is the ability to learn and grow.
So I really encourage the team to always think about how they could be doing things different and better.
And I think AI has just made that even more fun.
You know, so we were having a discussion about, you know, should we encourage people to take smart bets?
And I said, I would like to encourage them to take some dumb bets too.
You know, if you're only taking smart bets, you're not taking enough bets.
If you're taking stuff where you're like, hey, I'm not really sure if this is going to work out.
Like that's the edge where I want people to be and not in a silly way of like, hey, I know that if I push this button, it's going to take down production.
Please don't do that.
But like, hey, this feature with AI is a lot easier to build.
And so we can cut off some of the debate about whether or not it's worth building because it's actually faster to just build it than have that debate.
But that's super fun.
Yeah.
Yeah, I love that.
And I agree with that.
Like adaptability is the name of the game.
And I guess a part of it is even how...
both what you build and how you build is all but guaranteed to be different in a year's time, which is not the reality we had before.
We had things that would change over a decade, not over a quarter.
Yeah, and I think even about how we're talking about AI in the spring, now it's October and we're using it differently.
The frontier models have come out with different things.
We've changed how we approach it.
And we know that this will continue to change.
And some people find this terrifying.
I find it really exhilarating.
I'm like, this is cool.
Yeah, you're sort of an outcome and kind of creativity-oriented.
And it's interesting to think about the people that are more craft-oriented.
They feel like someone's taking away their craft.
And those who are more problem-solving or outcome-oriented, I think, get more satisfaction out of it.
To go back to the factory model, I think of it as moving from perhaps a home workshop where you have some limited tools to full makerspace where you have access to all of a sudden all these things that can help you build.
Yeah, yeah, lots of excitement in that.
Edith, this has been a great conversation.
Thanks for the time here.
Excited to see how you continue to sort of build and adapt and so much more to build here.
So thanks for tuning in.
Yeah, thank you.
It was great to spend some time with you.
And thanks everyone for listening in and I hope you join us for the next one.
The AI Native Dev is brought to you by TESL, the package manager for skills and context.
Your hosts are Guy Pajani and me, Simon Maple.
Our producer is Tom Dowler.
The AI Native Dev is not just a podcast, it's a community and we host monthly meetups at the TESL offices in central London.
Visit tesl.io forward slash community to learn more and I hope to see you there.