# Agentic Engineering: CI/CD, Context, and Open Models

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-04-28

## Transcript

So much software is yet to be produced that we now have the option to do because implementation is so cheap.
I am talking about why CI-CD is dead.
When you think about how engineers are using traditional CI-CD tools, those were built for humans pre-cloud, and you get this massive volume stack in PRs that we never saw with humans.
Now we're seeing about 1,300 pull requests without human assistance be generated in land per week.
You can't have so many different agents taking versions of the same code and merging them.
The AI Native Dev is a podcast for developers and engineering leads at the cutting edge of AI and agentic coding.
Join your hosts, Guy Fajani, and me, Simon Mayful, every week as we chat with the most exciting voices in AI and tackle the biggest questions facing developers today.
This is the AI Native Dev.
Back in November, we hosted the first ever in-person AI Native DevCon in New York.
This June 1st and 2nd, we're bringing it to London.
It's two days built for AI Native developers and engineering teams.
One day full of hands-on workshops and one day full of practical talks on agent skills, context engineering, agent orchestration and enablement platforms, and how teams are actually shipping AI in production.
Join us at the brewery in London near the Barbican for all of that, plus networking, parties, giveaways, and a room full of people building the future of AI native development.
You can also join us from anywhere in the world via the live stream.
As you're listening to this podcast, you get 30% off your ticket with code POD30.
Just head to ainativedevcon.io and we'll see you in London.
Hi there, Simon Maple here.
And on today's episode of the AI Native Dev Podcast, We're here at AI Engineer London.
This is the European event for AI Engineer.
Probably one of the most premium events here in the AI Engineer space.
So just take a look at the sheer number of sponsors and the quality of the folks that are going to be here in the Expo Hall.
And the Expo Hall, of course, is right behind me here.
We have a whole bunch of wonderful people.
We're here to talk to attendees as well as folks who are here speaking and also on the Expo floor.
We'll hear about what's new, what's happening in the keynote, and also what people are here to take away and learn.
Let's go in.
So joining me is Steve Kuliski, an engineer at Stripe.
Steve, how are you?
Good, thank you for having me.
Pleasure, enjoying an AI engineer so far.
Yeah, I love it.
It's really awesome to just, like, you're online so much reading about what's happening, but to actually see the people here is super cool.
And what are the people here like, right?
Because this is one of the premier events in the AI engineering space, AI development space.
Do you get a higher class, a higher level, higher levels of adoption of...
developers using AI in their environments?
What kind of conversations are you having?
I think what's interesting is like Every day there's a new announcement about some new models or new technology.
So everyone here is really operating at the bleeding edge.
Everyone here is really trying everything the second it comes out, everything that's new.
So it's really interesting to get the frontier perspective of how people think about new tools and new technologies and so on.
So it's really exciting.
Awesome, awesome.
So tell us a little bit about Stripe and how Stripe is adopting and using AI in your product.
Yeah, sure.
So Stripe is using AI in a few ways.
So helping build Stripe.
helping our customers build their Stripe integrations, and then also generally providing economic tools to agents and things like that.
So I can go through all three quickly.
So the first one is how we're building Stripe with AI.
So we have an internal coding agent called Minions.
And it's sort of like other coding agents you might have seen, but it's very tuned for Stripe specifically.
So it's a model and an agent?
Yeah, so we're using common models like Claude and OpenAI and so on.
Our harness is forked off of Blox Goose, and then we run it internally within our pre-existing development environment.
So, maybe I see a Jira ticket come in or feedback from a customer.
I can click a single emoji in Slack.
It'll spin up an entire replica of Stripe, as I sort of normally would have built, but it'll now run an agent loop off of that prompt, and then it will use all of our internal docs and tools and CI and so on to try to resolve that prompt.
So now we're seeing about 1,300 pull requests.
without human assistance be generated in land per week, where the only intervention is sort of the typical code review process.
So is there still human?
Yeah, we're still reviewing all the code, yes.
Do you think they'll get to a stage where human won't be in the loop for a bunch of these, like lower priority, less critical?
Yeah, so I think there are certain changes where there's not a business logic effect, right?
So you can imagine something as simple as a typo on the docs.
in general, might already be something you'd be comfortable with landing today already.
So I assume we'll start that base case.
But then increasingly we'll move up towards features that have a monetary impact, where the sort of risk-reward blend of it landing faster is probably not in excess of having a human reviewer to double-check that it's right.
So I think there'll be a balance where some things land automatically.
It's sort of the same way even with GitHub today, with Dependabot doing upgrades and checking out CI.
you could probably just let that land.
Things that you're happy to accept because it's low risk.
But then on the flip side, you might not want that to land, right?
You saw the Axios and all those other kind of security things happened recently where you might not want an automated package update to go through.
So it'll be really interesting to see how that plays out, but I don't think anytime soon.
Yeah, yeah.
And I'd love to hear a little bit more about the environment that you use there to build within Stripe.
How important is context as part of that whole structure?
Oh, I mean, it's everything, right?
in the literal context, Stripe's been around for 10 plus years.
There's 10 plus years of context that doesn't exclusively live in the code base.
So any kind of integration with a payment method or a card network or some sort of business logic around how we treat funds, that has a legal and compliance and all these other kind of contextualizations that we try to manifest in the code base, but also live in documentation and live in our knowledge base and so on and so forth.
So to be a human engineer at Stripe, I have to read all that content.
So if we want our agents to be able to code effectively, it needs to have that context as well.
And how do you manage context?
Because obviously there's so much of it, and it's not just, I guess, context about the code, but also, I guess, your policies, your ways of working, how you want codes to be written, and the CICD process, all that kind of stuff.
How do you manage all of that?
Because there must be so much.
Yeah, so I think this is part of one of the advantages of us having built this kind of thing at home, right, which is, you know, The engineering you do at Stripe obviously looks different than the engineering you do at another company or at a startup or a totally different business domain.
So maybe 80% of coding at Stripe follows some blessed paths, right?
Maybe I'm making an API change or change to documentation or change to introduce a new currency or so on and so forth.
And because we can sort of pattern match to one of those pathways, we can make sure that we're just introducing the appropriate context to really nail that.
So the context you need to make a docs change looks a lot different than the context you need to make a cross-border, money transfer kind of thing.
So us understanding the type of products we're building and the type of changes we're making lets us thoughtfully put in context in advance.
Instead of saying, hey, here's the entire thing.
Good luck.
Which would be impossible for a normal person to do.
Awesome.
Let's wrap up, but I want to ask you one final question.
What is it you want to learn here at AI Engineer this week?
For me, obviously there are a lot of people who are Stripe users, so I'm excited to talk to them just to get feedback so we can make our product better.
But I also think to that first point, everyone's working at the bleeding edge, and it's impossible to keep up.
So being able to hear different perspectives and how people are thinking about the tools that are coming out and how we can best use them in those kinds of conversations seems very useful.
Amazing.
It's been wonderful chatting, and I hope you enjoy the rest of AI Engineer.
Thank you so much.
Appreciate it.
Thanks.
Well, we're lucky enough to chat with Madison Fortner, one of the speakers at AI Engineer here in London.
Madison, welcome.
Thank you so much.
Happy to be here.
So tell us first of all a little bit about who you are, what you do, who you work for.
Yeah, so I am an investor.
I'm a partner at NEA, which is one of the biggest and oldest VC firms.
We do have a London office, but I am based in San Francisco.
Nice.
Although I do have a lot of investments in Europe.
And I actually am a former AI researcher.
I used to work at Meta and led data and AI teams.
So I have a little bit of a unique background that more aligns with the people here versus venture.
So yeah, very excited to be here.
Awesome.
And what is it you're talking about here at the conference?
Yes.
So I am talking about why CICD is dead.
Oh, my God.
That's actually the reaction I did on this video.
But, no, it's a really important next category for us to think about in agentic software development because today everyone's thinking about agent-native engineering.
I mean, I actually have an investment in that space that's really exciting called Factory.
They're also speaking here this week as well.
And when you think about how engineers are using traditional CICD tools, Those were built for humans pre-cloud and you get this massive volume stack in PRs that we never saw with humans.
Humans were submitting two, maybe three diffs a week and now it's just thousands, maybe tens of thousands and you start to get to a place where your code base is merging them is just, it's impossible.
You can't have so many different agents taking versions of the same code and merging them.
So we really need it to be more inference native.
So it's almost like, is it replacing our existing code-based kind of software?
I think that's where we're headed.
But I think right now it doesn't quite look like that.
Right now, I think the best approach is to have a complementary layer where you are speeding up the build times, development times.
There's a company called Namespace that I invented.
invested in and we're actually speaking together on this.
And namespace helps to speed up your GitHub action builds, Docker, anything in that ecosystem so that you don't have to be sitting there waiting for your agents to deploy.
But I do think over time this will likely replace GitHub into other CI CD tools that are just not built for this infra.
It's interesting as well because obviously it's such a...
It's such an important part to GitHub's business, to GitHub code being such an important thing and those workflows.
It's going to be very interesting to see how or if They're even able to change fast enough to...
Well, you know, there's a lot happening at Microsoft these days.
If someone takes a look, we've been seeing a lot of reorgs, a lot of leadership from GitHub are leaving, like Thomas Domkies.
Yeah, we had him on the podcast recently, actually.
And I'm an angel investor in Entire, so a big believer in that.
Entire's a...
I mean, that's going straight out and saying, we are going to be the new GitHub.
Which entirely makes sense.
I mean, Microsoft's had the opportunity for a long time, and they haven't...
run with it.
So, like, why not have a new tool that actually is built for the cloud, built for agents, capturing new code and creating a better developer experience?
You know, GitHub has been the friend of developers for forever, but it's not that user-friendly.
It's not that agent-friendly.
Particularly with the agent friendliness, right?
It's the agent experience.
And then Harness, too, right?
Harness, CircleCI, I mean, GitLab, like, so many of these tools are just not really scaling to agent-native software.
So let's go through a bit of your portfolio.
Let's hear what would you say is the startup that you're most excited about that people have never heard about?
Okay, love that.
So, I mean, I have to give a shout out to namespace, but I already talked about them.
That is probably one that has the least maybe brand awareness, but so much developer love.
I mean, Warp is built on top of them.
Sierra, Decagon, OpenAI is a customer.
I mean, they have so much love.
But I'll put that aside.
What's the one that no one has heard about yet?
I would say probably Golden Analytics.
This is actually super interesting because Golden is just launched yesterday.
It's a stealth.
company and the idea is to bring the golden era of data and analytics to builders.
So this is the first tool that was truly created to be end-to-end data autonomy for everyone versus just the data team.
And they're really trying to blend data autonomy, as I was mentioning, analytics and semantic understanding of your data.
But then the third piece, which I think is the most compelling, is design and storytelling.
So how do you actually ask questions of your data and get full end-to-end stories where your data is being, you're finding the right data versus just using the data you have.
So I think it's going to be a really big unlock because I'm very bullish on...
structured data getting its moment in the vibe coding era.
And that name again, sorry, Golden Analytics.
Golden Analytics.
Just launched yesterday.
Check it out.
Amazing.
Really cool products.
They have a big wait list.
Yeah, yeah.
Excellent.
Sounds good.
What are you here at AI Engineer to learn about?
What are the things that you want to unlock?
Yeah.
So, I mean, I'm always thinking about, as I was mentioning before, kind of category foresight, where...
When you look forward in one to two years, does the existing infrastructure start to crack?
And that's where I want to be understanding both for my portfolio companies, but also for the enterprise heads of data and heads of engineering I work very closely with.
So I'm looking for where is this not going to work?
Where are things going to break?
Who are the thought leaders of that new era?
And I mean, just to get deep into some of it, I think foundation models for tabular data are an area I'm very interested in.
Domain specific foundation models.
Harness engineering, I think, is already a huge topic, but just trying to understand how that sits in the ecosystem alongside OpenAI and Anthropic.
Do you think when talking about those foundational models, do you think, do you see, obviously the billion dollar question a while back was around general models versus specific models.
And we saw companies like Poolside, a little bit quieter these days, but Poolside, you know, they were creative, they were putting huge investment in this, massive, massive amounts of money that they were valued at.
Do you see that now as the unlock, particularly with Mythos and things like that being announced, there's like killer models which have been trained very specifically.
Do you see general models fading away for specific?
I think general models will always be very helpful for orchestration and routing to other models.
And maybe that looks like mixture of experts where there's certain expertise on small models.
That's actually a part of my bet with the tabular data space.
For example, I think data is a really good example of one where foundation models will not be able to come in and build precise structured data.
But the tabular data architecture might be one that a large language model is able to route to.
So overall, I'm actually very bullish on small specialized models for different domains.
I don't think coding is necessarily one that will be in that specialized domain, but I do think there's a big opportunity for harness engineering there.
Some of the more specialized areas I'm excited about are material science, not possible with a large language model.
We have a company called Cusp AI that's actually based in London.
So sovereignty and language specific models, so like reflection in the US, Sakana in Japan, Mistral in Europe, I think are very interesting.
And then we have another one that's a foundation model for CAD.
So think about mechanical engineering and product design.
Very difficult problem.
This is not something you can just release a large language model or even a coding model to do.
But it's very precise for that use case that's built for the multimodality.
So I definitely think there are huge markets that haven't even been touched by AI yet.
with small specialized models, we're being very thoughtful about betting in a lot of those categories right now.
It's been wonderful chatting with you, Madison.
Thanks for having me.
Enjoy the rest of AI Engineer and good luck with the session.
Thank you.
Thank you.
We're here with Omar Sanseviero, who is the Developer Experience Lead at Google DeepMind.
Sounds amazing.
Tell us a little bit about what the role is and what you do at Google.
Yeah, of course.
So I lead all of the model launches at DeepMind.
So Nano Banana, Bio, Gemini and of course Gemma.
So last week we released Gemma 4, which is the latest addition to the open model family of models.
Amazing.
And tell us a little bit about what each model is trying to address.
Why would someone use one of those specific models?
Yeah, of course.
So there are models that are for Gen Media.
So we just released Lyria as well for music generation.
We have Bio for video generation.
Nano Banana, which is pretty much for image.
editing and image generation.
And then we have Gemini, like the core famous Gemini model, which is a flagship model if you want the best capabilities, the most knowledge, all of these organic, high-performance coding capabilities.
And we have Gemma, which is, as I was mentioning, like the open models that we have released.
Last week we did Gemma 4, which is the most capable model that can fit in a consumer GPU.
That means that these are models that you can run in your own devices.
And that can mean your own desktop computer, your own laptop.
laptop and even the smallest Jema models can run in your own home.
So trying to pack as much intelligence per parameter as we can.
Amazing.
So the key feature of Jema is its size.
It's its size and also the control that it provides you.
So you can run these models in your own hardware and you can even do something called fine tuning, which means that you can modify the models for your own use cases.
So let's say that you work in a setup where the...
information has a data type that is very tricky like legal setups or maybe like a specific medical type of data that most proprietary models have never seen before so you can pick Gemma and you can modify it for your own use case and it can work quite well.
And then you can serve that model in your own infrastructure and your own devices.
Amazing.
So you did a launch just recently, did you say yesterday or this week?
Last week.
Last week, last week.
So last week on Thursday, so that was six days ago.
And now we have over 10 million downloads already.
Wow.
Yeah, yeah, and over 1,000 Gemma 4-based models built by the community, which is super exciting.
I suppose you had launch partners and things like that.
What are people starting to do with Gemma already?
Yeah, so what is critical for us is that when we release new models, people can use the models with whichever are their favorite tools, right?
So we collaborate with Ollama, with HoggingFace, with Onslaught, with Lama CPP.
So pretty much no matter which is their tool, people should be able to experiment with the Gemma models.
And it's the same strategy for Gemini.
We work with Langchain, Lama Index, and so on.
And people are building very exciting things.
So people are running the models in their phone.
They are doing even agentic stuff within the phone.
So as part of the launch, we did an application called the Google AI Edge Gallery.
You can install that in iOS.
You can install it in Android as well.
And it allows you to do things such as adding new skills, even like controlling your phone, like turning on the flashlight and doing different UI.
device control things which is quite exciting.
Interesting, interesting.
I'd love to lean the conversation more into context now as well which we seem, it feels like context always opens models up in terms of getting them to get the best app models.
It's almost like customizing context and skills and things like that seem like a really good way to provide that information about how to customize your usage.
With the different models that you just mentioned, would you say context is as important with various models or would you say there are specific use cases where it's so important and others where it's less important?
Yeah, that's a great question.
So let's talk about Gemini and Gemma.
Of course, Gemini has a much longer context window.
You can pass videos of one hour.
You can pass very long documents.
In the context of Gemma, these are small models designed to run on device.
And the issue is that the context does consume more GPUs.
So you cannot just provide a huge context without having to burn much more computer, right?
So first, that part is quite important.
So the smallest GEMMA models support a context of up to 120K.
The larger GEMMA models support 156K, which is nice for on-device, but it's still much shorter than what you could try with Gemini.
So the context curation is actually far more important to make sure the context is small enough for the model to actually use correctly.
And the other part is also thinking efficiency, right?
So there are different open models on how many tokens or how much thinking or reasoning the model needs to do to get to an answer.
It can vary per model.
So that's something that we also look quite a bit.
So making sure that when models do reasoning, they don't generate just like a super long chain of thought, right?
And that they can actually generate like a...
dense, highly useful, highly dense kind of information in the context.
So we're at AI Engineer here in London.
It's great to be here.
What are the things you want to take away from the conference?
Yeah, so the first part is to just meet with the ecosystem.
As I was saying, for us it's critical to have ecosystem partners to collaborate with them, to contribute to the open source ecosystem.
So it's great to meet with all of them in person.
And then the other part is just listening to all of these exciting use cases, like what people are building.
The last time I was at the AI engineer conference was one year ago in New York.
In New York, New York.
It was in the winter of last year.
Just in the last year so much has changed.
It's so exciting now to see what people are building.
even compared to six months ago or three months ago.
So now I want to see where we're standing six months from now, right?
It will be quite exciting.
Well, there's a lot of exciting things coming out of Google DeepMind as well and very keen to hear what's coming and what's going to be in the future.
Yeah, yeah.
First, what is critical is to also build based on the feedback from the community.
So pretty much for Gemma 4, for example, we...
We gather all of this feedback from the last year.
So people were, for example, asking for function calling, agenting capabilities, better license.
So we changed our license to be Apache 2, actually, as well.
So pretty much it's a kind of opportunity also to gather as much feedback about what we're doing right, but also what we're doing wrong, right?
And how we can iterate.
And we're executing as a startup as much as we can.
So trying to release models, release new capabilities, release new tools as quickly as possible to get feedback and iterate based on that.
Amazing.
Omar, it's been wonderful to chat and enjoy the rest.
of AI Engineer.
Yeah, thank you so much for coming here.
Cheers.
I'm here by the Eleven Labs phone booth and I'm going to go in and try it out.
It's basically going to ask me a number of questions about British AI.
Let's see how I do.
That was 2016.
Was it chess?
We're back here at the Eleven Labs booth and joining me, Boris Starkov, who is a growth engineer at Eleven Labs.
What is a growth engineer?
Well, a growth engineer is a little bit of everything.
a little bit of marketing, of engineering, and a little bit of fun projects like the booth.
So that booth, you were the creator of that, you built that under the covers.
Tell me a little bit about the technology behind that.
So the hardest part was actually reverse engineering the actual phone.
We used cloud code to reverse engineer the low-level protocol of it.
And then we connected it to Twilio and connected Twilio to 11 Labs agents.
And that's how it works.
That sounds cool.
And it really was a phone line.
You were actually calling, made a connection, and then there was an agent at the end that was answering back.
It really was Michael Caine, right?
It is.
It is.
We licensed his voice.
Amazing.
Cool, cool.
So tell us a little bit about Eleven Labs, what Eleven Labs does.
Well, we started as a text-to-speech company.
Right now we do...
everything.
We have an AI agents platform, we have a creative platform with a bunch of tools for creators and yeah, we do a little bit of everything now.
We even have a music product now.
Oh, amazing.
And actually one of the things that I heard Matty, who was on the podcast, co-founder of Eleven Labs, one of the things that he said was voice will be the way that...
developers, engineers will engage with AI.
How much do you use Eleven Labs internally during your development and during your engineering?
Yeah, this is a fun one.
Actually, we've built a tool internally to use voice with Cloud Code before Cloud Code actually integrated the voice into their product.
And we also have a tool, again, like internally to use voice with OpenClaw so that OpenClaw can call you and you can call it.
Oh, wow.
truly conversational.
Yeah, yeah amazing amazing and what's new in in 11 labs these days?
What's new?
Yeah, well, we're growing really fast.
It's we have a hundred around as well very recently as well, right?
Yeah, yeah, there isn't a race the new round 11 billion Yeah.
Congratulations.
Thank you.
What are some of the conversations you're having with people at AI Engineer today?
Are people mostly using 11 Labs, interested in integrating voice into their product or development?
Yeah, people are very curious about the booth.
They use it as an entry point to show what they...
agents, the 11 Labs agents are capable of.
So usually people come, go through the quiz and then I show them the actual agent that they spoke to and I can even show them the actual conversation.
Oh wow, amazing, amazing.
Boris, it's been great chatting.
I hope you enjoy the rest of AI Engineer.
Thank you, Simon.
No problem.
Nice to meet you.
Awesome, well we're here just next to the OpenAI booth and we have Ryan Lopopolo who is a member of technical staff at OpenAI joining us.
So Ryan, welcome.
Thanks for having me.
Is this your first AI engineer or you've been to plenty in the past?
This is my first one.
This is your first one.
I'm excited to be here in London.
It's been a great crowd.
And where are you based normally?
Seattle.
Seattle.
Oh, okay.
So this is a bit of a track for you.
So, Ryan, first of all, what are you talking on here today?
Talking about harness engineering and sort of the arc of implementation that my team and I have been on over the last nine months to remove ourselves from the process of writing code as much as possible.
I kind of have banned my team from opening their editor.
And it's been wild to see how quickly we've been able to stack capability into the code base to let agents do more and more of the software job.
And that's really interesting because you'll get different types of developers, some who, like, that scares them completely.
Like, my God, that's my happy place in an IDE writing code.
But realistically, I think there's going to be different types of developers, those who are creators, that will absolutely love this because the job of writing code can happen faster and it can get taken away from them and they can create faster.
What's your reaction to, or I guess, what's your team's reaction to whipping out the IDE from underneath them?
I am a Rubyist.
Ruby was my first love.
So metaprogramming is kind of a thing that is like in my bones, right?
I want to kind of lift myself up a level, use high-level extractions to kind of do the job for me.
And I do feel like the harness engineering is really similar to this, right?
I am working through a super high-level primitive in a coding agent to do the code production itself.
So the levers I have around me are different.
It's all around refining output rather than generating it directly myself.
A trick that I have used, though, is to just, my team is all external hires, and they just go full YOLO insane into this thing.
This is the way we do it.
This is why we're at a frontier lab, to invent the future.
Amazing.
So talk us through some of the principles of Harness Engineering.
Fundamentally, the models are limited on two things, attention and context.
In order to maximize attention and limit context, we want the code, the process, the test to be the same as much as possible.
And we want the agents to largely cook with the minimum amount of instructions, giving it context just in time in order to efficiently respect that context and let it operate over very long time horizons.
Instead of front-loading all the context into agents.md, I want to just-in-time inject how to remediate a linter failure at the time it happens.
And this allows us to have much wider periods of black-box behavior where the agent is reasoning and cooking and writing code and resolving its own failures, which allows me to kind of look at the beginning.
and the end of a task rather than the messy middle sort of thing.
Yeah, interesting.
And so how is that done then?
Is that done through skills and things like that?
Some of that almost like progressive disclosure of context into the agent?
That's right, that's right.
We use skills for this.
We actually have very few skills in the code base that we want to centralize around.
I think we have five or six.
All the engineers on the team contribute to those.
And this way, we can kind of optimize the human involvement in the process to not have to change too much.
We have a fixed number of entry points into the model.
And then...
It's all about figuring creative ways to inject prompts into context.
Those skills are prompts.
Error messages from tests are prompts.
Review feedback, which comes mostly from agents, also prompts.
And it's all about getting them just-in-time injected and making sure that they are somewhat reliably followed.
The models crave text.
We need to figure out ways to give them text.
So in order for this to be successful, does it reflect that?
It's all about the skills being accurately written, accurately crafted, so that the agents are essentially guided in the right way.
Otherwise, they're never going to get to that end point.
Yeah, yeah, yeah.
This is why I don't really use plan mode very often, because...
I'm hearing that more and more, actually.
It's surprising, right?
You get these big plans that I really don't want to review, and I hit yes to accept them.
But if those plans actually encode instructions that are wrong, like I'm...
pushing the agent in the wrong direction.
So things we learn as we go that will actually affect how we've done that original planning.
Yes.
Very interesting.
Well, the time I want to spend is on the most ambiguous, hardest work over the course of building a product, right?
And I don't necessarily know the right shape at the beginning.
And the agents operate the same way when they're writing code for a PR.
We want them to kind of figure it out as they go to kind of go from a lot of white space in front of them to a nice, refined, high-quality output.
And getting feedback along the way is a big part of that.
Amazing.
Now, you're also going to be talking about exactly this topic at AI Native DevCon, which is happening.
June 1st and 2nd right here in London.
Super chast about that.
Which is what, like less than two months away now, so not too far away.
What is the reaction to this session?
When you present this session, are developers typically on board with it?
Are they a little bit hesitant with it?
I get a mix, right?
Folks don't want to move their hands from the keyboard, right?
There's this...
fear that the code is going to be misaligned, right?
And this maybe feeling that it's going to provide me with more work to clean up the mess afterward.
But if you invest your time instead to just simply saying, I will not accept slop, I don't let the agents produce slop, and I systematize the reduction of slop, you have kind of accomplished the same goal with much fewer synchronous attention cycles from yourself.
You want to kind of...
Think like a staff engineer, and you have five or 50 engineers at your disposal.
And with that amount of resources, there's a lot of damage you can do in sort of reducing bad code from being produced, rather than having to shoulder surf along the way.
Yeah, yeah, yeah.
So let's talk a little bit about your environment.
In terms of when you're doing Harness Engineering, what does your environment look like?
Presumably using Codex?
Yes, using Codex.
I'm hearing on Twitter, by the way, I'm hearing so many people go like, oh my God, Codex is just unlocking me.
And it's like, I see so many people jumping into Codex right now.
It's very interesting to see.
Codex does the full job.
I can trust it to go from prompt to merge PR with high quality code, reproductions of the work that it has done in the same way I would expect a human.
When I'm working on backend systems, I'm supposed to deploy it to staging.
My teammates don't, like, come over to my computer and, like, look at my command history to see that I deployed to staging.
I make an attestation in the PR with some proof that shows that I did this, and they largely accept it as true.
And Codex is able to operate in the same way, which gives me high confidence that it's able to do the job.
I see it executing for 6, 12, 30 hours.
I have this insane post on Twitter where the laptop is buckled into the backseat of my car.
It's, like, burning tokens as I'm coming to and from work because...
it is able to do these very complex changes.
And the way we do that is QE, the web product, the app, which is super, super awesome.
I love the automations in it.
It's constantly cleaning up after itself.
Every hour, making sure my CI is green, making sure that we have adhered to our golden principles around what good looks like.
I have a sticker on my laptop and it says AI works while I sleep.
That's right.
We're in that space now.
It's capable enough now that we can do that.
We're at AI Engineering, as I mentioned.
What is the one thing you want to learn from the AI Engineering Conference?
Is it chatting to people?
What's the thing that you want to take away?
The thing I want to learn is where folks are at with these tools, what will it take them to 10x the level of trust and usage of the model to enable them to build more and more.
products for the world to solve more and more user problems.
So much software is yet to be produced that we now have the option to do because implementation is so cheap.
So talking with folks, figuring out where it is they get hung up and how we can bring that to the product is something that I'm super, super excited about.
Amazing.
Ryan, it's been wonderful chatting with you and I totally look forward to your session at AI Native DevCon June 1st, 2nd in London as well.
I'll be there.
Excellent.
Thanks very much.
Hey everyone, hope you're enjoying the episode so far.
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
Whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
But for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
Well, we're here still in the expo hall at AI Engineer and we've bumped into Jordan Juritz, who is an applied AI team lead for the Incubator for Artificial Intelligence project for the government.
So tell us a little bit about...
your role and what you do.
Very good to see you.
Yeah, so as you said, I'm the Applied AI team lead in the incubator for AI.
We're a team of about 50 technical hall, permanent civil servants working in UK government to prove that AI can be used as a tool for the public good.
Amazing.
And so tell us a little bit about how the government is using AI today.
Yeah, so there's a lot going on in a lot of different places.
But what we think there's a real need for in government is to have really strong technical capability and like a full-fledged product development applied AI team that can ship things from zero to one and like get things and scale them out across the public sector.
So three things that we're working on at the moment which we think are really, really exciting.
First of which is government consultations.
We built the default app for analyzing government consultations that's saving hundreds of thousands of hours a year and many, many millions of pounds and is also more quantitative and better than humans analysing these consultations.
Rolling that out to look at consultations like digital ID and other sort of ongoing things and big contentious political issues.
And for those who aren't in the UK, recognise the UK government, what is a consultation?
Consultation is when government goes out and asks the public what their views are on a major piece of legislation, how it's going to affect what you can and can't do in the UK.
So it's a really important part of the democratic process.
And getting that analysis right is really important.
One of the things we're really happy to be able to prove with our AI models is that humans agree with our AI consultation analysis tool more than they agree with each other as a result of it.
The second thing that we're doing is digitizing the planning system.
This is super interesting.
You showed me this earlier.
Let's take a look.
So tell us a little bit about...
Well, tell us a little bit about the problems, I guess, that exist in that space today and how AI is used.
Come and get a look at this.
This is one of the basements of a council in the UK.
And what you can see here...
are boxes and boxes full of old planning records which tell you what you can and can't do with the land in the UK.
This is stressing me out.
It's really stressing me out.
I've had people say, I hope that the basement floods so that all these records get destroyed and we don't have to go down and look at them every time you apply for, let's say, an extension on your house or getting a new asset.
So a planning commission application comes in.
Someone physically has to go down.
They're trying to identify a card from these places.
pull it out, read what it is, and then understand, okay, yes, you do have these permissions, or no, this tree has a conservation order on it, or whatever those issues are.
It's a very, very manual process.
Very manual, and we've got what is often a very paper-based system.
This isn't an AI problem.
This is a data problem, as you mentioned earlier.
So what are the stages that need to go through in order to digitalize this, and then how does AI help?
Cool.
So what we're doing is we're taking these nasty old documents.
And we've created multimodal agents that can go and take them and turn them into nice geospatial data.
So this is a little demo of our app here.
You can see we can get our local planning officer can go and upload their PDF.
We use models like Glawd and Gemini to go through and do structured information extraction.
Then do a lot of extra hard work on top of it to go and figure out where in the country this map is, what the correct objects on that map are.
and segment them and then figure out the coordinate transforms from this old pixel coordinate map to this modern geospatial data.
You can then see here, we're also working with the planning officers to get them to go in and tweak and edit it and correct it.
That will enable them to go and upload their data to the planning portal, planning.data.gov.
So we start to fill in this map of what you can do with the land in the UK.
The other thing is we also get to keep all that data for our evals.
We can create a self-improving system so we can digitize the system even faster.
The evals are so, so important.
So crucial.
We showed it to the PM and he committed to digitize the UK's planning system by the end of the year.
So what we're doing is trying to prove that AI can be used as a tool for the public good and be pretty radical about the things that we try and achieve with it.
And when you say by the end of the year, what scale of data are we talking about here?
I want to see that map filled in.
And what does that mean in terms of the sheer number or the volume of documents and things like that?
So with hundreds of thousands of documents at least, and millions and millions of trees and old fields and lots of different, a whole history, a whole stack of different land use that we need to go through and digitize.
And from a productivity point of view, if we think about...
a human doing this?
There's still a human in the loop as we mentioned, but a human doing this, what kind of a productivity improvement are we talking about?
So it often takes a planning officer about an hour to digitize one of these records properly.
They also don't necessarily always fill in the information in exactly the right way and ask them to get it done in about a minute, two minutes, or about 10p.
So there's another problem which is there aren't enough planning offices in the UK to be able to do this.
So we need to kind of like think about how we can use AI to transform like the labor market problem here.
Yeah.
The bottleneck is people.
And then you can ask, so that's one layer of efficiency.
That's kind of like one useful thing to do.
We save time, we save money, but think about what we can do on top of this.
So like once we have a digital planning system, what can you do?
Well, you can start to do things like this, which is...
build agents on top of that like digital planning stack so you could say create an assistant for a planning officer so when you put in your application the agent can go run draw down all of the evidence that it needs suggest the right decision and get instead of an answer in 13 weeks get an answer in a week yeah like that's the sort of future we want to enable through digitizing the planning system um and there's a ton of whole like brilliant prop tech companies in the uk that are also pretty desperate to get their hands on this information as well so yeah we think there's a role for ai to be used to be able to curate and then publish more open data in the UK for people to build.
Amazing.
I feel like the UK government is actually very, very progressive with a lot of the digital assets that it's done over the last 10, 20 years even.
In terms of the AI usage, are there a number of restrictions based on the sensitivity of the data?
There are certain models you can't use or things you need to be extra careful about here?
So in the incubator for AI, fortunate in that we're given quite a lot of trust and quite a lot of control over our own stack.
So we're a team of developers.
We're very engineering first in what we do.
So we are able to use pretty much whatever we want.
We've got a lot of funding to be able to go and play around with these models.
I'll also mention we're hiring for a load of applied AI engineers.
So if anyone wants to come and join us on that mission at the moment, they definitely should get in touch.
Where's the best place to get in touch?
They can go to ai.gov.uk.
And there's a careers page there that you can check out.
And yeah, but like we also have to think about how we kind of deploy these tools responsibly and safely.
And that's a big part of the work is evaluating like really, really carefully the quality of the output of our models.
So we're both applied AI engineers who want to build stuff, but we also want to really like rigorously test our own work and prove that it's good.
One way that we're doing that, if I can shoehorn it in, is the work we're doing in education.
So there's for the last sort of six months.
The Department for Education has really had its fingers in its ears about the use of AI in education.
And kids are already using AI as a tutor at home.
But we don't know whether that's safe or effective.
We don't know if that's a good way to learn.
And as government, we've got a responsibility to make sure that people are learning well.
So what we're doing about it, we're constructing a first-of-its-kind benchmark for...
AI tutoring and that involves a lot of things but primarily that's working with teachers, edtech market and the department for education to define what good pedagogy looks like so are you giving the right sort of advice to children are you doing that in a way that's conducive to them learning you're not just feeding them the answers but you're teaching them how to learn and also doing it in a way that's safe so if you are creating dependence for instance or creating like an emotional attachment can you like prevent it from doing that So coming up with a wider range of metrics that can be used to analyze these long-form student-teacher interactions over here.
We're then also working with the Frontier Labs to turn these into benchmarks that they can go and test their models against to see, before these things get into the classrooms, whether they're safe and effective.
Excellent.
Amazing.
And it's so wonderful to see AI being applied in such a way that is actually, like you said, for the good, for public good.
Thank you so, so much for sharing.
Really, really, really, really great to hear.
One last question before we go.
We're at AI Engineer.
What are you hoping to learn or share with the audience?
Cool.
So the first thing I'm wanting to share is basically the vision.
I want people to know that they can come into government for a tour of duty, come and work with us for a bit, do some of the most impactful work of their career, affect everyone in the country in the process.
And there's a place for you to do it.
which is incubator for AI.
They should also check out the number 10 innovation fellowship program, which is for like the top tier technical talent to come into government on a tour of UT2.
Really exciting opportunity.
And that's something you're a member of as well.
Yeah, yeah.
So that's how I got into government.
I was in academia doing like some computational bioengineering stuff, worked in deep biotech.
So I want to work on stuff with massive social impact.
And I find it really gratifying.
I've chosen to stay.
The other thing I'm really excited about in the conference here is just the quality of the people in the room.
It's like the most incredible crowd.
Huge credit to, like, Sean and the organizers for pulling this off.
I've had this thing in the back of my mind, which is, like, what does the government agent stack look like?
We've got this, like, really gnarly sort of tech setup in the UK where you've got lots of delegated kind of different sort of cloud environments with different responsibilities.
But it doesn't really reflect how people think about what government is.
They think of government as one organization, but we don't operate like that.
So I'm thinking, like...
How could I work with something like WorkOS or some of the other kind of like sort of deployment and like infra platforms that we can see in the room here to build agents that could work seamlessly across government?
Yeah.
I think that'd be cool.
Can you give me any pointers on that?
Absolutely.
Yeah, yeah.
Awesome.
Thanks so, so much.
It's been great chatting with you and enjoy the rest of our interview.
Yeah.
Cheers.
Cheers.
Look who I bumped into, Nupur, who we actually yesterday had a panel at the TESL office for the AI Native Dev community.
So it's great to see you here at AI Engineer.
How are you?
I'm glad.
I think yesterday was a really nice session.
It was amazing.
And of course, you're a Solutions Engineer at Kodo.
Tell us a little bit about Kodo, what you do at Kodo and what Kodo does as well.
Yeah, I'm a Solutions Engineer with Kodo, so try to solve problems.
the AI creates by AI.
So it's like a loop.
AI creates problem and AI is trying to resolve.
Keeps us in a job, right?
Exactly.
So at Coder, what we are trying to solve is the code.
So when you use AI, you create so much amount of code that it practically becomes impossible to keep track on what exactly is being created.
Is that up to the standards?
Can we actually promote it to productions?
Yeah.
Initially and currently as well, we see every now and then.
things, security concerns coming up in the production with major players as well.
So we are trying to make sure we can reduce those gaps as much as we can by providing guardrails, by providing more patterns, what the organizations work with so that the secure code can be promoted to the production.
Now when I first heard about Codo, it was Codium back in the day and then it moved over and Codium was built around testing.
Yes.
And essentially creating unit tests, those types of things, and then rebranded to Codo.
Much, much more about, or it feels like it's much, much more about code reviews these days.
Yes.
That's true.
So when we started three years back, we started with code gen, we started with testing, a part of code reviews.
But as we grew, we understand before even testing can come into picture, code reviews is a step where we have a major gap.
And that's where the whole thing thought, like, okay, let's try to fill in that gap.
And we are good at it.
For example, for the last years, we are just trying to make sure we don't do only code reviews.
We reduce the noise, which usually nobody is looking at.
If you see the code gen as well, it's more the better but what we are saying is it's not more the better it's more about more quality the better it is so we shifted all our focus to code reviews and now we do code reviews so there's two important things I think about code reviews one is with the with agentic development we obviously need possibly a human in the loop possibly some deterministic testing to say is this code of reasonable quality The second thing is the sheer volume of code that is being delivered these days and that will be delivered in future.
I think Logan Kilpatrick from Google DeepMind said he's expecting in five years a million times more code that we generate to today.
With that volume, how important is code review to enable that we deliver at speed while maintaining that we're not just shipping rubbish into production?
True.
I think that's the main focus of current code review industry.
So I think this is a problem not only us, anybody in this industry is trying to solve.
Because, again, everybody thought process is we have to ship faster, we have to develop more.
But at what cost?
Do you want to ship faster in a week and spend maybe two weeks trying to figure out what went wrong and how to fix it?
Or do you want to make sure you can ship faster with guard railings, with making sure that...
code you are providing to the production is good enough.
So I think there should be a balance where it's not about shipping faster, but it's finding a balance where you can ship faster with the quality code in production.
And what's more important, would you say?
Because the industry, there are startups flying out everywhere.
Everyone's going fast.
Would you say it's more important to have high speed?
Or would you say it's more important to have like zero issues?
Because of course, if we can react to known issues that we ship, is that better or worse than shipping perfect code, but it taking longer?
Yeah, I think in the previous days as well, there were senior developers who can actually build your code in a couple of weeks and then ship it.
And now as well with the AI, it's the faster.
I think it should not be very restricted.
Otherwise, we are restricting the tech or the thing which we are working towards.
But we should differentiate between what is critical and what not.
Why not experiment with going faster, creating more experimental guardrails on the things which are least important, but still making sure the critical work we are doing around can be saved and can be checked a little bit more better.
And I think we are in a place where we are becoming more and more comfortable with AI, doing a lot of automation.
So, yeah, we can experiment with the non-critical stuff.
And once we are comfortable, maybe try it with.
we should find a balance not restricting because then the whole point of getting ai and shipping more code goes away yeah but on the other hand not let everything done be by ai so that it you have to pay a lot of cost yeah yeah how would you advise developers use context and skills to create you know better higher quality code from scratch that then obviously we still need to validate and test against But really getting that baseline to be better.
How would you encourage people to use skills and context to really enable their agents?
Yeah.
When you say context, context is crazy nowadays and this question is crazy too.
Because if you see the context, people don't understand what the context means for them.
When you say a developer, okay, the context is important, they think, okay, everything what I have in an organization, the whole code base, the confidence documents, the Jira documents.
Although the context window for our AI models is increasing day by day, but does it actually impact or does it understand the whole context?
You can dump the whole part base through the LLMs.
Does it understand?
Does it actually look into things and say this is important or not?
And if you see the current, we are not into that.
You can dump data onto data, but...
LLMs cannot.
They just choose the first part and the last part and all in between context is not.
It's about the right context, the right skills, the right context.
Exactly.
And then the second part is the mature developers, things like they can build context engine, they can create whole vector DB or the mapping.
But is that really important?
Is that really worth spending the time on?
So for me, it's important to understand what exactly I'm trying to target.
what kind of problem I'm trying to achieve.
And based upon that, choose what's important, what kind of context mapping I can do.
Instead of building a whole context engine, can I do iterative reduction or can I do hierarchical mapping and then create something to make your results better.
Awesome.
And let's wrap up with a final question.
What do you want to learn?
What do you want to take away from the AI engineering conference?
Oh, I think the conversations I have is the most important key for me.
conference because when you look when you work with a product or when you work with a specific organization you tend to start looking in a specific way if you see AI it's being used in so many different ways so many use cases talking with people gives you an idea what exactly is happening what are the problems and next time maybe I'm solving a problem I have that I'm a solution architect that's what I do yeah so it gives me a new idea about what actual problems and are and How can I help as you work now?
Amazing to put.
It's been awesome to talk with you again and enjoy the rest of our engineering.
Thank you.
Have a great day.
Thank you.
We're on the Zed booth and joining me is Cameron McLaughlin, an engineer at Zed.
Cameron, how are you doing?
Pretty good.
Happy to be here.
Very excited.
Were you here last night as well?
Yeah, last night, setting up and meeting a few people, checking out the other booths.
So tell me, like Zed is an amazing IDE.
We've had a couple of people at Tessil play with it and use it a little bit as well.
Tell us a little bit about what Zed is and also what's happening in the world of Zed.
Well, it's a text editor.
It's a genetic coding tool.
It's, you know, it's a review tool, I guess, if you want it to be.
It's kind of, you know, it's what you expect from a text editor at IDE.
It's why should you use Zed?
I mean, I like it because it's really fast.
It's not a browser, it's not an Electron app, it's just written in Rust from the ground up.
We have our own UI framework, our own everything.
We built it all from scratch.
What's the reason that someone would use Zed compared to a cursor or something like that?
It's a lot faster.
That's something that really affects me.
I've used cursor for a bit and sometimes you press a key and there's a little delay and you use Zed and you press a key and it's the next frame every time without fail.
That's something that's really important to me.
I think also, we just launched our parallel agents feature.
Our demo's not up, but we have this new sidebar.
You can run agents in parallel.
You can have all of them going.
You can use ACP, so you can have Z-Agent, or you can have code code, codex, any of these, open code, whatever you want.
Zep is actually one of the first to truly use ACP, right?
Oh, we created it.
That makes so much sense.
We all remember the world before LSP and how every editor had to make their own Java extension, C extension, all of this stuff.
And then LSP was pretty revolutionary in allowing lots of editors to speak the same language.
And I think it's obvious to us that we need a similar protocol for agents.
you can enjoy one harness and not particularly like its UI, or you just don't want to keep switching key bindings, you don't have to shift enter to send a message or just enter in another one, all of these things.
The friction adds up over time, and this is something I've really come to appreciate with Zed, it's very optimized for low friction.
Everything's fast, everything's how you're used to it.
The other thing I really appreciate is the collaboration features.
We don't have the demo set up at the moment, but it's a bit like a Slack huddle, you can join a huddle.
Pair with your teammates, you can talk to them.
Within Zed?
Within Zed, yeah.
So there's built-in voice chat, screen sharing, and then this is very, very deeply embedded in the editor.
So every buffer is this CRDT, this replicated data type that you can send these diffs across our protocol.
There is a server, but it's effectively peer-to-peer.
you synchronize the diffs that you're sending.
So I can hop in a call with Bennett and he can be moving around.
I can follow him.
I can see where he's looking.
Maybe he comes across a function that I don't know.
I can go to definition on that.
We sort of split off, join back, and we can edit the same code, run the same code.
Do you have other huddles as well that you can do in Slack and then have that integration?
Yeah, it's basically, you know, we, you know.
We don't use Slack, but we don't use the huddles because we just hop into Z and just phone people.
And then if you alt tab out of Z, it will switch over to screen sharing so you can see if you're reviewing a PR, you can look at the browser, that sort of thing.
Nice, nice.
So we're here at AI Engineer, probably, I'd say, arguably the premier event in the world for developers and AI engineers to visit.
What are some of the conversations that you're having?
What are people interested in?
And also, are you kind of like...
Are you happy with the level of adoption that people here are kind of like saying that they're using a very...
Yeah, it's been funny.
I've been trying to sell Z to people as they walk past and most of them just say they're already using it.
So it's made my job quite easy.
I think a lot of interest in ACP.
I think, you know, from our point of view, ACP is a way that we can, you know, as a text editor, we can let you use Claude code with your familiar key minings in your familiar UI, not stuck in a terminal.
But for other projects, it's like a way to abstract over agents.
It's a way to, if you want to send a message, if you want to write some code to send a message to 10 different agents and have them compete to provide the best solution, ACP's a really good choice to do that.
I don't remember the details.
I think OpenClaw's using ACP just for this, for this abstraction layer.
JetBrains, I think, is also...
Yeah, they're working with us.
They're working with us on ACP, and it's sort of this big collaborative push, because you would think JetBrains and Zed would be big competitors, but I think it's in everyone's interest to have this standard.
The LSP parallels are clear.
A world without LSP isn't necessarily better for anyone.
It's a frenemy relationship.
Exactly, yeah.
Healthy competition.
Yeah, absolutely.
Awesome.
It's been great talking with you.
What advice would you give to any agentic developer out there as maybe an unlock perhaps that you feel people don't really know about these days?
I think just try lots of things.
Just like try everything.
Set it up.
I will say this is a nice thing about ACP.
You can just try stuff and you don't have to get used to a whole new environment.
But yeah, just try lots of things.
Things are changing really quickly.
Keep trying as well because what you tried one month ago is probably quite different now.
Yeah, it is very different compared to the beginning of January for sure.
Yeah, keep trying things, keep experimenting, see what you like, see what you don't like.
Build some stuff.
Awesome.
Cameron, it's been great chatting.
Enjoy the rest of the show.
Nice to meet you.
Cheers.
Hey, joining me is Nick Arcolono.
Nick, how are you?
I'm great.
It's great to see you.
Absolutely, likewise.
You just came out of the keynote as well.
Tell us a little, actually, first of all, tell us a little bit about who you are, what you do, who you work for.
Yeah, I'm the head of AI and research at Jellyfish.
We're Boston-based, and we're in the AI observability space, understanding how companies are using agents to transform their software engineering teams.
And we've met a few times, actually.
I think in New York.
AI DevCon was the most recent.
Yes, yeah.
We've got another one actually June 1st, 2nd.
Hopefully you can join us there maybe in London.
Yeah, I would love to come back.
Yeah, but that was a great event you had in New York.
Oh, I appreciate that.
It was really fantastic.
It felt good to do.
It felt, you know, the content and everything was amazing.
So tell us a little bit about the content here.
You just came out of the keynote.
What was your biggest takeaway?
There was some great stuff.
The Google stuff was very fascinating.
You know, all the things that they're doing outside of large language models.
For instance, the Omnimodal Embeddings thing.
I would love to have a model that understands documents on a visual and spatial and kind of language level all at the same time.
The weather stuff was very cool.
And I've been a big fan of the world model stuff that they've been doing.
So that was great.
And then the open claws.
That was very spicy.
That was a lot of fun just to see.
I mean, it's unprecedented, that project, right?
So just to see.
what actually goes on behind the scenes was very interesting.
Amazing, amazing.
Seemed like some cries for help there, right?
Yeah, absolutely.
Like they need help, you know, so glad to see they're getting it.
I see Peter on Twitter all the time kind of like asking for contributions versus just people asking for things, and I think absolutely need that.
What are you here to learn at AI Engineer?
I mean, the biggest thing why I keep coming to these events is it's just everything's moving so fast.
So just seeing, you know, the difference between...
where things were a couple months ago is the most exciting thing for me.
And one of the things I like about coming to the expo, seeing the talks is, you know, people say like, oh, I remember coming and like auth for MCP like isn't solved.
And you come back and there's entire businesses devoted to that, right?
So just the pace of innovation where there's just, there's new products, new open source, new companies popping up to fill all these holes.
And you learn how these gaps you didn't know existed.
and then you know they exist and then they're being filled like that that flywheel is just so rapid it's like we don't need this once a year we need it every month or every other yeah every quarter i mean i've never had a time in my career where going to multiple conferences a year was so valuable and every time you're learning all these new things you're not just going to hear the same thing yeah absolutely what's more important to you is it the is it the the corridor track here or the sessions that you go to um I mean, you know, the...
You can say if you came here just to talk to me, I'm fine with that too.
Well, you know, it's the kind of live in-person experience because all this content is available online and it's fantastic and you can get it and I could be watching the live stream at home.
So the things that you can only get here by talking to the people, you go to the after events and actually hearing, because this is stuff, when you're terminally online, you get a skewed view of what reality is.
And when you talk to someone, you're like, what, how are you...
you know, your personal agents are building them.
How's that going?
And everyone's like, it's harder than everyone on Twitter makes it out to be, right?
So that's interesting.
What are you finding the level of folks who are attending AI Engineer?
Is it different to that online?
Are you getting the elite people who are, you know, on the bleeding edge of technology?
Well, it's more just like the Instagram thing where everybody's putting their best selves online.
And when you talk to someone face-to-face over a pint, then you find out, like, all of these things are harder.
Learning new stuff, changing what we do, and tinkering with all these things is hard.
And we're all working with piles of broken glass and trying to build useful stuff out of them at the bleeding edge of this.
So it's tougher than maybe it seems if you just sit around and read everyone's post about how great my open claw setup is.
It's harder than that.
Let's end by talking a little bit about jellyfish as well.
What's new in the jellyfish space?
I mean, we're, you know, I'm very much on the data side, and we're just excited about all of the new data we're getting, you know, and it's making its way into the product very rapidly where, you know, we used to be focused on understanding AI adoption and then a lot on kind of productivity and transformation.
You know, are your teams getting faster?
But now we're getting very much into the space of understanding tokens.
You know, what does effective token usage look like?
What does effective kind of agentic engineering look like?
Because we're able to get data and monitor usage and understand workflows at the token usage level, at the model level, at the kind of agent tool usage level.
So really just understanding, you know, it's not just about did I use the most tokens, like did I use them well?
Which is surprising how many companies today are saying, look, if you're spending more money and using more tokens...
you're adopting AI more.
Well, who's saying that?
The people that are selling us the tokens?
Surprisingly, no.
Surprisingly, people have targets in organizations to say, look, we need to use AI more effectively.
Not more effectively, we need to grow adoption at any cost almost.
And it's like, greater token usage is almost showing that more of the business is using it.
But the problem is, I think you said effective token usage, right?
That's what we're missing.
It makes sense to first order, but you're not going to run an engineering org based on how much electricity you use, right?
That we use the most electricity.
At these early stages, it makes sense, but everyone's going to want to know why, you know, if you're not having a good time, why?
Why isn't the machine working the way that you want?
You know, that's what engineering leaders, that's what they do, you know, so.
Nick, it's great to catch up.
Yeah, absolutely.
Great to see you.
Yeah, absolutely.
And I'm sure we'll...
Trip over each other again at another conference soon.
Enjoy the rest of AI Engineering.
Yeah, you too.
Well, that's it for today's episode.
I hope you enjoyed some of the conversations we had here at AI Engineering London.
Just the incredible groups of intelligent, smart people that we've been chatting with.
I loved how many people are really here at this conference to learn and understand what's new from not a year ago, but really a week, a month, a couple of months ago.
It's amazing how fast this industry is changing.
and really talking to some of the smart minds here and chatting to people in person is really what makes the difference.
So thanks very much to AI Engineer for having TESOL be here in person and sponsor the conference.
And we're also looking forward to our own conference in just a couple of months, AI Native DevCon in London, June 1st, 2nd.
Hope to see you there.
Bye for now.
The AI Native Dev is brought to you by TESOL, the package manager for skills and context.
Your hosts are Guy Pajani, and me, Simon Maple.
Our producer is Tom Dowler.
The AI Native Dev is not just a podcast, it's a community, and we host monthly meetups at the TESL offices in central London.
Visit tesl.io forward slash community to learn more, and I hope to see you there.