# AI Native Development: Context Engineering and AGI Productization

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-04-21

## Transcript

As the amount of software in the world increases, there's going to be lots of problems.
There's going to be lots of things that don't work.
There's going to be lots of edge cases.
And like the difference between those two things is where I think traditional software engineering developers will add a huge amount of value because that gap is going to change the way to use AI tools today.
three months ago was different and three months before that was different.
It's mostly if you want to get the like frontier level productivity gains, you need to sort of continue to evolve.
The rules are being rewritten under our feet and you can either sort of like ride the wave or you're not going to get these like frontier level productivity gains.
A couple of years ago, you asked how long until AGI and Musk replied next year.
That didn't happen.
Where do you feel like we are today versus a year ago on the path to AGI?
We're sort of like close to what folks will feel as this like narrow super intelligence, I think is like actually more close to the way of describing it.
Like if you could have a model or a system that could like build anything with code, I think about that as like narrow super intelligence.
AGI is not going to be a model, it's going to be a product that somebody creates.
Who's down for building that then?
Is that one company, one individual, or is that more a combination from an ecosystem of products?
That's a good question.
I think...
Back in November, we hosted the first ever in-person AI Native DevCon in New York.
This June 1st and 2nd, we're bringing it to London.
It's two days built for AI Native developers and engineering teams.
One day full of hands-on workshops and one day full of practical talks on agent skills, context engineering, agent orchestration and enablement platforms, and how teams are actually shipping AI in production.
Join us at the brewery in London near the Barbican for all of that.
Plus, networking, parties, giveaways, and a room full of people building the future of AI native development.
You can also join us from anywhere in the world via the live stream.
As you're listening to this podcast, you get 30% off your ticket with code POD30.
Just head to ainativedevcon.io and we'll see you in London.
Hello and welcome to another episode of the AI Native Dev.
And we have a special episode today.
We actually have Logan Kilpatrick from Google DeepMind joining us today.
Logan, a massive thank you and welcome to the episode.
How are you?
I'm hanging in there.
Lots of, it's another week of chaos in the AI ecosystem.
So trying to, trying to stay on top of it all.
There have been wildfires all over Twitter and the social spaces already this week, haven't there?
It's been quite the ride.
And for those listening, it's April 1st today.
But this is the week in which Anthropic, knowingly or unknowingly open-sourced clawed code and things like that.
So lots of fun things happening in the world right now.
Logan, for those of you who don't know Logan, Logan was previously a leader of the Developer Relations.
team at OpenAI and currently a member of the technical staff, Google DeepMind.
So Logan, why don't you tell us a little bit about, I guess, some of your time at OpenAI and moving over to Google and what your day-to-day looks like?
Yeah, there's lots of parallels.
I do think the ecosystem has changed so much.
I've been at Google.
Actually, today's my two-year anniversary at Google, which is crazy.
Yeah, thank you.
I joined OpenAI end of 2022, a week after ChatGPT had launched and sort of got to launch GPT-4 and ship a bunch of, you know, the Chat Completions API, the Assistance API, plugins, GPTs, a bunch of other stuff across the ecosystem.
And just seeing how fast things have changed and how much progress we've made.
I think, like, my most interesting reaction is, like, you know, all the agent stuff 12 months ago, there was like lots of conversations about agents in the developer ecosystem and more broadly and like to see them actually work now and like people are building agentic products.
People are using agents to like do the things that it was sort of like the promise.
And it's interesting to see this and we should talk more about it because like it was kind of like a joke 12 months ago is like, oh, you're doing agents like ha ha ha like that doesn't really work.
It's kind of science fiction.
And I think today it's like very real.
And so I'm always thinking about like, what are those next set of things?
And so spend a lot of time like with our team thinking about like, what are those next set of things that we need to be thinking about that are sort of a little bit science fiction today?
My like, we have a bunch of these like research to reality monikers internally that we sort of like think about of how we, it's our team's responsibility to bridge the research to reality because we sit in a research org inside of DeepMind and our job is to build products for builders and developers.
So.
the day-to-day looks like shipping products and shipping features and building a platform so that we can make that research to reality fly will happen.
We'll start off with the today and then we'll kind of like lean into the tomorrow towards the end of the episode.
So one of the super interesting pieces that have come out of Google recently is AI Studio.
And of course, many are already familiar with, you know, more of the terminal UI style of using Gemini and things like this.
So first of all, Talk us through a little bit about what AI Studio is and why people would maybe lean into AI Studio more than Gemini and when people would kind of like more lean into the terminal versus the UI.
Yeah, it's a great question.
I think one of the beauties and one of the challenges of Google is that we have a very wide ecosystem.
We're sort of touching every part of the developer lifecycle.
We're touching every part of...
uh the different ecosystems across like web and mobile and cloud etc etc ai studio i think the most simple way to frame it is uh is a platform, a set of things that help you go from prompt to prototype to production and do that really fast using all of our AI infrastructure.
So historically, AI Studio was like a UI playground that you could sort of test all of Google DeepMind's latest models and experiment with them and then actually take them into the API and go and build products and tools and apps on top of the Gemini API.
And I think what's happened in the last six months is we've sort of continued to evolve the platform.
We still have that.
Playground functionality so you can test all the latest models on day one and sort of experiment with them.
But we also now have an entire Vibe Coding experience.
And I think part of this is like it's a natural extension of the work that we were doing with the Playground, which is like help people understand how to actually get these models into products and like bring their ideas to life.
I think the main thing that's also shifted is like.
There's just way more people now.
It's not just developers.
It's a sort of like next gen developer builder persona who historically couldn't use these tools who now can.
And I think it's part of our mission in a studio and broadly inside of DeepMind is to bring this technology to as many people as possible.
So now you can go to a studio slash build.
You can like vibe code entire apps that are built on top of.
Firebase, which is our storage solution in parts of Google.
You can deploy using Cloud Run.
You can use Google Search.
You can have grounding data with Google Maps.
And we're exposing all these bits of the Google ecosystem and lots more cool stuff coming on that front in a way that's just simple.
You can click a couple of buttons.
You don't need to sign up for eight different accounts.
And we're bringing together the ecosystem and building this.
platform for people to build on top of, which has been really, really exciting.
So lots of lots of cool things.
To answer the question specifically, like Gemini, the app versus AI Studio and like Gemini CLI versus AI Studio and then even anti-gravity versus AI Studio, Gemini app is sort of like a personal assistant for your everyday use.
So like, you know, I was asking a bunch of medical questions to the Gemini app this morning.
The Gemini CLI is like a form factor of the CLI.
So if you're a developer, you use CLIs and sort of that feels native to you.
great product experience as a developer to go and use that.
And then Antigravity is sort of this full stack IDE agentic developer platform where you can actually go in and similar to what you would do inside of like cursor or cloud code, et cetera, et cetera, you can build apps.
So if you're a developer, it's sort of your daily driver to go and write code in really complex code bases, et cetera, et cetera.
AI Studio sort of has a batteries included app builder and then a bunch of underlying infrastructure that folks can build on top of.
And do you think that, you know, a lot of this is very kind of dependent on how the engineer or the developer wants to work?
Some people prefer to like terminal UI, some people for the more of the...
the IDE approach.
There's a version that people believe where sometimes, you know, Vibe coding is maybe for more for non-developers and actually, you know, senior engineers don't really need to change the way they work and things like that.
Do you feel like we will actually converge into an approach whereby if you look at development today, 99% of developers, or not today, sorry, five years ago, 99% of developers were using an IDE to develop.
Today, that's very much more split.
Do you feel like when we get to a stage where agentic development is just the norm, we will have a single typical way where people will develop code?
Or do you still feel like there will be the more technical folks that maybe want to lean into the terminal?
Other people will maybe more want a visual aspect.
Are we in a moment in time or do you think that's going to continue?
persist?
My assumption is that it will persist.
And part of this is like back to the thread of like personal preference.
Like this has always been the case.
Like there's been developers pre AI era who were using IDEs.
Like I've always been sort of an ID user myself.
I was using, you know, VS code and whatever the one that was built by GitHub back in the day that there's another, they had their own GitHub at one point had their own ID before like pre Microsoft acquisition that I used a bunch that I really liked.
Um, and at the same time, there's always been like Vim and Emacs and a bunch of these like more terminal CLI centric tools.
And I think this is just like a developer comfort thing.
There's like trade-offs from an ergonomic perspective, the trade-offs from our productivity perspective.
I think that will actually increase with all this stuff.
And I think part of this, part of the worldview of this is like the ability to.
create new software, the cost is going down.
So I think there'll be even more exploration.
There'll be even more weird stuff.
You could imagine each developer actually has this very customized experience of how they like to build software, how their mental model, the way I think about software may be different because we learned in different languages and we come from a different place or whatever it is.
So you can really customize that.
I think developers will actually probably end up building a lot of this stuff for themselves.
I also think there'll be like platforms that have this level of extensibility where like, you know, you could imagine ever in the future, every developer has like a fork of VS code, for example.
And it's like, you know, there's the editor part of that, but then there's an agentic part of that as well that they sort of build themselves and customize.
That's maybe like an extreme worldview, but I think there's like the answer is probably something in the middle.
And I think you'll see a lot of this like.
developer choice meet people where they are all these different ecosystems require different stuff people's level of like comfort and familiarity with ai and these tooling is also different so like it warrants a different product experience and it's actually hard to build that like highly extensible customized product experience at least today if you're like one of the teams that are building this product absolutely i'd love to kind of go a little bit deeper into the kind of like the how we can vibe code well.
So one thing that you mentioned was if you want 30 things as part of your vibe coding task.
ask for 30 things in the first prompt.
The model is now smart enough to handle that logic.
And that was kind of like a little bit different to the original form where we had to essentially ask for something small and build up because giving too much information, too much context, essentially, would overwhelm that model.
Do we need to rethink the way we were kind of originally taught to develop with AI?
Yeah, 100%.
I think it's not.
But my framing of this is like, This phenomena is going to continue.
So like you need to have this sort of, it's part of the challenge of this moment.
And I fall into this, you know, the bucket of the folks who have a difficulty with this as well.
So I'm not saying that I've figured it out, but like you need to have the mental plasticity to sort of like.
change the ecosystem like the way to use ai tools today three months ago was different and three months before that was different it's mostly if you want to get the like frontier level productivity gains you need to sort of continue to evolve and that's hard it's just like difficult as a human to continue to do that but like that is the reality of what needs to happen and this like ask for 30 things um example is like a very acute way of of uh feeling the difference like truly for me 12 months ago i was like let me ask for the bare minimum thing possible because otherwise the model will sort of and the agent will fumble over itself not be able to actually do what i ask and now i'm like constantly kicking myself to be like maybe i should ask for three extra things um or four extra things or five extra things or all 30 things that i want and the rate limit is now like how quickly can i ask for things um and that's just a very different world to be in and i think that it's it's been like literally in the last six months that the shift has happened i think and what this was not the case a year ago so um yeah you kind of need to the rules are being rewritten under our feet and you you can either sort of like ride the wave or um yeah or you're not going to get these like frontier level productivity gains yeah another i guess kind of thing that has really shown its face here is context and i think prompt engineering was a thing which we discussed and talked about you know a couple of years ago but seems to have died off now in terms of the importance compared to something like context engineering and in particular i guess skills over the last six months or so or less skills have kind of like you know been built up and used so heavily by people for productivity gains how much do you use or how much do you hold context and skills as a part of a real efficiency gain in your work in AI Studio and in Gemini?
Yeah, we have a ton of skills that the, like when I do like engineering work and use anti-gravity internally to sort of like build AI Studio, we have a lot of engineering teams build like tons of skills, which is great for me because I don't know how every, I'm not a sort of day-to-day engineer at Google, so I don't know how a lot of the systems work so they can bring that context into the skill and sort of influence like the architecture decisions and stuff like that, which is really helpful.
my worldview has always been that like prompt engineering was a bug like if you go and talk to users they don't want to prompt engineer and actually like the things that you're asking them to add in already exist somewhere else so like your job as the human using ai systems and sort of like the llm or ai app 1.0 era was like to do the context engineering.
That was the value add and of like what you were providing as the human in that experience was going and finding all these disparate sources, bringing it into a little chat box and then sending it off to the model so it could do something useful.
I think my worldview sort of shifted when deep research came out, which was you could sort of like take this really ill formed.
question or idea or hypothesis and you could sort of send it off to deep research.
And Gemini would sort of like go and browse the web and find all the different data sources and like do this context engineering on the fly to really sort of like bring all the information in.
And it was a, I had this magic aha moment where the deep research UI would like show you all the different like sites it was visiting and all that.
And you'd see like hundreds of sites and thousands of sites, even in some cases being visited.
And it would just like, was the epiphany.
to me that like this is clearly the way that the products are going to end up going, which is people do the thing that humans do, which is like we ask our very like context, thin questions, comments, requests.
And then the system actually goes and does the work in order to go and find that.
And you see that now actually with coding tools, which is like I can ask for some change.
I'm in a million line code base somewhere and like the model will then go like.
grab all the files and sort of look through and try to find, you know, pull in the right pieces of context.
I don't need to say like, and I think it's sitting in this random HTML file on this folder here.
Like that is a complete bug.
You shouldn't have to do that.
You should be able to with like pretty minimal context, go and do these things and the model should be able to figure it out.
And so I'm, I'm very happy that on the coding side, we've like seen that as the direction of travel.
And I think hopefully we'll see that in like a bunch of other domains as well.
But this is the unlock is.
the model on the fly doing context engineering.
It's an interesting thread of skills because they're obviously very helpful today.
I think it will be a similar direction of travel, which is like the models will probably learn how to make skills on the fly and thus not.
And actually, the reason why this will end up being the case is because it's a token saving efficiency thing, which is if you can get some really solid skill that has the right context, it just makes it so that the model can figure a bunch of this stuff out.
It is just going to fumble its way through.
It's going to send 100 requests to the Google Drive API, and none of them will work, and it'll self-correct a bunch of times and read.
20 web pages and look at 70 examples, it'll figure it out.
But like it just takes a lot of time and it wastes your waste your tokens.
So skills, I think in the short term are like a helpful way to just get around that.
But I would expect over time that the model just like pre writes a bunch of this stuff or pulls from some repository of like domain authority skills, whatever it is, and then solves this problem so that like humans aren't hand crafting skills in the way that I think they are today.
Yeah, it's super interesting.
And I think I love the fact that you called prompt engineering a bug there.
And I think that's absolutely the right way of thinking about it, because there really was that disconnect really between human and LLM, whereby typically it's the human not asking for something in a way that the LLM really expected to be asked about a certain thing.
Context is very interesting, because I guess, particularly with skills, because the model will maybe not understand or know exactly how that user or that developer wants.
something to be presented there could be the average how the industry works but it's more how a company maybe it's policies maybe it's in you know very specific things in a skill i guess the model can find that out locally um but i really like your approach there of saying you know it could do all that research but if the skills are just there they could be handcrafted once or they could be they could be built by the llm but once they're there you might as well kind of like just have that in some repository and just pull that as needed so i i love that approach i'd love to ask a question actually uh about um about when you talked about uh in a previous quote where you said software volume is going to be a million times uh what it is now in 10 years that's both exciting amazing and scary at the same time um what what do you feel that's going to do to the value of development or a developer how's that going to change when we have to look after and deal with that much code yeah it's it's an interesting question because i think like the landscape is shifting so quickly and so dynamically i think there's a few things like a in 10 years like what we consider to be development will like likely look significantly different.
There will still be things that are similar.
People will still be fixing problems with software.
I don't think that's going to go away.
I think the way in which the tools are wielded, the scope, the level of detail, I think a lot of those things are up in the air in my head as far as how it's going to shake out.
The interesting thing is why I'm relatively...
bullish for software engineering or just like engineering as a discipline in general is because like as the amount of software in the world increases and as the like number of people who are creating software and having software created on their behalf increases um there's going to be lots of problems there's going to be lots of things that don't work there's going to be lots of edge cases there's going to be there's always going to be a frontier of like what can the tools do that like the average person wielding them can't do.
And like the difference between those two things is where I think engineering or sort of like traditional software engineering developers will add a huge amount of value because there will that gap is going to change.
But like the gap is also is going to be ever present, even if even if the models and the tools get get really, really good.
I also think it's like I think people get caught up in a lot of the like.
pedagogy of like this, this conversation, which is like, what I think about software engineering, I think about like a way of solving problems, a way of thinking about the world, a way of like, yeah, a way of approaching problems.
And I don't think about it like, I'm not at least personally myself, like attached to this, like I type keys and characters show up on the screen, and then those things represent.
you know, some formal structured programming language like Python or JavaScript or whatever.
I think it's like more general.
And actually, if you look at like computer science education, like in a lot of cases, there's like there's a there's a difference in the way that it's taught different places.
But like there's lots of computer science education, which is like what I'm talking about, which is this like very problem solving.
It's a way of thinking about the world.
AI doesn't.
minimize the value of that.
I think it actually accelerates the value of it.
And that value of the way of thinking about the world, the way of problem solving, I think is going to be super, super valuable.
Again, I think the value of typing keys that make characters render is probably going to go down, though I still think there'll be reasons to do that and there'll be value of it.
But this like way of solving problems, I think, is going to persist.
And like, I'm grateful that I like spent the time to think about those things.
And like it manifests in the way that I build stuff today.
Yeah.
Yeah.
Do you think there's a mechanical sympathy there, essentially, in terms of us understanding how things have been built to allow us to actually architect and make applications more reliable, more robust going forward?
100%.
Yeah, I think someone needs to have the level of depth to understand all these things.
I think it's like, does everyone need that level of depth?
Probably not.
But you want to have experts to go and go deep in these different areas and think really deeply about the systems and know the right questions to ask.
Also, a lot of it is like, as the means to build software is dramatically increased, Lots of life and lots of these like technical decisions don't have a real answer.
They have.
somebody who has a strong opinion.
I think you still need somebody to have the opinion based on their own experiences or their sense of the direction that you want to go to make a bunch of these technical questions.
There's often not a single right technical direction.
There's many possible technical directions.
It's based on people's own lived experiences and intuition and understanding of technical constraints, they make a bunch of decisions.
I think the same thing, all of that will continue on in the future.
Interesting.
Let's continue looking into the future a little bit and talk a little bit about the wonderful topic of AGI.
Now, a couple of years ago, you asked how this is on Twitter on X.
How long until AGI and.
Musk replied, next year.
That didn't happen.
And actually, I really like your sitting underneath the hype approach to this.
And you said someone is going to weave together the right components at the product level with a model that's really smart, and people are going to call that AGI.
And I think that's really interesting, because obviously the AGI timeline predictions that many have thrown around haven't really aged that well.
Where do you feel like, first of all, we are today versus a year ago on the path to AGI?
I think this conversation has gotten even more complex over time because I think when a lot of these conversations started a few years ago and even like 10 years ago, you didn't have tools that could do any of these things.
It was very academic, very philosophical.
And I think a lot of the definitions and sort of like...
preconceived notions are grounded in that initial stage where there wasn't like we didn't have tools that had any of these capabilities.
And or like the most advanced version was like, you know, AI playing games.
But like it didn't generalize to a bunch of other things yet.
I think the the like product adoption and the tooling and the way that this is impacting our lives has changed so dramatically in the last three years that.
I've almost decoupled myself from some of these AGI conversations because I think it's important that somebody has these conversations and thinks about it and has this more academically rigorous point of view.
I don't have that point of view.
I think my worldview is very grounded in, I think, how the world, how the average person is going to interact with this stuff.
If you took the technology we had today and you brought it.
three years ago, they'd be like, holy crap, that's the future.
These systems are so smart and can do everything.
In some sense, the goalpost keeps moving.
In some sense, I think we are like, AI coding is an example of so much value being created, all these research tools, all these other things.
It feels like we're close to what...
folks will feel as this like narrow, it's like narrow super intelligence, I think is like actually more close to the way of describing it.
Like if you could have a model or a system that could like build anything with code, I think about that as like narrow super intelligence, humans can't.
compete on the same level in that respect.
I think from the academically rigorous point of view, this AGI question is grounded in whether or not the models have the generality.
And so I think they still don't have generality.
At the same time that I can basically build anything I want with software today using AI coding tools, I can trip the models up with all of these goofy things that humans are able to really easily do.
I think the academically rigorous argument is like, well, we don't have this general intelligence until the models don't get tripped up with these things.
It's pretty easy to beat the models at poker or chess or any of these things that humans can be relatively good at.
My worldview is like...
For the average person, it doesn't matter.
For the average person, these tools are super impactful.
They're already creating tons of economic value.
I think this system from an AGI perspective is my worldview of AGI is not going to be a model.
It's going to be a product that somebody creates.
And I think that's already the...
Folks would probably agree with this worldview, even those who have this very academically rigorous point of view, just given the way that the models are.
Now, to do coding, as an example, you need an agent harness and you need a product, whether it's a CLI or an IDE, to bring the model to life.
You need all these things.
I think this worldview is like...
likely tracking the progress that we're making.
My last quick comment is I think it's a realization that like there's this capability overhang.
And so I think that's part of my worldview of like why we'll see like it's likely that it won't be like.
And this model lab just dropped a new model and then everyone thinks it's AGI.
I think it's going to be that like there was a model that came out three months ago.
Some really smart product and engineering team found a way to like do something really interesting with it and put it into a system that I think people generally think is AGI because there's this huge capability overhang that exists today.
And that resonates so strongly with me because if you think about what's actually unlocked, you know, development today with AI, it's not a model that has, you know, got significantly better than any other model.
It's the fact that agents has been layered on top of that and it's that interaction between agents and, well, although it's an LLM itself, an agent and a back-end LLM that has made agentic development so much more powerful.
And I think that when you say it's, AGI will probably feel like a product release.
It's that environmental, how you actually build your system that has various products in versus a single LM that kind of like says, okay, ask me anything you want, I'm AGI.
For me, that doesn't make sense.
What you're saying there as a product or a feeling really with a set of products in an environment makes a ton more sense.
If AGI, I guess, does arrive...
kind of more as a product experience than a single model who's who's down for building that then is that is that one company one individual or is that kind of more a combination from an ecosystem of products yeah it's a good question i think this is one of the interesting threads about like you know i'm grateful to be at google because i think we have product distribution across all these things, across like so many different verticals and places that you might expect need to be good if we really were to have this like general level of intelligence.
So yeah, I don't know.
It'll be interesting to see.
I don't know if I have the answer, but I would expect like your.
You know, the Gemini app as like a as like a personal assistant sort of like to help you in your everyday life, I think should if it's like artificially generally intelligent, it should be able to like dispatch and work with all these different tools and ecosystems to like help me.
complete any task that I want.
And I think we're like, you know, we're trending in that direction with tool use and all these other things that are like coming to the to the app.
And I think that that paradigm will continue as you'll likely have this like again, that product experience where like one system maybe like relies on many other systems.
But I think it will be very I think I think it's underappreciated probably the level of breadth and depth in the like complexity of solving this artificial general intelligence thing.
I would be very surprised if it's like we end up with this just chat UI where you go in and you just ask your question and everyone's like, this is AGI because it can do anything.
I think it's going to probably be this very orchestrated, lots of different stuff, a ton of UI complexity and all these different things.
I think it's going to be much more verbose than I think this simple AGI idea that...
that folks have to, that's at least my personal feeling of how things will shake up.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
Cool.
Logan, this has been awesome.
I want to jump into some quickfire questions to wrap up this episode.
First of all, and honestly, Logan, OpenAI or Google, where did you have more fun?
I'm having more fun at Google these days.
I think it's, yeah, it's an incredible place to build products.
Awesome.
When you were at OpenAI, what was one thing Sam Mountain got right that no one really gives him credit for?
I think people give him, Sam, a lot of credit for a lot of things.
I think he was right about the level of compute.
I think Sam was like maniacally focused on making sure that they were resourced for the level of like AI consumption.
And I think he was right to make that the focus, even like, you know, basically the first meeting I was ever in with Sam, he was talking about that.
So he was definitely right about it.
Amazing.
There are a lot of big labs out there now all jostling for position.
If you had to bet on one lab not existing in five years, which one would it be?
That's a good question.
I think maybe the bet would more so be that like, I think the way that we see them as labs today, I think will be different.
So I think like one of these labs will likely evolve into something that like probably doesn't look like the like lab of today's era.
But I think all of them have my senses, like there'll be many winners in the ecosystem, but I think people will like pivot into different.
And there's like a good, actually, I don't know if the social media analogy is perfect, but like a lot of the early social media products looked very similar.
And over time, they ended up like it's very clear to most people that like Snapchat's a completely different business and product than Instagram, then, you know, X and Twitter is, et cetera, et cetera.
So I think we end up in that kind of set up a little bit.
Awesome.
What would you say is most overrated today?
Agents, RAG, or prompt engineering?
I think people have moved on from RAG and prompt engineering.
So they perhaps were overrated before, but I feel like they're probably adequately rated now, which is that people don't put a lot of stock in them.
Obviously still important, both from like a conceptual point of view, but I think the frontier has moved on and people are focused on other things.
Sounds good.
What is the most overhyped AI benchmark today?
I like Sweebench a lot.
I think that my only comment on this is like, and there's better versions now, and actually the folks who built Sweebench have done a great job of continuing a bunch of the different versions of Sweebench, but some of the original versions of Sweebench is completely out of distribution of how I think most people do development work.
I think there's, it's like 40, this could be, it's like, I'm off on this.
I don't remember the specific numbers, but it's like 40% of the original SweetBench is like the model setting up Django, which is like fine.
And it's great that it can measure that, but like probably out of distribution from like how developers spend 40% of their time.
So they've done a great job of continuing to evolve.
So I think maybe not they're overhyped, but they've done a great job of like meeting the moment with some of the new SweetBench stuff that they've done.
Yeah.
In 10 years time, will there be more or fewer software developers than today?
I think the absolute number will probably be very similar.
And I think there'll be like new role profiles where there'll be like maybe 10 to 100x more people who are touching code on a daily basis.
They're probably also just like this, like member of the technical staff profile is a good example of this.
They're probably just doing other things.
So absolute number of developers.
Same number of people touching code, I would expect to be like dramatically more than what it is today.
What would you say there have been a lot of amazing research papers over the last few years?
What would you say is the most important AI paper of the last couple of years?
It's probably scaling laws or the original transformer paper just sort of as, yeah, setting up this industry that we're in today.
And those two things have really held true and had a huge impact.
Which AI company would you say is most underestimated right now?
I think people underestimate DeepMind, actually.
I think every six months, I think to myself, is Google the best place in the world to be doing this work?
And every six months, the answer is the same, which it is.
I think there's just so many great things about being at Google.
the the like talent and the way sort of like Demis runs DeepMind I think is all deeply resonates with me so um it's it's hard Google's a big ship but when you steer it in the right direction uh things go really well so I'm I'm excited amazing and the last question Logan what would what's the one thing you wish developers would stop asking you I wish they would stop asking, you know, rate limiting stuff.
And I think there's like a there's a threat of this.
It's not because I feel I have a lot of empathy for folks asking this question.
I think it is, and we're doing a lot on the product side, both on AI Studio and across Google and other products to solve this problem.
The future version of the world that I want is abundant compute.
Developers can just do the things that they want.
They're not worried about rate limits, and they're not worried about quota stuff.
I think we need to do a bunch of stuff from a product perspective to make that possible.
So I selfishly hope they stop asking because we solved this problem, not because I'm annoyed by them asking.
That sounds amazing.
That's a future we can definitely live in.
Logan, this has been absolutely great fun.
Thank you so, so much for taking the time out to speak with us and I appreciate it.
Thank you for all the thoughtful questions.
This was a ton of fun.
Awesome.
Thanks everyone for listening and tune in to the next episode soon.
Bye for now.