# WebMCP and Client-Side AI: The New Web Architecture

**Podcast:** The AI Native Dev - from Copilot today to AI Native Software Development tomorrow
**Published:** 2026-03-31

## Transcript

We need to stop thinking on LLMs as the only solution.
Apple released an open source model to do some image detection.
It's a model of 200 megabytes and works client-side.
So now you can have an OCR system that can detect objects completely local.
It opens opportunities for a lot of apps and web apps that can run even without internet, just offline, and you don't need LLMs.
Tell us a little bit about your recent work and focus on...
webmcp like what it is and what problem it's trying to solve right now jadgpt agent mode or cloud agent mode they're browsing the website typically just taking screenshots and then say oh analyzing with an image model but then they trigger the click on that portion of the screen but what happens if in those five to ten seconds the javascript move that button that is inefficient both in time and in cost.
And that's where WebMCP appears as a solution.
WebMCP is kind of an API.
It's just JavaScript functions that you expose to the agent.
Instead of browsing the website as a human, it will just execute those functions.
Back in November, we hosted the first ever in-person AI native DevCon in New York.
This June 1st and 2nd, we're bringing it to London.
It's two days built for AI-nated developers and engineering teams.
One day full of hands-on workshops and one day full of practical talks on agent skills, context engineering, agent orchestration and enablement platforms, and how teams are actually shipping AI in production.
Join us at the brewery in London near the Barbican for all of that.
Plus, networking, parties, giveaways, and a room full of people building the future of AI native development.
You can also join us from anywhere in the world via the live stream.
As you're listening to this podcast, you get 30% off your ticket with code POD30.
Just head to AINATIVEDEVCON.io and we'll see you in London.
Hello, everyone.
Welcome back to the AI Native Dev.
Today, we will be jumping into the world of the web, which is actually a fun space because my background in sort of the entrepreneurial world has been in making websites faster, especially with the rise of the mobile web.
And that's where I got to meet Max, who is...
kind of been building up a ton in the web at the time, was optimizing for mobile performance, for responsive websites, for many aspects of the rise of the mobile web, and today is in AI world.
So Max, thanks for coming on to the show here.
My pleasure.
Thank you.
I can kind of wax poetic a little bit about your background, probably for you to describe the journey about it.
So tell us a little bit of, just for people to have the mindset of where is your information coming from, a bit about your experience.
Yeah, sure.
So I've been a web developer for 30 years.
So that's a lot, 30 years now.
So I have started doing websites in 1995, actually.
When we were chiseling them in full.
Yeah, I was actually, so my first website was created in the edit of DOS.
So we are talking in text mode.
And then I was getting into Windows 3.1, opening Netscape, and opening manually the HTML file to see how it looks like.
So that was...
the idea at the time.
And yeah, in my journey, I've seen everything.
So from all the different design patterns of the different languages, like ASP, PHP, then the move to the front, the Web 2.0 stuff, HTML5.
And on mobiles, I'm also a mobile developer, so native iOS, native Android, Objective-C, Kotlin, then Java for Android, then Kotlin.
And I always like the like the the merge of the two words, the mobile and the web.
So I I author a couple of books on the mobile web, then moving into the performance side of that.
So I offer a couple of books on on the mobile web as well.
So I have books on JavaScript.
So actually, I'm authoring right now, even right now in the AI era, like an old fashioned book.
So then that's going to be my 15th book.
How much are you using AI in the writing of this book?
Actually, no.
And that's part of the deal.
So I'm writing a book on vanilla web.
So how to write to websites without the libraries that actually is useful on the AI work because it's better for context and things like that.
So it's better for agentic AI tools to code some web apps without a lot of dependencies.
So that's kind of my journey.
And of course, as every developer, because I'm a developer, I'm in AI, right?
right now.
So there is no other way, no other option.
So that's why I've started doing even some courses on working with ChatGPT three years ago.
So as soon as the API was released from OpenAI, I started doing articles and content and courses on how to integrate your websites, your apps with OpenAI APIs, how to do prompt engineering for your apps, things like that.
And as soon as Also, ChatGPT, because I'm saying ChatGPT because it was the first one.
When they created the first browsing plugin, I started doing research on how is that working.
So how is ChatGPT rendering your website?
How does that work?
So if you want to optimize your content for that, how to do that?
So I started writing some articles on that.
Technology is evolving every day, actually.
We know that.
So there is a fever right now on that.
So I'm trying to keep updated on every new technology, design pattern, tool that will help developers to merge into this AI world.
Yeah, no, amazing.
And I, yeah, it is, you know, fever is a good, is a good word for it.
There's definitely, you know, some intensity around the pace of change of it that is exhilarating and daunting at the same time.
So what triggered this conversation is your recent work and focus on WebMCP.
And I found that super interesting as I think about, you know, the problem at a high level resonated with me that it felt...
agents keep needing to reverse engineer a web page, and they arrive there, and they try to kind of find the signals on it.
They're doing it impressively well, but it still feels somewhat inefficient.
So it really kind of caught my attention when I saw the title, and I think you had a big tweet that got some big reach around it.
So maybe let's start the conversation by just digging into WebMCP.
Tell us a little bit about, first of all, what it is and what problem it's trying to solve.
Okay, yeah, sure.
So WebMCP, first, is an experimental API that is still not stable.
It was presented by the Chrome team.
So it's right now working in Google Chrome under a flyer.
And the idea is to offer AI agents another way to use the web app services.
Because right now, when we're talking about AI agents, we are talking about charge GPT agent mode, Claw, Agent Mode, or Gemini, but also we're talking about Open Claw, that is maybe browsing the website, and also agentic browsers.
So that's JGPT Atlas, Perpexity Browser, even now Google Chrome or Microsoft Edge, they have an AI mode or an Agent Mode, that they can browse the website for you.
But right now, they're browsing the website typically in two ways.
The most common way is just taking screenshots and then say, oh, analyzing with an image model, analyzing the image and saying, oh, I need to click here in those coordinates.
And then that takes time, a couple of seconds.
But then they trigger the click on that portion of the screen.
But what happens if in those five to 10 seconds, the JavaScript moves?
that button.
So it doesn't work.
So then another screenshot is taken and then the agent says, oh, maybe the website was scrolled or the content changed or whatever.
So that is inefficient both in time and in cost because in tokens, we need to pay for that.
Why are they doing images versus understanding the web page itself?
Well, the other way to do that, that's the most common way.
The other way is to kind of analyzing the DOM.
The thing is that the DOM, you know, we've been mostly after the React world.
When you look at the DOM that we are shipping to the user, it's not semantic.
So it's just a list of 100 divs.
It's just a generic tag that you have on a website that has no semantics.
You're actually understanding what's there with just the DOM might not be useful on every website.
So, for example, on mobile apps, on native mobile apps, for example, if you want an agent to use an iOS application in your iPhone, so you can also do screenshots, but also they have some kind of an accessibility tree that is actually really useful because it's the same tree that accessibility tools such as screen readers are using to understand in your app.
So I think that's pretty cool.
And on the web, you can do something like that, but it seems like on some tests, the results are not really good.
So that's why most of them are still using the old way of taking screenshots.
Okay.
So which works, but it's completely competing efficient.
And I think that makes sense, I guess, when you describe it, like if you try to build an application today, it's so dynamic, you know, it is, you know, sort of changes all the time on it, but it's also really hard to decipher.
Like even as an expert developer, opening up that DOM, figuring out what's what without looking at the visual is actually quite hard.
Well, it's optimized for a human brain, actually, right?
So it's optimized for that.
So that's kind of the A-chain is still...
It can do that, but on screenshots.
And that's where WebMCP appears as a solution.
The idea is that WebMCP is kind of an API.
So the web app developer will expose an API to the agent.
Say, hey, agent, if you want to talk to me, to my website, here you have a couple of services that I can provide.
So you register a list of services or tools that you make available to the agent.
And it's just JavaScript functions that you expose to the agent.
So then the agent, when the agent is browsing a WebMCP capable website, it will check all the services available.
And it will try to see if one of those services is suitable for the goal that it has.
And instead of browsing the website as a human, with the eye and the hand touching on the screen, it will just execute those functions.
In this context, the service provider of these functions is the web page.
It's not necessarily the thing behind the screen.
So services can be anything that local execution can satisfy, things like click a button, give me some piece of content on the page.
What are examples of services that you see people starting to build?
The service executes on the client, so in JavaScript, so on the web page, but that service can also use normal web APIs to connect to the cloud, to connect to a hardware using Bluetooth or wherever.
So actually, the simplest example is...
an airline website so you get into an airline website and you are searching for flights so you need you need to go to the one text field that says your origin then your destination then you need to open the calendar or type the date and you know that every airline is different so the calendar is different so the asian has to understand how the calendar works well instead of that you offer just a tool, a function that is search flights.
You specify the schema of the input data.
So you're going to specify to the agent, hey, I need origin, destination, and two dates.
And if it's one way or round trip, wherever you want to define there, and a function.
And you actually pass a JavaScript function to the WebMCP API and say, when the agent wants to execute this, This is the function.
On that function, you can do local stuff or you can go to your own server, your own backend, or whatever you are used to do on the website.
Actually, you can share the functions with your standard UI.
There is a flag that you can check if it's currently an agent executing this or a user executing this in case you want to do things differently.
But you can take advantage of your current architecture.
and just call that.
And you will return asynchronously.
So that's why the agent will wait for you.
And in that wait, you can go to the cloud, you can go to a hardware sensor, or even you can ask the user.
So there are some, for example, if you are ordering or you are confirming something that will have a cost.
you as a web app developer, you can say, hey, agent, you know what?
I would need to ask the user.
So the API includes a way for you to interrupt the agent and call the user.
Yeah, you tell the agent that it should ask the user or do you literally?
No, you ask the browser.
You ask the browser and the browser will say, agent, you know what?
You will wait because I will ask the user.
So in that, it can be like a dialogue on the screen asking the user, hey, will you confirm to actually buy this flight?
And if the user says yes, then you go back to the agent and confirm the operation.
And this is relevant, I guess, when the user is actually actively behind the browser.
So like these agentic browsers, for instance, it would work for them.
But if I asked...
Chachapiti or Claude some question and it went on and it browsed the stuff, then the browser is not interacting with me as a user.
Yeah, I guess that the standard will still need to figure that out.
So how that's going to work.
It is important.
WebMCP as it is right now, remember, experimental feature, works only on visible browsers.
So it's not working on headless browsers.
Got it.
Okay.
So that's an important distinction for now.
It's not the intent of the protocol.
It's just the original, kind of the initial implementation of it.
Exactly.
And so how, so if it's the server side actions, how is this different than having an API?
Like why wouldn't I just have a, you know, an open API spec, you know, that I make available and properly linked from my site and have the agent just programmatically interact with my system?
Yeah, well, that's actually a very good reason, a very good question, because even the same question happens on the MCP work without the web.
So MCP is another protocol that we use to connect AI tools with servers.
And right now, even OpenClaw or other agents, some of our cloud co-work, for example, they're, I mean, you can connect MCP tools, but they can just go and execute CLI.
tools in the terminal.
So they don't need MCP.
So something like that happens here on the web.
So if you have a RESTful API, for example, or another kind of API, or an MCP API, or a CLI, then you don't need to use your website.
Yeah, that's a fair question.
I think it has to do with the kind of project that you have, for example.
Some tools need authentication.
They have that...
They have a lot of maybe local database that you have on the client.
So that tool works actually better in the browser's context for security purposes.
So you don't want to open the API to the outside.
So it's kind of a different way to do that.
But it's important to understand that WebMCP...
it's not working automatically for your website.
So you need to actually go to your JavaScript code and implement the API.
So it's not going to be applied automatically.
Yeah, and I think in that sense, it is the same as building an API.
But I get that.
There's increasingly functionality.
That's why I perceived it initially as something in which the service provider is the page.
The API call, the WebMCP call, the recipient of it is a JavaScript function, right?
That is, there's a code on the page and then it can do whatever wonders.
So at the very least, when you have rich functionality on the page, and as you pointed out, you don't want the agent to just kind of do the heavy lifting of uncovering it within the system, but you want to give them an agent-friendly programmatic interface to it, to your client side.
So at the very least for that, this is novel.
I guess you could have...
You could have hoped the agent understands this with some sort of LLM's TXT or whatever it is, but this is a standardized way to do that.
And then there's some overlap between, for server-side activities, there's some overlap between that and an OpenAPI.
Yeah, there is, definitely.
But, for example, one quick example of why WebMCP might be better than server-side only, Apple Pay or Underpay.
So if you want to finish the payment process and you want to use Apple Pay, it has to do it on the client.
So that works on the client only.
So you're not sending the credit card details over the RESTful API.
So that's one example on why you want to take advantage of the client-side architecture and you want the agent to somehow get into that architecture.
But that's one example.
So that's why typically you will see shopping carts as examples of where to implement WebMCP.
So the shopping cart lives in the client, not in the server.
So then you can just work with that client side and you make an order and that...
that making an order is actually the one that is going to the server anyway.
So the question of why is the agent not going directly to the server and making an order is actually a fair question.
And I think right now it has to do with more following the standard UX flow.
And instead of just designing a web app that is 100% optimized for an ancient, maybe in the future, we will be creating this system for agents only.
So then it will be a completely different architecture.
I think so very interesting.
And you were saying that in the initial iteration of it, it is focused on visible browsers.
And you're right that that introduces also that sort of user interaction.
So some of it is client side.
Some of it is user interaction.
And then, you know, some of it might be convenience or sort of, you know, like a preference that you want to prefer.
You want to provide this via WebMCP versus OpenAPI.
I'm curious about the path or the jump from there to headless browser.
I mean, per your starting point description, you know, like headless browsers also do this.
you know, as unpleasant tree of sort of like trying to grab an image and reverse engineer in a variety of ways, a web page.
And so they probably benefit from even access to content and things like that on it.
What is your expectation in terms of the primary use cases for a headless browser?
You know, I guess, I don't know.
What's your guess on when does that happen?
Most of these agents are using Playwright or Puppeteer.
So these tools that were originally created for automation, so browser automation.
So actually they are creating a headlight browser, executing your web up there.
So JGPD initially was just using a tool that was converting the HTML into Markdown.
So it wasn't even executing JavaScript.
So the initial...
ChatGPT plugin was really simple.
And to be honest, if you're browsing, if you're asking ChatGPT or Cloud or Gemini to read a website and summarize this website and you get the URL, on most situations, they're still doing that.
They're just downloading a markdown version.
of your HTML.
So if your website is 100% client-side rendering, it's not going to work.
So it will say, I can't read this website.
But if you're using an agent, that's like a next step.
The agent is typically assuming the user's role.
And in that case, it's using some tools, like right now Playwright, Puppeteer are the most common one, to try to act as a user.
So in terms of expectation, I think that we need I'm not sure if the Google Chrome team will think too much about that.
So they are targeting how to increase the Chrome agent mode.
So they are working on that API.
So we need more companies, maybe OpenAI, maybe now OpenAI that is the sponsor of OpenClaw.
We need them on board of the W3C to discuss the API for those agents.
I'm not seeing them doing that right now.
They were not part of the web community, so probably it's something new for them.
But Anthropic, that created the MCP protocol, may be also interested in getting into WebMCP.
I'm not seeing that yet.
Is that happening yet?
Yeah, quite yet, but I guess that will happen at some point in the next few weeks or month, and that will trigger some updates in this API.
I think that makes sense.
And I think Anthropics is actually an interesting example because they are actually not that strong around the image analysis as compared to some of their competitors.
It's actually kind of interesting for them to maybe bias in favor of do it correctly.
So you mentioned MCP as well.
What's the trigger behind the WebMCP name?
What is similar and what is different between that and the non-WebMCP?
Well, I think that...
What is similar is the concept.
If you look at the technical details, it's completely different.
So, yeah, MCP works with JSON in and out.
It's kind of, so not WebMCP, the standard MCP for AI agents.
They work over different, there are different versions and different protocols that they work originally.
They were working on HTTP, but using a, using a, technique that was really old that I brought about technique in an ASIA book three years ago.
And now they were using that technique because they were not supporting sockets, web sockets.
So it was kind of a long polling technique over HTTP.
But and also they have a socket version for like binary socket for local server and clients.
So when you have a local thing, you can just talk with sockets.
It's a completely different idea.
And you expose, but the similar concept is that you expose your services to an LLM.
And by the way, exposing the services is just typing a name of the service and then a description in English.
And the LLM will understand that description and the schema.
that you want and that's a formal schema so I really need four arguments.
The first one is an integer, the second is a string, blah blah blah.
Well the same idea is here that you cannot just export your MCP into WebMCP.
You need to write it from scratch because it's a completely different architecture.
This is JavaScript, WebMCP is JavaScript based.
It's actually pretty simple from a coding point of view.
You just say navigator.model.context.registertool And you pass three arguments.
The name, the description in English, like a long description, and a JavaScript function.
And that's all.
Yeah.
So the tool, the agent, will actually query on all those tools.
It will read descriptions, and you will say, oh, I need to change a flight, a reservation.
Okay, you will look into all your services to see if any description matches that behavior, that tool.
and it will say, oh, this one, and it will execute that function, passing the arguments that you requested as the input schema.
That's roughly how that works, and your API will return some data.
It can be a Boolean, it can be an integer, it can be a message, an object, and that object goes in JSON back to the agent that will process that with the LLN.
Yeah, it actually, I think simplicity is, it's great to hear that it's simple.
And I think it's actually probably one of the things that helped MCP also get adoption because you create something that is very streamlined and slightly capitalizes on the fact that you can communicate all sorts of complexity and things like that in that natural language line.
and rely on the fact that you have a consumer there.
And the same goes for the response, right?
You can reply.
The whole complexity of what is it that you are returning can be simplified because of that.
And in that sense, it is a lower lift than any even REST APIs, which were already in turn a simplification.
But as compared to that, it is still simple.
So it just lowers the barrier and just allows more of these types of interfaces to be created.
Yeah, sure.
Also, there is even a simpler version because you can also use the declarative version.
In that case, it's just HTML.
So you don't even need to write JavaScript.
If you have forms in your HTML, you add some attributes in the form and you're specifying to the agent, hey, if you want to trigger this action, this is the form.
And here you have the fields that you need to fill for triggering this action.
So WebMCP also works in HTML without JavaScript.
And something interesting is that it's also adding some ideas that we don't have right now.
For example, as a web app developer, there is no way to know if your website is currently being managed or used by an agent.
So there is no way.
So if you want to detect for whatever reason, even for analytics, you want to detect that it's actually an agent using your website, there is no way to actually do that.
So on X...
Twitter, they changed something now to stop Asians for publishing content on Twitter.
And they say that they are actually verifying if there is an actual finger touching the screen, which on the web, I mean, you cannot actually do that.
I mean, you can check with the touch events.
No, it's your risk.
It's always like that.
Yeah, but basically...
With any tool like Puppeteer or Playwright, you can emulate the touch and the website will never know it's an ancient.
Well, now with WebMCP, we have new events that you can listen in your website over the window object.
This is the JavaScript technical stuff, but there is tool activated and tool cancel.
That means that the tool or the ancient, it's named tool within the API.
It's actually controlling the website.
So then you can change the UI.
There is also some CSS pseudo classes that will let you change the user interface a little bit when the agent is in charge.
So then if the user is actually seeing the screen, you can tell the user that the agent is in charge, despite the browser UI that is also making that notice somehow.
Right.
Yeah, I think so.
It's interesting to think about the drivers for adoption, I guess, of people like why would they bother writing this?
And I think there's one, like maybe I'll go on a tangent a bit and then come back to that question, which is there is a broader trend in the world of AI.
of unifying things that we use to split apart.
And so the separation of the API from your web page is something that in many aspects is good architecture, right?
Like it is the right way to do it so that the two are not coupled and you build those.
And in a sense, here you're kind of building an API into your web page or you're intentionally violating it.
But in agent land, when you start creating multiple entities that the agent has to orchestrate within them, it gets confused.
And so, for instance, there's been a big rise of monorepos, right, and pulling everything into the repo because then, you know, you can just have the agent load the repo and it's all within the reach.
And so it's interesting to think about, like, one of the advantages of a WebMCP is that it puts the API or it kind of puts the actions within the web page and it's a more contained unit and therefore easier for the agent to...
interact with, right?
But coming back a bit to my question there, it was like, what would you say are the top motivations, and I think you've alluded to a few, for someone to embrace this?
Like if I'm, who would be the first cohort to come in when this is no longer experimental, which this is the island, so maybe that happens faster than we think.
Who would be the, what are the strongest motivations?
to put in the effort?
I think that the strongest motivation will appear on e-commerce websites.
Wherever e-commerce website you have, I mean, you want to sell products or services.
You actually don't care if it's a human or an agent, so you want the money from the consumer.
And so that's, I mean, that's a motivation to actually offer as many tools as possible, as fast as possible, so they can actually purchase your products or services quickly.
So I think that's the first one.
When you look at other kind of content, I don't know, you have a blog, you have a newspaper.
Actually, I'm seeing the opposite sometimes.
And so I'm seeing authors actually rejecting agents.
So I don't want the agents to come into my website.
Yeah, more about IP.
Yeah, because IP or because the user will not get into my website.
They will just get the content.
I mean, the authority gets out, so there is no credit of that or not enough credit or whatever.
I mean, you can still apply WebMCP there, but I don't think it will make a lot of sense.
So in a blog or in a newspaper.
But if you have anything that has to do with service support, so if you have a problem with...
I don't know, your phone bill, then your phone company, your Asian.
If you are asking your Asian to go and try to solve the problem, I mean, working with support tickets and all that stuff can be also another pretty useful use case.
Either way, it sounds like it, and by definition, you know, or by design, it is oriented at interactive pages.
And so pages that are...
that are predominantly content beyond their own agenda and sort of like or dislike of agents, it's also probably a little bit less useful for them.
Well, if you look at the API, when you use the declarative API, it works on forms.
So if your website has forms, then it can be a target of WebMCP.
If you don't have any form in your website, it seems like you don't have a use case.
Yeah, there's no real need.
What about web testing?
So, you know, the other entity that suffers from the ever-changing visuals of websites is the kind of pursuit of end-to-end testing that doesn't require you to rewrite your tests with every pull request.
Is there a use case for this, for, like, WebMCP to be...
It can't do the visual testing, but to do like a navigation, you know, to the extent that, you know, you've blended, that it's the same form that is now marked as tool, right?
Or the same JavaScript function that is called by the web page and the tool.
Right now, when you're doing end-to-end testing, we are talking about also tools, using tools like Playwright and Puppy Tear.
Those are the same tools.
as the Asians are using these days to browse the website.
But I think that WebMCP is more for unit testings because you are actually testing a function and see if the function is working properly.
And it looks more than that, like the end-to-end human user testing because they have to do how everything looks like.
So I think that you would still need to do that thing.
That doesn't mean that you can not...
start using, you can start using also LLMs and maybe local eMash models to improve the speed and accuracy of the testing.
Because it's also possible that right now with some very small models that can actually do web testing pretty good, even locally, without consuming cloud tokens.
Super interesting.
So let's actually shift into this sort of local models path.
And just to sort of wrap up a bit, this WebMCP, it sounds like, you know, you should, like, I guess kind of everybody trying to kind of keep up on this sort of content.
So if you have an elaborate web applications on it, you should stay on top of it.
It's in experimental mode, which, again, I assume it will graduate out of.
appropriate to the pace of change that is in AI.
That's probably a reasonable bet.
And it is currently focused on interactive websites and visible browsers.
And I guess kind of the people that need to be most attuned to it are probably people that are either looking to block bots and things like that, like that post on X, or people that want to enable browser-side functionality like purchasing and e-commerce.
Yeah, sounds like a good summary.
If you want to try it now, you need Chrome 146.
and enable the flag, and you need to install an extension that is called Model Context Tool Inspector that will let you debug your WebMCP code.
And it's pretty simple.
So you don't need to spend like two months of training or it's actually pretty simple to implement.
Hey, everyone.
Hope you're enjoying the episode so far.
Our team is working really hard behind the scenes to bring you the best guests so we can have the most informative conversations about agentic development.
whether that's talking about the latest tools, the most efficient workflows, or defining best practices.
But for whatever reason, many of you have yet to subscribe to the channel.
If you're enjoying the podcast and want us to continue to bring you the very best content, please do us a favor and hit that subscribe button.
It really does make a difference and lets us continue to improve the quality of our guests and build an even better product for you.
All right, back to the episode.
Yeah, probably the harder question is to sort of think about this mindset and say, what are the actions that you want and what are the user flows as opposed to the technical complexity of it.
Yeah, correct.
And so let's move a bit to that sort of model.
So the other topic that I've sort of seen you, you know, kind of discuss and, you know, be kind of tracking at the cutting edge of is this whole world of models within the browser, you know, actually.
We talk about giving an interface now to an external browser that is agentic, but this is literally within the web page, I think, browsers.
So first of all, tell us, I know there are two modes there, right?
Of sort of ship your own model versus something that's in the browser.
What is this world of browser-built models?
As of today, most, if not all, AI-based API or apps powered by AI are using the cloud.
I mean, it seems like the normal situation, you're consuming tokens from Gemini, from OpenAI, from Cloud, from whatever provider, cloud provider that is actually inferring your models or the models you want in the cloud, and you're paying for that.
But that's cloud AI.
That's the standard today.
But there is another way that is growing every year that...
Right now it's called WebAI.
I'm not sure if all the developers actually are using that name or client-side AI.
And when you go to that with the WebAI, the idea is that you will execute AI models, even LLMs, locally on users' devices.
So we are not talking about executing locally on your web server.
We're talking about the client.
No.
the user's device.
And for that, maybe you're thinking, well, but we don't have the power of ChatGPT.
Yeah, of course, we don't have the power of the latest Opus 4.6.
But the thing is that on a lot of cases, you don't need that power.
Mostly when you are integrating AI in a web app, maybe you just want to, for example, categorize a post.
You want to check if the user is using any height, in the text or if it's adding insults or wherever you want to filter that and you don't need the latest opus 4.6 to do that so you can use a very small model that it works perfectly for that and it's going to be cheaper for you because you don't need to pay for the cloud and you were talking about like uh just sort of clarifies this is about like a web model so it's different like a part of this don't pay for the cloud run locally is already available as well by running actually sometimes even like decent sized models, not Opus, on your machine through Olama or like one of these.
Or LM Studio, yeah.
But this one is something that's in between or like, well, it's like maybe another variant.
To be honest, it's the same.
The difference is that with Olama or LM Studio, you are downloading.
your models to your computer.
In this case, the models will be downloaded and executed in the browser for the user without the user even knowing that.
So the user doesn't need to install anything.
The user doesn't need to understand about models.
The model is already on the user's computer, and that's the first mode to execute this.
That's a built-in model that right now that will be basically be available on a lot of browsers.
Today, only Chrome is supporting that feature.
That's built-in AI.
So in that case, if you have Chrome right now on desktop, this is on desktop right now, Android is coming, right now it's on desktop only.
Chrome can download a small version of Gemini, Gemini Nano, and it's available client-side to infer from JavaScript from any website.
So then if Gemini Nano is even smaller than Gemini Flash, the one that is on the cloud, but for a lot of, for example, you can translate content.
It's pretty good at translation.
I think that the test is like there are 25 languages where that passes the test, so it's pretty good.
And you can translate content completely client-side just by...
using a JavaScript API that talks to a local model that is already in the user's Chrome.
You said Chrome can install.
Like, is it if I'm writing a web page, can I assume that it's there?
Because by default, when you install Google Chrome, the model is not there.
The first website that requests the usage of this API will actually say, will download...
the model for the first time will install the models on users' devices and it will be available for the rest of the website, for that website and other websites.
How big is it?
I think it's around 8 gigabytes, something like that.
So that's why it's not coming with Chrome.
Yeah, it's not a small investment.
Yeah, exactly.
But once it's downloaded, it's between 4 and 8 because actually there are three models, no one.
So based on...
what are you requesting, it will download one or the other.
But it's between four and eight gigabytes.
So yeah, it's more than Chrome itself.
So that way it's not built in with Chrome yet, but maybe in the future, it's going to be built in with the OS.
So maybe we are heading towards that future, right?
So then the browser will just ask the OS, hey, do you have a model?
And the OS will say, yes, we have these models.
And then the website will just execute it.
locally.
So I think that we are heading towards that future.
And so that's one way to execute models locally, to use built-in APIs that today is Chrome only, which is good, but not good enough to say I will use this and nothing else.
And the other way is to use libraries.
And today there are some low-level APIs on every browser, including Safari, including Firefox.
and that's WebAssembly to use the CPU, WebGPU to use the GPU, and now on some operating system you have WebNN, Neural Network, that actually if you have a computer that has a special AI chip inside, it can actually use it from JavaScript.
So on top of these three APIs that are low level, there are a lot of open source libraries.
like eight different libraries now from Chrome, from different providers that you can use that can run models using CPU, GPU, or AI chips, the TPUs, or there are different names for that, or NPUs.
And so that means that your JavaScript, your web app, can download an open source model.
You can download Lama.
You can download Gemma from Google.
You say, well, it can be...
Now you have some versions that are half a gigabyte, so 500 megabytes.
You can download that in JavaScript and then execute it client-side on an iPhone.
It works in Safari, in a website.
So again, it depends on what do you need to do.
Maybe it's not for using for a therapist, okay?
Or for...
for teaching users history, because those models are not pretty good for that.
But for summarizing, categorizing, to actually, for support chatbots, you can create your mini RAG.
So RAG is like an architecture that will let you connect your data with an LLM.
So you can make a support chatbot that talks to your own information with...
those LLMs.
And in that case, the inferring cost happens client-side.
So it's been executed client-side.
And you always need to fall back into a cloud API.
So now Google is offering a Firebase API that will execute client-side.
And as a fallback, it will execute a model server-side.
If for some reason, yeah, the model can be an infer or executed client side.
Yeah, I think that's super interesting as you think about sort of future capabilities and use cases for it.
It sounds like right now, I mean, the numbers sound a little bit scary until you compare them to like a YouTube video, right?
Or stuff like that, right?
Like it's not really that massive a deal to download, you know, half a gigabyte, right?
Or something like that.
It's not something you would want to...
every page loads to require.
No, but remember that also you can use APIs to store that offline.
So then if it's a recurring user, it's just a one-time download.
But it's an interesting opportunity to provide things that are, you know.
I guess kind of reduce one optimization or value proposition that you mentioned is one of costs.
And so you might not need to worry about everybody hammering your support bot or your translation bot.
The two is probably some element of latency.
I guess that depends.
Sometimes the cloud servers will be faster, but there might be cases.
Well, it depends on the case, actually.
For some reason, for images, for probably the latency, it's equal, so it's the same.
For text, it depends on how much text you're sending.
If it's very small, like you are adding a label to something, sometimes the local model works really fast.
Yeah.
And then I guess on top of that, there's also like the privacy or the security aspect of it, which is if you want to allow in certain SaaS applications, you know, to allow...
some functionality to happen only on the client side, right?
There's a bunch of these, you know, end-to-end encryption functionality and things like that.
So the server actually never sees the data at the moment that, you know, might get translated to, and therefore you cannot use LLMs on it.
But if you're running them locally, those might be reasonable, right?
Within WhatsApp, like within your app.
Yeah, also we need to maybe stop thinking on LLMs as the only solution.
So there are models.
that are specifically targeting one use case.
So for example, Apple released an open source model to detect, so to do some kind of image detection.
So you can put something on the camera and it will tell you, oh, this, that's icebreaker means cinnamon.
Yeah, I like those ones.
And that is maybe 200 megabytes.
It's a model of 200 megabytes and works client-side.
So now you can have an OCR.
system that can detect objects completely local.
So it opens opportunities for a lot of apps and web apps that can run even without internet, just offline, that can be very specific for some use cases.
And it actually works.
And you don't need LLMs.
You need a model that was specifically created for that purpose.
Yeah, it's interesting.
It's probably...
You needed LLM in the ecosystem probably to create those models because they probably are distilled versions of a larger model.
Yeah, sometimes they are.
But the L, you know, like I guess in this case, it's neither L nor L, right?
It's neither large nor language, you know, necessarily.
It is kind of an image model, which I guess is a language model of sorts.
But yeah, based on what you said, right now you can download, there is a new version, for example, of Q-Win, one of the Chinese models.
The 0.5 billion parameters, that one is really small, and you can fine-tune it.
So then you can have a very small LLM that is pretty bad enough with facts, but you can fine-tune that model.
In your computer, you don't need a very large computer to fine-tune that with your information.
And then you have a pretty decent LLM, small.
that knows about your stuff and can be executed locally on every device, even mobile phones.
I think that is important in terms of the trajectory of the industry.
Because I would say that today, you're probably on the cutting edge if you're doing that.
And you need to be quite thoughtful.
There's probably, of all the people listening and engaging with these technologies, probably a very small fraction have use cases that they need right now that truly match the kind of...
cost as in effort versus reward on it.
But I do believe that open models, and I don't think I'm unusual in that sense, that open models are getting much, much, much better and they will get better at AI speed.
They tend to lag six-ish months behind the world.
So tuning in might further close the gap if you have enough of a specific use case.
But even without it, I think as a competency to build, if you're building a new business, if you're building a new kind of an interaction approach for your application.
These are probably core capabilities you should keep in mind, thinking about what needs to go to the server and what goes on the client.
I guess not dissimilar to the thought of the responsive websites, but then even more like the rich, the progressive web apps that run on the client side, where a lot of server-side functionality has moved to the client.
Right?
Yep.
That's right.
Super interesting.
Are there examples of like, you know, who have you sort of seen out there that's like most interesting in terms of using AI right now?
That's maybe like...
By inside AI?
Yeah.
So I've seen a lot on the customer support bots thing because they found that they can, I mean, their tokens bill can get pretty high.
Mostly when they're...
They're hacking their prompts.
And so at some point, they're losing control on the money they're spending.
And so I've seen that they're interested in seeing if they can replace that with client size.
So if you want to hack your prompt, go ahead.
It's your computer.
So it will be you're hacking your own model.
So I don't care if you do that.
So I think it's about cost at some point, right?
Everything is fine when you're doing your first project, your release of project, your MVP.
I mean, yeah, maybe it's $50.
You're okay.
But when you're scaling that, maybe you receive a bill of $100,000 of tokens that you need to pay.
And when you look at that bill, you say, well, let's see if we can cut this without like...
changing the quality of the service by moving some parts to the client.
Yeah, it's so interesting.
It's like, it's funny because it feels like a, it's like an escalation level, right?
You're talking to the frontline support, which is your local model.
And then, you know, it's like, no, no, no.
Let me talk to your boss.
Let me go to the cloud.
Yeah, exactly.
To the cloud side, you know, to get it.
It's also interesting how support has become a bit of a role model in terms of identifying trajectory of application use for LLMs because probably, you know, you can debate sort of software development, but, you know, you see the pattern over there as well.
But in support, you definitely sort of saw that, you know, be the first role to...
I guess you can sort of say be displaced.
And in software development, I have this sort of intro slide for TESL on it, and I talk about agentic development kind of traits.
And one of the ones I have on there is that it's cheap and expensive.
Because like it's initially cheap because a single person can do so much, you know, so it's so much cheaper.
And then you get the bill.
It's like, do I need to be spending that much?
Can I do this a bit more cost effective?
But most people are not there yet.
Like most people are still in the discovery phase.
And so it's interesting to think about support being kind of the forerunner over there.
This is getting better and better every year.
So still, I mean, if we look at some data that we don't have, I think that probably companies using client-side AI are less than 1%.
But it's getting better and better in terms of quality and cost and performance.
So if other browsers, I don't know, maybe Safari without the new Apple intelligence contract with Google, maybe they will also do something similar in Safari.
So they might get a local model on the local side.
So if that happens, well, maybe we will see more built-in AI APIs in web apps.
Yeah, I think that.
Makes perfect sense.
And it's almost hard to imagine how that doesn't work.
I guess one thing we didn't talk about in the context of WebAI is the sandboxing of the browser.
And I guess if you operate in this fashion as compared to an Olama or whatever it is, you're still working within the sandbox.
So if you have two tabs, I guess, the sort of sharing capabilities that browsers support, in theory, each of them will have its own copy, its own sort of data set.
malicious website wouldn't be able to sort of siphon off, you know, there's probably going to be some set of vulnerabilities, but to the, you know, putting those aside, wouldn't be able to siphon off another tab's information.
Yeah, correct.
That's correct.
Yeah.
Also, we need to remember that the LLM is still a today black box, that you say an input that it gets you something back.
So if, in this case, two websites will send different inputs and they're not tied.
together in any way.
It's just two prompts to the same LLM.
So the security thing is going to be the same with or without the LLM.
That's what I'm saying.
Yeah.
But it's interesting to think about.
I think so all this conversation really revolves around the browser as the AI sandbox, right?
And how you can...
Have the browser, since the browser is the entry point to the world and to your systems and to your data in so many fronts on it, how do you get it to interact with agents?
And increasingly, we have the rise of the agentic browsers, and so very much the AI sandbox.
So that's in the WebMCP land.
and create better interaction points, many of which revolve around security, authentication, purchase.
It's about trust delegation, sensitive actions that you might bump up.
And so I think that's very interesting.
And then you can augment all of those with some local LLM action that is part of your browser, which is independent of the other path, but they also point to the browser will have LLM capabilities itself.
So maybe in the first layer, it's more about...
like a brokering or an interface layer, but increasingly it might actually embody the engine to perform activities.
So from a website builder perspective, first you need to think about how do agents interact, and then you need to think about how do you embed agent functionality literally into your local web.
Yeah.
Also, I think this will become more important because even when some people think, again, for the probably nth time that the web is dying, So if you think about that, by coding and all these new apps that are appearing because AI is coding them are web apps.
So actually the web is growing, not dying.
So I think that that means that we will have more opportunities than even before then to actually start thinking and using these new architectures.
Yeah, super interesting.
So I guess, you know, maybe to close off.
if you can kind of take out your crystal ball.
And, you know, we've sort of seen the journey on how responsive web applications...
I'm not even sure if I'm using the water.
That was the name that I moved a little bit away from the web in my world.
The Rich Internet Application.
Rich Internet, right.
ROAs now, not the RWAs.
The Rich Internet Applications have become, I don't know if they're the norm everywhere, but in many facets of applications, they have become the way to build many apps, and a lot of functionality moved to the client.
I guess, what is your, if you...
if you dare to cast your eyes three years out even, right, in the world of AI, do you envision there's going to be applications that really innovate and this will become the norm to have substantial in-browser, in-website LLM functionality?
So I think that, yeah, so in the future we will see less native apps.
and more, at least on mobile devices, because maybe on the desktop there is, now with AI, a new approach, a lot of people are doing native apps, which is a security problem.
But anyway, on the mobile world, I think more web apps, actually, in the future.
Maybe without using the term.
It's just a link, it's just a QR code, it's just an interface that users are using or agents are using.
And maybe we are not going to call them web apps anymore, just AI apps or apps, bi-coded apps, or however you want to call that.
There's going to be web.
And on that, I think that the price and the quality of local AI will improve a lot in the next few years, which will trigger a lot of new use cases of local AI.
Not all of them will be local.
Not all of them will be web apps.
But I'm seeing a lot of them moving into that future that will be faster and probably more performant.
Yeah, super interesting.
And it's funny how every time the web gets stronger, it raises this debate of why do you even need native?
Can I just have it be the web?
And I guess there's a claim here that if functionality becomes more agentic, if the interfaces actually get simpler, because they get very kind of chats-driven, right, or sort of, you know, natural.
Like API-based.
You're picking up that gauntlet again, right?
You're sort of picking the glove, whatever it is, and saying, you know, you think this debate will come up again and the web will win.
Yeah, I think so.
Cool.
Max, thanks a lot for coming in.
You know, super interesting conversation.
I think an exciting new frontier, you know, for the web.
It sounds like it's still...
like today, today, probably still for the pioneers and maybe for the e-commerce shops or those who really care about controlling agentic behaviors.
But again, this is kind of AI pace, so it's probably going to become relevant to many, many more very quickly.
So thanks for sharing the great information and coming here on the podcast.
My pleasure.
Thank you.
And thanks everyone for tuning in and I hope you join us for the next one.