# Optimizing AI Costs with Open-Source Model Sequencing

**Podcast:** The Startup Ideas Podcast
**Published:** 2026-06-23

## Transcript

Okay, you've probably heard of GLM 5.2 that's going viral everywhere on Twitter.
Yes, it's this new open source local AI model that people are saying is the chat GPT moment for local AI.
But no one's actually gone and shown you how to use it and how do you actually set it up.
So I figured I'd bring on my friend Amir.
He tells us exactly how you should think about running GLM 5.2, how you should think about running local models, how that integrates to something called Open Router, how you can use it with your codex or cursor or cloud code.
And this episode, in 20 minutes or less, you're going to get everything you need to know about local AI models, why GLM 5.2.
Welcome to the show, Amir.
By the end of this episode, what are we going to learn?
We're going to talk about, essentially learn about how local models are kind of keeping up now with the pace of these closed models as well and how you can kind of use compounding models or fusion models as open router calls it to be able to do sequencing between a more extensive thinking model and a more execution based model we'll show you how glm 5.2 actually compares and stacks up against other models and how you can effectively use it and get set up with it as well cool i did an episode on local models it was a hit people wanted more tactical How do I actually, you know, how should I think about local models?
How do I implement local models?
How I can actually make this a part of my daily workflow?
So I brought on Amir.
Welcome to the show.
And let's, I give you two welcomes, by the way.
That's how excited I am to have you share with everyone everything.
So let's get into it.
I'm super excited to be here.
Let's jump right into it.
So let's talk about what happened this week.
ZAI came out with JLM 5.2.
And I think this was a big inflection point because we've typically seen with local models, it's either, you know, storage extensive that you can't essentially run it or install on your computer, or you need a better like GPU RAM performance to be able to actually run it locally as well.
Now, JLM 5.2 is also resource extensive, but what we're seeing is with open source models like open source providers, like Open Router or Lama, being able to help you run these models in the cloud.
and effectively being able to essentially pay slightly less for input and output tokens compared to the more closed models.
What I want people to take away from this session is, one, how to actually get set up with it.
We're not going to go through the detailed setup process, but I'm going to just cover how you can do it in Cursor, using Open Router, or in Codex, and then effectively talk about how GLM 5.2 stacks up against the other models, and then how you can effectively use it as well to do model training.
And then we'll do model training, and then we'll do just kind of maybe a quick walkthrough of how I'm currently using it.
You know, I want to be very honest.
These local models still have a lot of work to do in terms of having tool capabilities to be able to have the modalities to be able to see images and conceptualize what they're looking at.
And I'm going to tell you how you can effectively circumvent some of that where you can use other models to explain what the image is back to GLM and then have GLM work on it.
And also just have a very live test on...
how this stacks up against other models.
You know, benchmarks are great.
Personally, I'm not an expert in it.
I don't know what any of these benchmarks actually mean.
The way I do it is off-lapse.
Like, let's build it out and see how this actually looks and how it stacks up against other models.
Sound good?
Yes, sir.
Let's do it.
Okay, so GLON 5.2 came out and essentially it has a 1 million context window and it scores 81 points on the Terminal Bench 2.1.
It's just about 4 points behind Opus 4.8.
And it does quite well on the long horizon task evaluation.
So this is essentially projects you have that you want to run long sequence tasks on.
And, you know, I think it's taking account like the thinking parameters and how it can think through and plan through some of the tasks it has at hand.
So you can see that across all these different kind of benchmark reporting or reports.
GLM 5.2 actually does quite well.
So in this case, it's 62.1% compared to OpenSense 69.2.
What's special about GLM is it's open source.
You can run it locally on the machine if your device can support it, or you can run it in the cloud through the open model providers.
It's a big leap from 5.1.
I personally didn't test 5.1.
I got straight into 5.2.
But from what I'm seeing based off of like Twitter and conversations with people, It's performing quite well, especially on the front end side of just execution based tasks.
I haven't really tested more on the back end resource extensive tasks, but just based on perception and what we're seeing in the reporting, it's stacking quite well.
So when you say stacking well, are we talking like, because like I look at benchmarks and honestly it goes through, you know, one year.
one eye and it goes out the other.
I'm like, I glaze over it because I'm like, what does this really mean?
Like, are we talking, is it like 4.8?
Is it like 5.5?
Yeah.
What do we, how should I think about this?
And yeah, give it, give it to me straight.
Honestly, man, I'm going to say, man, I don't get it.
You know what I mean?
Like I'm, I'm not smart enough to understand how these benchmarks actually stack.
So I'll be honest.
Like for me, it's like, let's just, let's just build it, use it and see how we feel about, you know, it.
how it performs compared to the other models.
And for me, I want to get the best out of it.
So if I feel like JLM 5.2 is strong in one part, but weak on the other, then I'm thinking about how do I actually use other tools or other models to essentially, now I think, like almost like a fusion approach.
And I think Open Router, which is one of the model providers coined this, where it's like, you're able to do like sequencing between two different models to get the best output.
So I'm totally game on, if I can run.
a local model on my machine to do certain tasks, but then call, you know, Opus or Codex to do something else and have them work together.
By all means, I want to be the most token and cost efficient and performance as well.
Yeah.
So on the setup side, I personally started using this through cursor using open routers API.
So how it works essentially is.
You got to go to ZAI, which is the GLM provider.
They created the GLM 5.2 model.
You get an API key from them.
And then you take that key and you go into your cursor settings, paste it into the OpenAI field.
And then from there, you override the OpenAI endpoint with this API endpoint right here.
And essentially from there, you go back to models, add a custom model, GLM 5.2, and you're able to then actually call GLM 5.2 directly.
in essence instead of openai key you put your glm api key from z.ai then you override the api endpoint for when you call openai chat completion with this one right here and then you go back into custom models and add the custom model protocol you can also alternatively do this using open router so if you want if you're using codex you can go to open router get your open router key and then go into the provider, get the endpoint, and then go into Codex, create a profile, and say, hey, I want you to install this model, open source model.
Codex does actually support open source models.
So you're able to provide the details of what the model is, the context window, and then essentially when you're running Codex through the CLI, you can switch to GLM 5.2.
Easy enough.
Yeah, easy enough, you know, and maybe we can have a page or something to show later on where they can kind of follow these instructions.
There's a lot of guides on Twitter and online.
You can follow those.
But essentially, in my opinion, the best way to get started with this is just go to open router and cursor and get that set up through and through.
So let's talk about the model.
We've talked about the benchmarks, how it performs.
Really, if we want to just at least put some weight to the benchmarks, I'd say.
if they're scoring it 62% and Opus 4.8 is at 69, you know, that probably means something, you know, for the normal people, we probably won't really know until we actually play around with it.
But I went in and was just looking at, for example, this website we have, this is a small app.
And I built this in, I think, Opus 4.8.
I was just testing it around.
And I started refining the design using Geolome 5.2.
So I was like, hey, redesign the hero section for me or refine it.
There's this section right here with all these images.
I was like, why don't we just do a little carousel style?
So it's fascinating in a way because I personally don't think the local models had this kind of capability to be able to get it so refined and accurate previously.
And I tested the models that we had.
And I find that GLM 5.2 is...
a lot more refined and it's able to follow the instructions on like what you want to do so in a couple prompts i was like hey you know let's do a carousel here let's make sure we you know are able to show the the images and then from there i want you to build out like a bento grid style of all the features that we have now this is all in one prompt obviously you know you can see it's a little bit vibe coded here it has a little side like badge the labels you know and and you can tell but at the end of the day For a local model, if you're running this on a computer and not burning any tokens, it's doing quite well.
I can see how it's staffed.
What is the main benefit of using a local model versus something in the cloud is you don't burn tokens.
The way to think about it is you're buying a machine.
And correct me if I'm wrong, but you're buying a machine.
And we should talk about some of the machines that people could potentially buy.
But you're buying a machine.
It's a cost.
It could be $2,000, $5,000, $10,000.
And then you can just run tasks, right?
So you're building a startup.
Maybe you want conversion rate optimization.
So maybe you say, every day I'm going to feed you customer feedback and every single day I want you to work on the front end.
And you just do that.
So my question to you is, for local models specifically, for people who actually want to be building companies, shouldn't they be basically running it all the time on certain tasks?
And how should people be thinking about it?
Yeah, so I think that's a little bit tough to answer and I'll say why because like this model specifically is really resource intensive.
So a lot of I think existing consumer computers may not be able to run this from what I've seen.
I've been running it on the cloud through Open Router directly.
And what's the cost with that?
Yeah, so I actually was trying to map out the token cost of model training.
about 50,000 input tokens and 85,000 output tokens, to get almost close to an Opus 4.8 level of output, it will cost us $0.44.
Whereas with Opus 4.8, it costs you $2.38.
So there's a big, almost like a big difference on almost like 5X price difference between...
Which doesn't sound like a lot when you're like, oh, $2 here, 44 cents here.
But when you're actually using these things and you're running it all the time and you have pretty big tasks that you're going after and you don't want to be constrained by token costs.
So it's a big deal.
5x is a big deal.
Yeah, yeah.
And this is based on kind of the averages on like the coding benchmarks and what cursor charges you to the API pool.
But I want to be also teacher proofing yourself.
If we got this far with GLL 5.2, I wonder what the next six months looks like, right?
So I think it's also worth thinking about making an upfront investment in your machine right now to be able to potentially download and run these local models so that when we get to GLL 5.3 or 5.5, we're essentially made the upfront investment in the compute and the equipment to now save a lot more in the long run.
with other feature models that are going to come out that are going to be much more extensive because i think this you know we're seeing the the i subsidy on tokens right like we're getting a lot more output out of cloud out of codex and i wonder you know i've personally seen it i'm sure you have too where it's like now we're hitting our usage a lot faster than we before especially when fable came out i ran i remember like i ran it in like in the first day i hit my limit you know totally so you're what you're saying is basically like it you know if you look at uh I mean, if you look at the history of VC-backed startups, think about Uber.
When Uber first came out, they actually subsidized rides and they got you hooked onto the app.
And then over time, they started increasing prices, increasing prices.
What you're saying is, in the AI age, with a lot of these LLMs, they're going to get you hooked into the workflows.
You're going to build on top of it.
And over time, those subsidies are going to go away as they go public and things like that.
maybe it's a good idea to actually invest in running this thing locally because, you know, the price of memory isn't getting cheaper and the price of tokens aren't getting cheaper.
So building it now, securing it while you can might be a good idea.
Yeah.
And to tie this all together, I'd say two things.
One, harnesses that are agnostic on models.
So like, for example, cursor, where you're able to run multiple models across the same sequence of tasks, are going to actually potentially benefit from this, right?
So, you know, I wouldn't be surprised if one, you know, cursor decides to directly support GLM 5.2 as a model provider and lets you kind of tap into that cost saving if you couple it with like Composer 2.5.
So this is where it goes into model training, right?
Where I was, for example, looking at earlier in this task here, I wanted to refine the hero section.
So what I did is I actually used Opus 4.8 to first import screenshots because JLM 5.2 doesn't support vision capabilities.
So what I did is I actually used Opus 4.8 to import screenshots and explain back to me what it sees, right?
I was like, tell me what you see specifically on the front-end design for the hero section and lay it out.
And then I switched to JLM 5.2 to study that layout.
and then actually act on making those changes.
So it's kind of a way to circumvent the fact that you have limitations to what JL on 5.2 can do on image capabilities, but you're able to kind of now train the expensive model to think through the plan and then get the same level of frontier-level quality, but at a much affordable price point.
I mean, that makes sense.
So your recommendation is basically, it's almost like free trade versus protectionism.
This isn't political, this is just an economic theory, right?
Which is when people are trading amongst each other, maybe it's in Canada where you are.
You might want to trade with Florida because we've got good oranges here.
We can't make maple syrup here, so we'll get your maple syrup, right?
We'll get the maple syrup, yeah, exactly.
Make the best use of it, yeah.
Make the best use of it.
You're saying is using cursor and basically you don't have to use cursor, right?
You can use whatever you'd like.
Codex, cloud code, yeah.
One of those to basically say, for certain tasks I'm going to be using local models, for certain tasks I'm going to be using the best in class cloud models.
And then together, ultimately you're getting...
you know, great results in terms of the output, but you're also not spending through the wazoo.
And if you're a token maxi, like you and me are, in the sense of we're always pushing into the limit around anything we're building to get the most out of AI, because we don't want to hire 100 people, 500 people, and stuff like that.
It's helpful to do that.
And exactly, and anecdotally, on two parts, right?
One, internally within our company.
I think Satya at Microsoft mentioned how human capital plus token usage is now a big factor into what they're doing.
A lot of companies are now moving away from having direct access to the Cloud Code API to run the tokens because of how expensive it's become.
So they're canceling subscriptions.
So we're seeing this firsthand with a lot of companies now are saying, okay, cool.
This first year was great.
We had the mandate, AI adoption.
Token maxing, that's how we're going to measure success and that's how we're going to become AI native.
Not that like, wait a minute.
Okay, cool.
We've done this, but we're spending way too much money on tokens.
How can we now be more effective?
And I'm seeing this firsthand too, where it's like, especially now, in a way, you can have some sort of direct ROI between the tokens you're spending within the engineering team because you're like, okay, cool.
We're saving a lot of time.
There's an output.
Engineering is expensive.
We get that.
But now you're providing the same level of harnesses and models to the non-engineering teams that, you know, are one-shotting, like, hey, help me format this email.
And they're using Opus 4.8 high thinking.
They're like, maybe that's probably not the right model.
And that's a governance issue, right?
That's a big deal.
And I'm having these conversations with companies right now where they're saying, hey, can you help us figure out, like, how to build governance and proper education on how to actually use the right models?
And this is where I think model training is a big factor, right?
By the way, you know.
John at marketing, maybe you shouldn't use OpenS 4.8 to just format this email for you.
And just helping them understand that.
And I think, I wouldn't be surprised if in a year from now, companies start, we've been thinking about it as well.
We're like, hey, why don't we just get our own machines and start learning some local models because it's a lot more effective, especially how much money we're spending on tokens.
What's the, like just to play devil's advocate, why wouldn't I just use OpenRouter and call it a day?
Please do, yeah.
Yeah, absolutely.
I think they should.
When I'm on X, I see a lot of people being like, buy a Mac Studio or buy these expensive devices.
For people listening, do they need to buy a local piece of hardware if the price even goes up 2x or should they just use open router and cursor or whatever harness they want?
If I just have this right machine, I'll get to this result.
That's not how it works.
No, you don't need a Mac Mini.
You don't need this equipment.
You can get started today.
And what I love about Open Router and all these other tools is, again, they're so agnostic.
They make it easy for you to be able to access this in the cloud.
They run the models locally.
And it's credit-based.
Load $20, get it going, and easy to set up.
I highly, highly recommend if you're starting to dabble with this and token usage is a thing for you.
get one of these agent harnesses set up now that a lot of them are model agnostic, run some tokens in OpenRouter, get these open models in there, and start vibing.
I love to experiment to see how far I can take this.
What if I plan with Opus, execute with 5.2, and then review with Composer 2.5 or Codex 5.5?
There's a lot of ways, and I think we can be really effective, and I think that's what the smart people are going to be doing in the near future.
Some people are saying, I don't care how much tokens cost because I think there's so much opportunity in building startups and optimizing and AI arbitrage that I don't even care if it costs me whatever.
What do you say to those people who are just basically ignoring this whole open source local AI movement?
I used to be the exact same person.
In our first episode, you're like, yeah, how much does it all cost?
I'm like, I don't know, man.
I'm just vibe spending.
That mentality has changed now.
I can see that my usage limits are being hit faster and my cost is going up.
And now that our team is expanding internally as well.
So I think as a solo person, it's a lot easier to build a case or rebuttal around why you should just token max as much as possible, which in itself is kind of ironic.
You shouldn't be token maxing.
You should be token minimizing as much as possible and output maxing instead.
So my answer to that is if it works for you and you can directly have an ROI that you can show that, hey, I spent $200 and I got a thousand out, great.
Otherwise, sooner or later, the subsidy is going to run out.
All right.
Well, I think that's the episode, you know, unless there's anything else you want to add before we bounce.
Yeah.
I mean, for people that are trying to dabble with it, play around, have some, you know, see what it can do at least in the front end for you and start working back in tests.
And yeah, I hope they learned something from this.
I'll include links for where to follow Amir.
He's always one of my first calls whenever I'm trying out new stuff.
I'm happy that you were able to jump on.
I appreciate you.
We appreciate you.
Give him a follow.
Like and comment this video.
Let us know what you think.
We'll be in the comment section.
Just out there trying to help and learn.
Thanks a lot, Amir.
I'll catch you on the next one.
Thanks for having me.