# AI Agents, Memory Scarcity, and the Death of Legacy Software

**Podcast:** Latent Space: The AI Engineer Podcast
**Published:** 2026-02-24

## Transcript

This crap makes mistakes all the time.
All the time.
It is still just like a like I think of it once again as like a junior analyst, right?
The analyst goes and does all this like really pain in the ass information and you bring it all together to make a good decision at the top.
Historically, what happens is that junior analyst who I once was went and gathered all that information.
And after doing this enough times, there's a meta-level thinking that's happening where it's like, okay, here is what I really understand and how this type of analysis I'm an expert in.
Actually, I'm very good at.
I consistently have a hit rate.
Now I'm the expert, right?
I don't think that meta-level learning is there yet.
We'll see if L ones do it, right?
Everyone who's spending one quadrillion dollars in the world thinks it will.
It it better, it better happen, right if you're spending, you know, a trillion dollars and there's not meta-level learning.
But for me in our firm, that massively amplifies everyone who is an expert.
Because like you have to still do something.
You can't just like slop it up.
It's very obvious to me what it's sloped.
Douglas, welcome to the InSpace.
Yeah, thank you for having me.
Yeah.
I after all this time, I just is it okay if I just call you Swix?
I feel like that's where my brain is.
That's how I known you for so long.
You call me Mule if you are.
I'm a parent.
You know, yeah.
Yeah.
I mean, it's been it's been a long time.
It's been it's been a long time coming.
I think I first met you at like New Orleans or like one of the one of the neurpses.
Yeah, yeah.
I mentioned one of the neuroses in purpose.
I think it was Vancouver.
Right.
Isn't that?
Yeah, no, no.
I think it was like some after party.
Yeah, yeah, yeah, yeah.
And you were like, hey, like who's this tall dude?
I'm like, oh, okay.
Yeah.
Yeah.
Well, I mean, it's just like I I knew about you, and we we've like been internet, you know, pen pals for a long time.
So it was like cool meeting in person.
Yeah.
Yeah.
I think that was the first time I ever met you in person.
So yeah.
Amazing.
I I didn't go to the New Orleans one.
I really wish I did.
I love New Orleans, obviously.
Yeah.
So two New Orleans is in a row.
And yeah, honestly, we should go back there.
Yeah.
Are you guys going to Melbourne or the Australia one this year?
I have I don't even think that far out.
Yeah.
But on it uh, but that sounds pretty interesting to me.
I think I can't remember which one.
There's a there's something in in in Korea this year, right?
Yeah, I think I CML.
ICML.
Yeah.
I think I'm gonna try to go to ICML in Korea.
And I know iClear is I don't know, man, there's so many conferences.
I honestly I hate to say it, I'm not much of a travel guy.
Well, yeah, I mean, I'm I'm glad to catch you.
I mean, I I I am traveling to you.
Yeah, thank you.
I really bring Yeah, so yeah, yeah, yeah.
I did not know that I'll be caught in a snowstorm.
Yeah.
It's it's funny.
I feel like people recently have been coming and they keep getting stuck in these snowstorms.
So yeah, first blizzard in four years or something like that.
Thank you for coming.
Yeah, yeah.
It's a pleasure.
And so you're gonna go back.
You used to be anonymous.
You used to be value meal, but yeah, how I know you.
You know what's funny is that value meal is like the very first one.
That's the yeah, the do you know how I don't know how I noticed you.
I I was just like, oh this guy's seen smart.
Yeah, I don't know, dude.
I mean, I I remember noticing you too.
So it's like, you know, you know, this was in the early like the primordial days of Twitter.
Yeah.
Honestly, I miss those the most.
Uh it was like twenty seventeen, eighteen, something like that.
Yeah.
But yeah, yeah, I remember from Value Mule.
So if you that's like the deepest cut.
If you are even aware of what that is, that is like the deepest cut that you could possibly have.
And then yeah, I have another account, and then I actually have a third account, which is my my main account these days.
Yeah.
So wait, oh, which one is that?
I don't have one.
Okay.
I don't want to dox your other account.
Oh, okay.
It's semi semi-dunking.
Yeah, yeah, yeah.
So so so it is there.
That's it's it's not that's okay.
That's like my oldest finance account.
I think of it as my legacy account.
Okay.
I I you know, I want to have some privacy, I feel like.
Uh yeah, yeah.
So so so now you've gone all in on the brand and everything.
Yeah, yeah, I got a brand and everything.
Yeah, cool.
Same profile pick, you know.
So yeah, so let's let's do a little bit of the Doug story because a lot of people hear about Dylan, and I wanted to just make this the Doug story, make me the fat dollar story.
You used to be a value investor, that's kind of how you you were value mule, and you had a mentor or something that nerds sniped you into semis.
Is that the the story?
No, actually, I solo nerd style myself.
So I I wouldn't say value because we'll well for for everyone who's listening to this podcast, might as well be value, right?
Maybe quality focus back in the day.
But we had this whole thing where we wanted to buy quality compounder companies, and the one I found that nerd signed me, all the like single shot me is I found ASML.
Yeah, and I like fell in love with it.
And then I like after ASML, I just like read about all this stuff, how complicated it's to make these, who are the people who are able to make them, and then I you know, semicopters, the whole downstream is all from there.
But it started with ASML in 2018.
I really fell fell in love with it, and then I read like textbooks, and I just like kept going deeper.
And my favorite part about doing that.
Yeah, yeah.
That's perfect.
That's a perfect one to do.
Johnny amazing.
John's a monster, honestly.
This one, right?
Yeah, yeah.
Yeah.
I mean, the thing that's crazy is he has, I don't know, he has a whole playlist about it.
Every single aspect of what goes into it.
And what what's truly great about it?
It's all science fiction.
Like, that's my favorite thing, is like science fiction exists, other than you know, the talking perfectly intelligent robot, whatever information L1.
Yeah, ASML is all science fiction.
So the semiconductor stuff's always been science fiction, always loved it, always thought it was cool, thought it was the most important thing that we ever made, and yeah, kind of followed from that.
Yeah, I don't know if you know, but obviously, you know, I used to be a in uh the analyst myself.
Yeah, I didn't.
I covered TMT, which is a freaking huge sector to cover.
It's absurdly huge.
Yes, a very large.
It uh like I was bringing Sprint, yeah, yeah.
And like, you know, Viacom.
Yep.
And then there's ASML.
Yeah, yeah.
Yeah, the the N, I feel like the T and the M and the T are actually three completely separate industries.
But once upon a time, I think in 2000, they were kind of really close together.
Right.
Yeah.
But but ever since then it's really split off, yeah.
Yeah, yeah.
Well, I mean, I just just my reflection is like I used to be in I I used to be, I guess, our tech sector guy, and like I did the flights to Taiwan, and I took those meetings with like Credit Suisse and all those of those guys that would, you know, tour you around and all those.
I never really felt like I got it because I was always being f filtered through like investor relations and all that.
And I think you have to do what you did where you sort of go muk mode into like textbooks and stuff and like actually learn about the the tech.
But then you you hard it's like really hard as an investor to like make the connection to okay, well that was that me for this this quarter, like or at least this this year at the eye.
Yeah, right.
Because like one, there's like just so much foundational knowledge, and then and then you're like, well, okay, everything here is taken for granted, it's already priced in.
So like yeah, you you you assume that all the Taiwanese people who are buying and selling the rumors of capacity are pretty well informed.
You assume all the people who are TMT investors in the United States are pretty well informed.
I think the thing that was like the foundational difference for me is like, you know, real thesis around one, I think being young and brash and believing in yourself to be like, no, this is something that's really matters and everyone else doesn't see it, really helps.
But for me, the thing that was like I guess radicalizing was I really believe Morris Lolf was dead.
And I was like, oh my god, not only is it this cool new technology is super hard to make and very interesting and technologically very fun to understand and and like I get it intuitively, but also everything all the old playbook is about to be thrown out because it's been like this is a super mature industry.
You really need these primers about it, like that's how you learned about things back before ChatGPT knew everything, or you had to go and read these primers of all this information, they're like, Oh, it's a very mature industry, it matured.
It used to be really immature in the eighties and nineties and two thousands, but now you're consolidated, growth doesn't go up a lot.
And everyone kind of had this old playbook from the early 2000s.
A lot of people hated hardware.
There's just this perception that semiconductors weren't valuable, weren't as valuable.
Actually, software was the most valuable thing.
Now software is getting shit on, but that's like outside of the scope of this.
But people just thought it was this old mature business that had nothing new under the sun.
Meanwhile, every single day just making a new chip was like science fiction.
People took that for granted.
And when the science fiction ends, because you can't make the chips as small as you c you could, all of a sudden, all those free gains you got go away, and you have to think about it.
And what happened for semicolonductors specifically is it created a lot of pricing power or value for everyone who knew how to make a good chip.
So NVIDIA is probably the best case.
You could talk about parallel compute and all that stuff, but it's not just like they know every aspect of it from the chip to the networking to the design to the scale up, the whole thing.
It it is like, you know, versus it w in the past it was just CPU gets better do burr, right?
And so I think that I had a really deep belief that Moore's Law was Moore's Law was ending and everything would change.
And so coming in with that like thesis at the top level just like made me want to attack every little assumption.
And something that really changed as well.
I and dude, this is honestly my my favorite post I've ever written.
It's like 20, it's like a check GPT 3 and the writing on the wall.
In like to 2020, you know, my early I get an early pitch to for fabricated knowledge, and I'm like, hey, you know, I'm gonna make a really, you know, Moore's law's over, scaling laws seem like a big deal.
If you simplify it all the way through, is like okay, supply, but you know, supply divide, you know, demand, right?
Yeah, demand is growing a lot because of scaling laws.
Supply is actually slowing down because Moore's Law is completely screwed.
That's probably really good for semi-conductors and parallel compute is gonna be a big deal, blah blah blah blah blah.
My conclusion then was that you should just like NVIDIA is pretty much the only one who's gonna benefit.
And and you know, so that's my my my my like good long-range prediction, I feel like I just like I just don't think I don't think I would have expected the magnitude.
I think that that's been the kind of the craziest part about this whole story is like I had all these beliefs and thesis, and like I really, really, really believe the reason why I met Dylan is he's the only person who is as semiconductor-pilled in the entire world as me, is how I felt.
So I remember like yelling at him, arguing about all these kinds of things and like our our DMs and stuff like that.
Was it just online or it was on the old school?
Online.
We met in person Taiwan and no no no.
I I've I've actually only been to Taiwan with him one time, I think.
Yeah, so so I I mean, like, look, we we just met in person, we yapped, we went to conferences, but I think that that's like kind of we were both really early to the thesis, kind of have a different background and perspective.
Dylan is technology first, and you know, obviously the technology matters.
I have a little bit more of a financial background, but always around him, and it was just like you know, he's the only one guy who like cared to the same level.
So, yeah, this the the thing that's crazy is like we called it, we were right, blah blah blah blah blah.
But like the thing that I think at still shocks me all the time is the magnitude of how right we are, you know.
Like be like, oh, NVIDIA was good, right?
NVIDIA is pretty good, and then it's like no NVIDIA is now the most valuable company in the world.
And I think if you had me read that and like truly, hey, I wrote that, I believed it.
Yeah, I still wouldn't have put that together, or like I wouldn't have believed it if you this is one of many theses at the time.
Exactly.
Yeah, yeah.
Like there's so many things.
Like, what else are you writing at the time, right?
That didn't work out, you know.
Yeah, yeah.
We can look we can look back, but I I I I I'm pretty happy with my with my long-term track record.
I really am.
But yeah, I'm just really surprised the magnitude of how everything happened.
Like, it's crazy to me that like co-ops is like a a not a household term, but like well relatively well known.
It was like an exotic technology.
So, all this stuff has been this like learning journey, really believing where technology is going, why chips are so important, and then obviously understanding the big scheme of all the things putting it together.
And so that's the yeah, that was like the early days, and it's I think it's all been downstream of that, like you know, one goaded insight, pretty much.
Yeah, I mean, and probably like a career maker right there, you know.
And I just like I love those kinds of like sort of quarterbacking those career decisions for other people who are also weighing a bunch of things, right?
Like I have ADD and like I I just chase like whatever is interesting, but at some point you just have to like really choose and yeah, like I I think one of my skills have always been like trend following and trend watching.
I think when you know, if we're talking like on my account, like value mule, you know, full of time, like I was always pretty good at trends, like being relatively early.
I remember loving and being obsessed with TikTok in like 20 20 19, and everyone's like, Why are you so obsessed with the dancing music show?
Like stuff like that.
I feel like I've always been decent with the trends, but I think the thing was when you see a really big wave that you have a lot of conviction in, it's worth going all in.
And that's that's kind of what it came down to.
It's like, wow, I see this really big wave, it's worth going all in.
And so I reoriented my life around it.
Yeah, yeah.
Cool.
We're gonna talk about other trends that you've spotted, primarily like the sort of memory cycle, which but also optics, which is an amazing story.
But we wanted to sort of focus this uh for the Cloud Code launch, Cloud Code Anniversary, and you've been a big Cloud Code show.
Yeah, I I am where's the chart with the four four percent of code?
Oh, it's it's it's actually go to the top left.
This one, yeah, yeah.
Oh, you know what's really crazy is we've updated that chart.
I think it's like five now.
I mean, in and like as you know, it's really easy to generate code now.
So, like that that number will continue to climb, but it's like just staggering the rate at which this is happening.
So that's recap for people who let's say that I I think this is one of the most important pieces I I've read in a long time.
And you know, you you let it, and it's and it's it's weird because I I think of you as like an analyst, right?
Like you like one of semi-analysis alpha is that you're kind of like the fun millennial semiconductor firm when everyone else is super boring and old.
Yep.
But like what are you doing, you know, getting so into cloud code?
Like you know, I I shouldn't you be reading reports and stuff, you know.
Like tell the story of your psychosis.
So, yeah, I think here's the thing is if you want to be good at any game, we're we're tool users at the end of the day, right?
If you are a good, if you want to be like, and and obviously this is like outside of my job as semi-analysis, like I have all these other things I need to do to grow and make semi-analysis the best research firm ever.
But like, let's say you're a fund manager or an analyst, right?
Your job is to find information edges and like new ways to put information together that no one else has done.
And so, like, I've always thought it's really important to know the most important weapons grade tool that you can do all the time, which is essentially chat GPT anthropic, all this kind of stuff.
And I've been pretty like I'm a I'm an early adopter in tools as much as I can be.
And like, for example, I've been running the our our case study that we have into Claude Code since it first came out.
Like, you know, I think over a year, like, you know, I want to say March, April, I started to do that.
So which case study?
So the case study for people when we're we're higher, like a financial analyst, like our core research seat or something.
Okay, hey, you know, can you can you take this company and do some analysis, blah blah blah, give us this format back?
And I've been running it through like the agentic things of like, hey, what what when agents really come around, they should be able to one-shot multi-step hard things to do, things that would take a human 24 hours to do, right?
And I always wondered, because I, you know, there's some good submissions and there's some bad submissions.
We pride ourselves in the case study and being good.
And and honestly, I always joked, like, well, you know, they're gonna start to beat the worst submissions.
And so, like, that was our that was always my base level.
I have a base level of k is it better than a chat GPT agent mode or anthropics cloud code, yeah, or Gemini CLI, whatever.
And so I started running these benchmarks a little bit.
And so I was very familiar with how good it could be, but then I was like, oh, it isn't quite there.
Um I vibe coded some stuff on Opus 4 for sure, but like, you know, it was like kind of interesting projects on the side.
It was really hard, it took a lot of feedback.
They would it would mess up, it just didn't.
And then, you know, everyone was freaking out about Claude Code 4.5, and I like took it for a spin, especially around the holidays, I had some free time.
And then I was like, okay, well, like how good is this?
And I just like one, it started like one-shotting everything, right?
Like all these MVPs that like, you know, you have to be like, well, the UI is whatever.
It's like, no, just one-shots it.
And then you ask it to do something better and explains what you're doing.
Like, that's actually really good.
And so I was like, wow, generalized it easily one-shot MVP of these like projects and able to like really build things on top of it because you can trust what it's doing to a certain extent.
And it felt like some level of capability was beaten.
It was very different than what I'd done in the past.
Oh, I also tried Codex 2 before this, like like Windows 5.2.
Never really got it to work in the way seamlessly agentically.
Oh, of course.
So this is af this this is recent.
Oh no, no, no, no, so so so this my re most recent when I was like, oh man, though the awakening, probably December 27th.
Okay.
You know it's in the day.
Something like that.
Something like that.
I'm thinking because it's between the days, and I I got home from Chr Christmas and I was like, my fiance wasn't feeling so well, so I had some time to mess around just by myself.
Yeah, yeah.
And I and then also there's 2x usage limits.
Oh my god, I miss those days.
But I mean now I'm addicted to FAST.
But but look, I I was playing around with these coding agents just like everyone else should or do or should in the space and like quad code versus it codex.
I was like doing you know simple testing to see if they can make a thing, and it never really like one-shotted like a total idiots thing.
And then 4.5 just started one-shotting stuff, and that to me was like a huge difference.
And so I was like, wow, it could just like one-shot stuff.
I have all these interesting ideas.
Is it Excel sheets primarily?
No, no, no, no.
Not Excel shit sheets primarily.
I would say it's usually a mix of like a dashboard or Excel or something like that.
But a good example where I like I I think Excel, it's moderately okay at like let's say one-shotting a basic financial model or like just taking and and putting information from one place to another.
It's not at human level, but honestly, if you know much about investing in the being in the business, it's like is your model, you know, being five percent more accurate, really gonna ever make a good investment decision or not?
No, never, not once.
Like, no one's saying, Oh, yeah, my estimate is always one cent more tighter than everyone else.
That's why I'm good at stocks.
No, it doesn't matter.
It's like sell side is ridiculous, Chris.
Like, everyone's like, I'm bullish because that my EPS estimate is is like 10% higher than the street.
Yeah.
And I'm like, oh, who cares?
Well, I mean, as you know, sell side, if we're gonna do this with like shots and uh across the bow on sell side.
I mean, look, one of the reasons why semi analysis has such a like a successful business is because I think sell side as a concept is very broken.
If you're talking about waves and things that are changing, sell side in a lot of ways is this hereditary child of like let's say 30 or 40 years of banking, where you had you know a company go public, so you needed someone to talk about it to issue securities and selling the stock.
You're literally selling the stock.
You have the but you have to be independent-ish, so your ratings buy sell hold.
One of the biggest sales you could do is like when you're when your company IPOs will talk about you so people know who you are.
That's the the core original part of the sell side, right?
And the problem is like all the research kind of has this like really kind of fallen apart.
It's just not different.
A lot of banking regulations has changed.
And so, like the primary information process, it's like a 40-year-old business model on its last legs.
And so I mean, that's one of the reasons why semi-alls is so good, is because we are not focused on being a one cent EPS thing, which I would argue isn't exactly skill, it's just mechanical maintenance.
Um, we are really good at understanding when technology changes and how that impacts everything, right?
Because it doesn't really matter if one EPS is slightly higher or lower, it does matter if like I'm I'm just giving an example of AMD's Helios rack is super on time and is like out at the gate ready to make tokens on this day, because that's gonna be billions of dollars of difference in revenue for AD, right?
Or some networking technology or something like that, some bottleneck.
Being really right on the timing and the magnitude of those inflection points will make a huge difference in the stocks.
And so that's our business.
We're a research firm, we're independent, and uh, we've had a really good hit rate, and we you know, we care deeply about the tech technology.
Exactly.
Yeah, you know, I didn't mean to characterize you as like no you are young and fun, but also you're extremely damn good.
Yeah, it's like it's almost like a triple threat.
And I always almost wonder if it's like, okay, it's like one, you have like deep understanding of the tech, two, maybe you're like sort of financially sort of uh literate, but also two, three, it's like this like X factor that is like, well, focus on things that matter is fuck everything else.
And I don't know what that is, but that that obviously is the alpha.
Yeah, 100%.
100%.
That's yeah, that that that's always been the analyst PM conversation.
It's like, hey, you know, there really is only one or like three things that actually matter.
Right.
Find me those new things.
Find find me those three things, right?
And then there's all this information.
What's actually what you know, that's the hard part.
But yeah, we I think the thing is like we're really focused on finding the things that actually matter, right?
Like the things that, like, hey, this 30s is better than this 30s, this case doesn't matter.
This one actually matters because now you have a giant opportunity.
And so that's that's what the game is all about, I think, in terms of the research, yeah.
And like the uh, and like a you know, finance perspective.
But on top of that, too, is just like when you do so much research, all these different little industry parts are so hard to understand, man.
Like you go to some networking conference and you're talking to a guy who works at a company with their talking about their new email versus whatchamacallit laser, you know.
I can't even remember what it what emails replacing, blah blah blah.
And you're like talking about all this stuff and their PhDs and you don't, okay.
Everyone has a PhD at the deepest level, and they're all doing so.
You have to understand all these deep understandings of these parts of these supply chains, but you also have to have a big understanding too, because you know this little part at the bottom of this supply chain is actually gonna impact this giant, you know, business at the top because it's all interconnected, but it's so complicated.
Just paying the tuition to show up is very expensive.
So I I think one way I'll bridge this for listeners is that this is the complexity of the problem domain that's ex this extreme depth, there's extreme width, and you have to kind of throw human attention at all of it to find what matters.
And you're you're saying you noticed some kind of breakthrough in December where it was suddenly clicking for you.
I I just really wanted to figure out like the the tasks, the task that I was nailing and the task that is still okay, it's not quick that yeah.
So let me specifically talk about my use case because hey, I am still a stock guy, I can't trade or do anything in semicupter or or AI world, but you know, I do still really enjoy stocks, it's one of the reasons like I'm passionate about it, and it's probably my my defining skill what makes me good or bad at stocks, quote unquote.
You you know, the people who are really like stocks they're like lifers, they just love this shit.
It's it's like an addiction, okay.
So I I'm like, hey, you know, here's like all my positions, and like here's some like thoughts on it can you just like kind of like start copy pasting some notes over and putting all together it's like yeah it does that's what you give cloud code yeah okay so I started doing this and then I'm like okay but like add it make the portfolio run some basic risk stuff and it's like yeah sure fine whatever and then all of a sudden like everything you do is perfect.
I'm like okay well like actually can we like make an investment framework for my investment style and start to grade all this stuff and then like attack it and do stuff like that.
You could just do like iterative work and then I was like whoa whoa this is like a crazy useful tool that systemized how I think really quickly like okay what else can I do with it and the answer is like fucking anything right and my joke on the the podcast is it's all a skill issue now.
And so I I've been I've been doing this systematically for every aspect that I can think of like hey now it's so much easy easier like I was actually a perfect example is this is this chart right hey cloud code is a really big deal everything's one shotting I'm reading everyone going into psychosis like me at the same time on the internet.
How do I actually know what's real and what's I wonder right yeah I wonder right so I'm like okay I heard about the fact that the quad code has the commits, right?
On onto the public onto your your commit and it says, hey, signed off with I'm like well why cloud code, scrape me all the commits, right?
And you know what?
Lo and behold, it pretty much did.
Like, and it's like, okay, well, like I'm looking for this signature right here, a copy paste was like, how would you systematically go about doing it?
Did like a big query pull for all the stuff, pulls all the like every single day.
The API is relatively open.
And then I'm like, oh my God, let's see how much this is growing.
And it's like, okay, chart go up.
And you're like, how big is it as a print of percentage of GitHub?
You're like, chart go up.
It's a huge deal.
And I'm just like watching your, you know, I have like a cron job updating it every single day, blah, blah, blah.
And I'm like, this is a huge deal.
Like, this is the the biggest deal.
I I love watching trends.
I love watching exponential trends.
And I've never seen one even remotely at this rate.
You would are, you know, four percent in like two weeks or two.
It's uh it's a previous attempt prior to you.
But somehow they didn't, they didn't they didn't talk about they just talk about merge rates.
But they don't they don't plot it as nicely as you do.
Yeah, well, and also you want to okay.
You you asked you at the top you asked the question of what is this as a percentage of GitHub, and this this guy didn't.
Yeah.
That's it.
Yeah, and and also, I mean, the other thing too is yeah, I have a lot of those as well.
Yeah, but but I thought the quad code, because I'm just trying to really, really, really focus on that specifically, yeah.
Well, and also you want to get give an example.
Bro, I didn't make that chart.
Opus 4.5 did.
Yeah.
Or or I think 4.6.
I'm like, hey, I want you to do it in this style.
This is a the semi-analysis color scheme.
This is re I like summarized books about visualization and like put it in.
Here's little style tips.
Yeah, here's some stuff.
I I don't I don't even know, man.
It has like it has like, I had it go read like 70 books or something.
I'm like, give me like you know, the it's probably a waste.
Like, no, no, you're I know I know it's a finding part.
It it is a waste.
Look, tokens are free.
The the cost of doing this is nothing.
That's the part that's so amazing.
Yeah, yeah.
The cost of doing this is nothing.
The information gathering and synthesis, like, hey, if it costs effectively the same doing 70 is three, who cares?
Yeah.
Right?
And so I like whatever.
And the answer I'm like, oh, this is too many tokens.
You better like really summarize this into like 90 tokens or something like that.
A really basic whatever, and then you have all the skill.
But like, okay, now you can put all that into a skill of how to make charts in the semi-analysis format using any kind of data, and then you can systematically just push this out again.
I'm like, hey, data analyst, please consider all the relationships you can and you generate information.
Like I think it's that one was not chat GPT.
That was not generated.
That was not generated, which I hate honestly.
I don't like that much that one as much as the guidelines and opening.
Yeah.
And and so you can just that was that was generated.
And so you can just what you can do is you just ask it to do is like, hey, here's all the dates that we have.
Can you like visually brainstorm with me a way to better represent this information?
It's like, yeah, actually I'm gonna generate you a timeline.
Okay.
You can just do things.
And I I mean it's it that that is your catchphrase, right?
Yeah, it that is my catchphrase right now.
You can just do things.
And so people were looking at this from the perspective of people who are coding and they're like, hey, this programming is uh automated, right?
But like all information work is, you know, I would argue coding is a big subset of all information work.
I think there's a a Brian Hobart tweet or something forever ago.
He's like, you know, coding and financial, you know, finance people actually are very like different types of abstraction, but you know, you are doing abstraction.
Excel is a ginormous abstraction.
You're building these relationships and you're describing what you think a financial thing is worth, right?
I think coding is a little harder if I'm being honest with you.
And you're telling me the hard one got automated.
Why can't the easy one get automated?
So I started to ask myself, how much can we do?
And the answer is it feels like a skill issue.
It makes issue, it makes errors on the on the margin, but you can kind of force it into like for me.
I love using rubrics, right?
Hey, I care about XYZ out of 10, score this, and then you can really do multiple things.
It helps with the stochasticness.
Do you put it all in one prompt?
Like the the task and the rubric for the task, or do you put the rubric after all the tests then?
I I actually have two versions of this.
A you can pull all this stuff together, just roll run the put for the rubric or whatever.
Yeah, or you can do the task and the rubric.
It just depends on how you want to do it.
Yeah.
Because obviously, if you put it task and a rubric, then it can iterate itself.
But if you put it after, then it's probably more like they'll pay attention to the the rubric.
Yeah, exactly.
And well, in the other part part of it too, yeah, it it will iterate, but like the context rot doesn't matter.
I kind of like it to be separate because the thing is it's like okay, it needs to be this like fresh look at it.
You have to think of it kind of like it would perceive anything anywhere, right?
It just each context window is just opening it up.
And I think sometimes if you have done if you do it together, it commingles the information to the point where it becomes biased or sus susceptible.
Opus 4.6, as you know, is like super sycophantic.
Like it loves to like say, Yes, okay, yeah, I'll do this for you.
Yeah.
I think having it separate keeps it like keeps some of that drifts kind of away.
And that's like one of the things that I've really personally I like the results better, but it's it's just complicated.
Like part of this is really weird because I am I'm weirdly now opinionated on taste in terms of how you should design things because you can like, for example, the context rot thing, until someone explained it.
I was like, oh my god, thank God someone said it.
This is a huge deal.
There's this like meme where it's like uh these guys.
Um, do you see the meme?
It's like of mice of men, and at the end of you know, at the end of the book, uh I can't remember what your character shoots the other.
I've never read it.
Yeah, okay.
So so one character shoots the other guy, and it's like some guy made a meme about it, being like, Oh, this is after your after your COD code is garbled, you know, five million tokens.
You're like, okay, it's time to put you down.
Because the context raw is huge.
So yeah, this yeah, this is the example where so what what are your compacts practices?
Do you sort of aggressively compact manually or so?
I personally with the what new one, it I feel like I try to do it at all in one context window.
I'm not doing ginormous projects.
The one mil is very new, right?
Like, yeah, one mil are very new, okay, very but it's a big deal too.
Because yeah, because your skills and whatever your cod MD is a percentage of one mil is so much smaller, so you just get so much more oomph, right?
Because the sm the 200ks we're just wiping over and over and over.
That's a big deal.
I think it's a huge deal.
And and also with how the agents are working, the sub agents will have their own c contacts window, and then the pasting kind of like really saves that that big, you know, the one million.
You just want a really high quality uh project within that.
That's the best, in my opinion.
Compacts just kind of start the compression of the noise.
So yeah, yeah.
Mentioning subagents in multi-chang.
Uh so first of all, I wanted to give a shout-out to this thing from Anthropic Research where they were like, here's our production traffic, and they they did a report that was kind of like their their equivalent of the meter chart.
And there's a lot of people saying that, oh, you should, you know, software engineering has PMF, but here's here's the the next list of everything else.
But what if they're all also also just software engineering, right?
Like software engineering is like 50% right now, but what it like there's nothing stopping it from continuing to go to 80.
I think maybe what's gonna happen.
This is like a maybe a giant take.
It has like data analysis in here, which that's what you were doing.
Yeah, that's in my opinion, that is downstream of like that is so so I think how we should think about it is software engineering might all be downstream of chips, which is downstream, like like chips is upstream, and then it's AI, and then it's software engineering.
It is all the extension of that same compute hierarchy.
And I think the like, you know, teaching where machine and code kind of inter or and the world intermingle right now is code, and so that's just gonna be the bleeding language that's used to to figure out everything else.
That's that's my belief.
Like it it doesn't make sense to build, like, for example, this is a perfect example of this.
Is like Excel claud for Excel is much worse than clawed code using Python to use the Excel skills to then deposit into it's all logic.
Much worse.
It's much worse.
Even when all the work they're doing.
Yes, 100%.
Because if you think about it, it's it's a legacy.
Why make a car engine fit into a horse carriage?
It should just be in a car.
Like it's like it's like a backwards compatibility thing where it does work because LMs are like relatively generalizable like this, but why bother?
Because that same abstraction of information on Excel, it's just in that because it's human formatted for us to understand.
And I think that that's the important distinction.
All of this information stuff, all this software stuff is just to be consumed by humans.
Doesn't matter.
Yeah.
If they're just as good at at putting the data together, we should be much more concerned about machine focused of like software consumption.
And so they can like, you know, the the LLMs and the agents can put and synthesize all the information and deposit God knows however you want it to be.
I don't need to make a chart in in PowerPoint or Excel.
It will just deposit it the MATLAB, the MatLob, yeah, Matplotlib.
Matplotlib in a chart to me in an image.
Fine.
That are you trying to use Matplotlib?
Yeah.
Wow.
Why?
Why?
You know, it's better, it's better understanding that code.
Yeah, yeah.
So why ever make a chart again?
Yeah.
If it if it's bigger, it's just like it could be inconsistent with like the other charts that you do.
Yeah.
We must I don't think that much about.
I I don't think we would care that much, but I think one, our new charts are better than our old charts.
Yeah, yeah.
And number two, I think if it increases the speed of information, that matters a lot.
Yeah.
And so I think we're much more so pretty much the new charts will outweigh the old charts because they'll just grow.
So yeah, I think it it it is a little inconsistent.
We have the same watermarking.
Honestly, I think it's better than our old thor formatting, anyways.
Well, the first thing this looks reminds me of is Bloomberg.
I was like, you guys are just like, you know, becoming Bloomberg.
Uh which is a nice view company.
Yeah.
Couple of things I wanted to sort of double click on because th th this is just a cloud cloud code like brain dump for one of the the biggest sort of cloud code shows in the world, which is sub agents and agent swarms.
Maybe I don't know if you've tried I have tried them.
Pick pick either one at whatever you want.
I have a controversial opinion that Claude does not do RL on the agent swarms or agent team or something.
Yeah, it's just an experiment.
It's just an experiment.
Um thank you, thank you.
Because no not eat we exactly, because the pro it's just via prompt and it's actually very bad.
I think sub agents are okay because they usually have a Claude MD to go do whatever.
But the agent team is ho is actually really Okay.
Well you know we can't we can't knock it because it's experimental.
So yeah, yeah, no, no, it is a what do you what do you try it on?
Well c it was like some big data analysis of like many, many different companies with different KPIs into a dashboard all in one.
Um I was like, hey, can you just make this all whatever split up the teams?
You know, speaking of that though, you say that, but can we can we to one agent swarm is actually good?
I have also tried that.
It is a that is actually really good.
So I did some like in oh, example of things that I was never available to me, like internal benchmarking of these models and be like, hey, here's a set of problems I'd like you to do 20 times.
Can you do them?
And then I can measure the performance between them and then like do qualitative, but like what's the difference between X and Y.
Yeah.
That that was completely out of the hands of me, a normal guy, like three months ago.
Okay.
Now it is completely available to me.
That's awesome.
Like I am very I care about this stuff, and now I have the tools that's able to automate and do a lot of this stuff because hey, all the software engineering is like partially automated.
And so I mean, my experience is the 2.5 swarm actually improves the model's performance meaningfully.
The agent team makes it meaningfully worse because there's clearly not RL done.
So it isn't context aware of what's the best thing to be done.
And yeah, so I think I think it's interesting.
I like subagents because it's usually a little bit cleaner on a task to go do it and then come back.
But the agent team is just very they had some post about how they did stuff, yeah.
Where it was yeah, there's there's a bunch of RL for for this.
And I tried it myself.
I thought it was that was like pretty it's cute how they do all these like little games and stuff.
Yeah, yeah.
Also, it's crazy how like the setup you have to it's a lot of compute to just run the swarm.
I think it's like a 16 node of H100s.
Oh, okay.
And you're just like, dang, so you and I are not gonna be running, and this is just to run, and I'm sure there's concurrency available, but um, yeah, I think it's really cool, and that's like I think that that's the sign of what's next, because you know, these agents are gonna get better to a certain extent.
They're they're you know, it's another benchmark and bench like another benchmark to hill climb, right?
But then it's gonna be how many of these together in a bigger chain can you get to work.
That you could argue it's kind of like a scale out of the reasoning problem, too.
Hey, how do you get these like this one agent to essentially get a verified whatever, put it into a bigger process and do more information work?
That that's the next thing, and it's important to have context windows that uh that don't garble up into random stuff, and it's able to do just like good enough with token efficiency.
I think is a huge part of that.
Yeah.
So yeah, that's that's kind of what our experiments have shown, at least in terms of like the agent swarm versus like not.
I think it's very clear the agent team out of quad is an experiment.
But Kimmy later.
No, they're definitely gonna do better.
But the Kimi 2.5 tells you that this is already boom, perfectly great new place to do more work on, completely available to us right now.
I think that's huge.
Because if these agents get any better, like I don't know, I'm never gonna sleep again.
That's not so honestly like I it's very interesting.
This sort of moonshot AI, and then this is a tangent, we're not we're not really gonna focus on this very much.
But you know how like the the sort of AI tigers out of China were were deep seek and Quen and then you were like, Well, who who are these like Kimmy guys?
And and these these sort of newer names, like I this I guess Minimax as well with with VAT and ZAI has been around longer, but only recently much more active.
Yes.
So like I I noticed that Kimmy is much more into productization phase, like as a scene, as opposed to like the Quens of the World, the Deep Seeks of the Well, who don't really care that much.
I mean, Quen, because of how it's it's a touched to Alibaba, right?
Like yeah, yeah.
They they have a way to productize it, but it's like it's like kind of like the Gemini version.
They have so much stuff to do elsewhere, right?
Yeah, yeah.
But yeah, Kimmy, Kimmy's pretty interesting.
It's pushing so hard.
They've got everything.
I know.
They got Kimmy Mammoth, Kimmy Claw.
Kimmy Claw.
Yeah, I know, Kimmy Claw.
I haven't, yeah.
Dude, I was gonna say, have you messed around with OpenClaw?
Because I did I oh yeah, I remember what was it first called?
Clawbot.
Clawbot, yeah.
Dude, I was gonna say it was really, really euphoric.
I was like having it read all my emails and my calendar and do all this stuff, and I was like, wait, wait, wait.
This is really, really, really prompt injectable.
And I was like, this is pretty secure and important stuff.
So I like I was like, you know, Claude Code psychosis is good enough for me at this point in time.
I mean, so what I do is I just have multiple emails, right?
And there's there's a safer email to give to the box, and I can let it use that.
And if it impresses me, then I can upgrade it.
But ClaudeBot didn't impress me at the way.
I'm thinking I'll be honest with you, I wasn't impressed either.
That was the reason why people were freaking out about this mold book.
I was like, bro, have you actually used this shit?
Because it's not even right now on Cloud Code in a relatively focused terminal, it will be like, oh blah, blah, blah.
I'm like, dude, in the.
You still have to wrangle this thing, but it's not like a perfect skill follower.
And the the context in each attention window is gonna like change, and sometimes it'll be lazy, sometimes it won't be, but it's definitely good enough to do a lot of information with.
Yeah.
That's gonna so I I use I use our I use our Discord as uh basically like a uh a way to just bring information in and out.
I I I just thought I saw this to you, where basically like a lot of people are just setting up things that they could have done in Zapier with ClaudeBot because they're like, well, you know, now I'm like AI pilled.
But actually, they should done it more securely with Zapier.
Okay, I think it's kind of interesting.
I guess I I do think it's kind of interesting, but I think there's but the the difference though is Zapier, I mean I remember I've tried to use Zapier before.
Yeah, it's and it's also not very good.
It's not also not very good.
The difference though is like, and that's okay.
Like it's okay to be early to something and just wrong because you weren't the one that made it happen, right?
Claude bot, the Claude Code, Claude Bot, whatever, all this stuff.
The reason why it's so powerful is it gets to completion, right?
And and like, okay, Zapier, maybe you can get to completion all the time, but like man, it probably took you like eight hours of clicking through things and like copy pasting crap to make sure it all works and it's all secure.
And it's like, well, Cloudbot did it, or Cloud Code did it in like you know, four and a half minutes, and that's good enough for me.
You know, that that's a faster achievement.
And so like it's totally okay that they were they're right, but they were just not the right mechanism, right?
You you see this happen in information like in the history of like compute.
I think there's also like an innovator's dilemma thing where Zapier as a pre existing business had this view of the world of automations as like very strict sort of on Rails workflow type things that their giant user base already uses.
They couldn't like really pivot that much.
Yeah, so that's why I I think like you know, one of the co-founders left too because they were like, Well, I can't exist within this thing.
Yeah, like like constraints.
You you end up becoming with you know the the the box will control you, yeah.
Yeah, you you you are it's your it's your golden handcuff.
Yeah, it's just like your cage, you know, you're gonna act like how you are on the cage.
And so yeah, that that sucks for honestly.
That's I feel like that sucks for the framing I have is like your priors become your president.
Ooh, that's pretty good.
That's pretty good.
That's pretty good.
Your priority becoming, yeah.
I haven't bogged that yet, but I should.
You should, you should.
Your priors become your prison.
I like that a lot.
Coming back to Cloud Coach, it's I also want to make this like the sort of cloud code.
No, no, no, that like I want to indulge because like that's how natural conversation goes, and I think people like enjoy that, right?
And probably that's the only time we'll talk we'll talk about Kibby.
Yeah.
So like do you use hooks?
Do you like give me like the the Dougleffin cloud code setup?
I had just like essentially a few base skills, and then I have a lot of APIs, and then we've also made sure to work, and this is like all work in progress as well, to have APIs for some of the semi-analysis information out as well.
Yeah.
And so that way we have an internal server, an internal server that is that that is accessed by people with an API.
So that like all the semi-analysis researchers are able to hit like some basic level of context.
Because I think the context is really what matters.
Yeah.
I'm too like too dumb to be really smart in in order to have well, I guess I guess I do have some hooks if it makes sense.
In terms of like I think hooks are very underrated, right?
Yeah, I I do think Because you can do like a Ralph Loop yes with a hook.
Yeah, like yeah.
I feel like I underutilize hooks.
I I think that is true, but I do I do run some version of them on like skill calls effective, like, hey, on this, then you have to start pulling all this stuff.
But I think in the beginning, I tried to do all this like hook stuff and like compounded stuff like that, and I found that, like, you know, the gas town Ralph Loop era, it's like it it is a sign of what will come, but I just don't think there's enough fidelity to like make crazy multi-turn something happens.
So, like, okay, actually less is more, try to have like a strong set of smaller skills with a good amount of context information to be pulled in, and then at the beginning of every session, ask and focus on what you want to do so that like it prompts the like not like you know a cloud within a clod, whatever.
So here's the goal to finish within this single context window and then get it done.
And this is like my generalized research thing.
Hey, I want to look at the price of NAND since 1984 or something like that.
This is what I want to do.
I want to so like the actually no, let me just give you the best example.
That is probably not gonna work.
I would like to fine-tune a time series foundation model to predict NAND and DRAM prices, okay?
I'm gonna first start by gathering as much information as possible for all this stuff, blah blah blah, and then we're gonna fine-tune it, evaluate which ones we're gonna do.
I chose Coronas 2 because of group covariates, blah, blah, blah, blah.
Try to set this all project up.
We'll make it a versile dashboard internally for for semi-analysis.
Maybe we'll external if we want if it's a good enough product.
Okay.
So then it like does all this stuff, and then I just like start planning away.
Hey, can you go research series of search API, SERPAR or EXA or whatever you want to use, to go look for all these different information sources and then bring it together, right?
So this agent goes and gathers all this information.
This agent goes and like works on like considering the fact that the price isn't perfect to do all this fine-tuning on, and then we like throw it in.
I also had it of like, well, what do I use?
It showed me which GPU, whatever we're renting on an hourly basis.
And so, yeah, we just pull all this stuff together and then we fine-tune it.
And I'm like, okay, cool.
How did this work?
And then we just have this constant iterative loop until I try to finish something.
I got to the point where I was like, okay, this this time series LN is probably not gonna work.
Unfortunately, unfortunately, the You said it was because of regimes or something else.
I think so as regimes, yeah.
But it's so messed up.
For for a lot of people who are like new to finance, this is why I have I have an issue with all these kids doing like stock trading games with LLMs.
They have no idea.
They they've never studied finance.
And like the no some like a past does predict the future a lot until something fundamental change and like the macro shifts and like risk on versus risk off.
They've never heard they've never heard those terms.
Yeah.
I had to explain it to the people at Cognition.
And like, yeah, like the the rules invert, like completely invert.
Like what used to work is exactly the opposite of what you need to do in when you have a regime change.
Exactly.
And it's very, very, very hard because and the other thing too is you you be like, okay, each of these, each of these are almost like a one-off onto their onto their own.
Right, which reduces your sample size.
Yeah, which reduces your sample size.
And so then at the end of the time, at the end of the day, you end up being like, well, it kind of just like, I guess it's here's some heuristics, good luck, have fun, right?
Here's your checklist to see it might be over, but you really don't know anything until then.
So, but but like, okay, an example of of where this project was helpful and it's like, okay, I'm not gonna have the magic LLM to tell me what the price of memory is gonna be.
Hey, it was a good weekend project, and I did burn quite a few tokens.
But I do happen to have after all this like information synthesis and analysis, all of the memory prices of everything I could possibly find, plus the things up behind API that we paid for, plus you know, in enhanced data sources, and I have all the covariates.
So, like, hey, WFE, what was the consumer sentiment, every macro thing of all time.
And you know what's really interesting is I am gonna just be like, okay, well, now can you go make a summary of each and every memory regime and what it looked like and what what what created the beginning, middle, end, and put that in a dashboard so it's relatable and like easy, shareable, consumable within my firm and company.
Yes, I'll probably be done with that today.
And that, okay, so that you're like, well, that's just gathering doing information stuff, like you don't understand.
No one's ever done that in the history of time.
I know for a fact, as the guy who like is like the cycle semiconductor guy, I've written and done more work on the cycles than I think anyone else has at this point, especially for like the older ones, like the 80s and 90s and 2000s and 2010s.
And like when I did it first time, the human grocked way brain is I went and I read these old in your reports and I put it together and I tried a string an error through it.
And I brought through all I'm like, okay, what was GDP?
This growth, what this year, what was all this stuff?
And you have to like make all this giant sheet to come to whatever, and then make the narratives.
No, none of that shit, dude.
I mean, it's just like too much information to gather.
It's like a lifetime of work.
It's like a PhD project.
I did it in a day, two days.
Yeah, I mean, I think the the kind of the pushback would be that then you don't have enough expert information to criticize the reasoning that went into the report that you're slopping out.
And you know, this is there is some slop.
I I definitely agree with the slop of else.
So so I think of it once again.
So right now, by the way, that's also existential for you guys if you get caught doing like putting on some slop to your clients, right?
Like you you have to at one point be like extremely AI pilled and like you know, yeah, number one in the world at applying AI to your productivity, great.
But also, like you gotta you have to so I think I think the thing that's really interesting is that this whole thing is like a game of hygiene now.
Yeah, because I I think it's like this is really hard, and I think about it all the time.
I feel very comfortable with doing all this work because the thing is, my at the end of the day, since I've done the work, I have like a lot of like embeddings in my brain, a lot of information, the vibes that have got me in here is actually like tons and tons and tons of information, setup scenarios and like pattern recognition, right?
But yeah, you're right.
This this crap makes mistakes all the time, all the time.
It is still just like a like I think of it once again as like a junior analyst, right?
The analyst goes and does all this like really pain in the ass information, you bring it all together to make a good decision at the top.
But the problem is historically what happens is that junior analyst who I once was went and gathered all that information, and after doing this enough times, there's a meta-level thinking that's happening where it's like, okay, here is what I really understand, and how this type of analysis I'm an expert in, actually.
I'm very good at.
I consistently have a hit rate.
Now I'm the expert, right?
I don't think that meta-level learning is there yet.
We'll see if L1s do it, right?
Everyone who's spending one quadrillion dollars in the world thinks it will.
It better happen, but if you're spending, you know, a trillion dollars and if there's not meta-level learning.
But for me in our firm, that massively amplifies everyone who is an expert, right?
And we are a firm filled with experts.
And so it's this hard part where I wonder if new people we will be less lenient in terms of like how much AI tools you're doing.
Are you like junior or junior?
Oh, junior to the firm.
Yeah.
You can't just like slop it up.
It's very obvious to me what it's slopped, right?
When it's slopped and there's no cognition, then it's like whatever the artisanal last five percent is like that really matters.
But for me, I know inherently what the five percent is.
I can like write it away with some really easy heuristics and time and like be like, okay, well, this is the last five percent you fixed.
This is what I believe, just fucking make up these assumptions instead, press enter.
Okay, cool, we're good to go, you know.
And so that's kind of the hard part.
That's a real hard part.
There is still a human in the loop right now.
One day, someday it'll be superhuman, but I definitely believe the where we're at today, where we're there, it's not there, right?
You just compound all this noise and it becomes just like garbled, just like all context rot.
But in terms of like the capability that is over hit, like you know, the human CPU in these this agentic swarm is very, very powerful now.
Yeah, you know, a huge, huge, huge multiplier of what you're able to do.
And for me, that was enough to be like the I feel AGI pilled, honestly.
Because if you if I define AGI as many common jobs, not like I'm not I'm not doing ASI that's like religion.
Can it automate or change or or take or you know completely shift a lot of the information work?
Yes, a hundred percent.
Yeah, yeah.
Like data analysis is a perfect example.
Hey, every quarter I want you to just find me some examples of some information that might be interesting.
I just can't imagine if I was an entry-level worker doing data analysis that a 22-year-old, an average 22-year-old would would murder the hell out of a relatively well thought out agentic system.
And so you're like, yeah, that job actually does seem at risk.
And so that, yeah, that the 4.5 capability enough, like that that we hit some level agentically where it seems to work and do bigger information work.
That's when I'm like, okay, yeah, this this does change everything.
And so, yeah, there's there's all kinds of mistakes.
I it's a new level of hygiene that we have to do.
You're gonna have to understand what the absolutely of gentic work is back to you, right?
I catch it making errors all the time, it doesn't always pull skills, like you can definitely tell, like context windows, definitely like it gets dumber over time.
It's not AGI today, but it can do these crazy long tasks.
And as long as you finish it at the end and deposit it as information work, that's very valuable.
Yeah, amazing.
So you do a lot of like climate visits, obviously.
I by the way, transistor radio, amazing for like understanding like what your world is like.
Yeah.
Are you also cloud code pilling your analysts and your I you know on the other side?
I've definitely cloud code-pilled the analysts.
Everyone in the New York office of like must try it.
I like really try to pillow not, I mean, not your like non semi-analysis.
Well, your customers and and all that.
Like I said, my perception is they don't adopt any of this stuff.
Okay, so yes and no.
Some people are interested, but you have to remember it's relatively more conservative.
But I think, but if you ask any analyst if they're using AI, every single one of them will tell you yes.
I use it every single day.
Of course.
How could I not?
This is like an a vital skill.
And so the the the basic the basic inference that I'm doing is I am a bleeding edge adopter.
I'm a relatively smart dude who knows what he's doing and if a tool is useful or not.
And I've evaluated the tool and I'm like, wow, this is an amazing tool that I literally like pry it out of my dead fucking cold hands.
Okay.
I'm like this, even if it's like makes mistakes, I will be using this for all kinds of work forever.
Then I look around to everyone else and be like, most of these guys are enough like me that if they have an opportunity and an edge, they will obviously apply it.
And they look at this tool and they start to use it.
If if they start to use it in their thinking like me, they're gonna obviously adopt it.
I'm like, well, I don't understand why everyone doesn't adopt it.
I would argue, well, we'll see in the 24 month view.
It will be a base level, I think.
I think cloud code, co-work, whatever is gonna be a base level of all information work very soon.
Yeah, and you know, you see one my my friend was telling you how his portfolio manager will have found co-work and he's like getting it to read his emails, and he's like, Oh my god, I love this, right?
Everyone's moment is gonna be a little different, but I think my moment, it feels like GPT 3.5 or 4 for me, where there's that first time where you're like, okay, I know it made some shit up, but like this is better than like if I went for hours searching, putting information together, it can and then also as like the analogy power, you know, where you can say, hey, this is the setup, can you describe it in this?
These like really strong pattern matching skills that are really powerful.
I just think it hits some level of capability.
I can't tell you what it is.
It is like my tape, my personal taste where I'm like, oh wow, this is completely over the the the chasm of what needs to happen for it to be a very, very powerful tool.
And so yeah, that's my Claude Code moment, I think.
There's some kind of automation chart that you know, XCCD has this automation.
Yeah, yeah, right.
And I think I we need a version of this that is the cloud code, like it's being much but it but what's crazy is this the cloud code thing like murders the access.
Exactly.
It just shifts everything like that right or something.
Yeah.
But also like it's what I was trying to figure out is well, okay, it is maybe dumber, less less human attention, but because you can spin it up so quickly and it can sit in parallel so quickly, and it and it gets done, you get more turns at the wheel.
Yes.
Whereas in as a human, you'd you get one turn.
You get one turn.
But it with with with cloud code, maybe you get three turns.
And the the sort of review process is the thinking.
Yeah.
And you just need to get very good at review or yeah, or hygiene.
Yeah, I think of it as hygiene.
The thing that's like really gonna be painful though, is like a lot of my expert opinion has been built by like, you know, it it's like pre-phones and not, right?
Like your attention span, like, you know, the children are cooked, okay?
Like, you know, the attention spans are really bad, all this stuff.
Like I don't I read this like really sad thing, like, oh we're getting dumber or something, first generation.
I don't know.
I'm not gonna maybe that's like you see the the coinbase earnings.
Yeah, it's all the cards.
So so like you have this thing where it's like, okay, and it's cute and all, but like it's such an addictive technology that like I feel very grateful that I'm like, well, I understand what I'm doing, have this history of doing stuff and able to apply a tool, but like people who are riding this curve, it's gonna be very dangerous.
It's like giving everyone I know I'm struggling.
Yeah.
So funny.
I I think you should just do that.
Yeah, he well we we we do, we do with some of the semi-analysis memes, you know.
And and the thing is you say some of this brain rot is like so bad, which is it is terrible, but some of it is also like, you know, it is hitting some attention mechanism in my in my my deep primordial monkey brain.
Stimming you.
Yeah, it's stimming me.
And you're like, you know what, I can't look away from the the subway servers.
So you couldn't look away.
I was I had to pause it.
Yeah.
Yeah, I was like, I literally I won't it well, hey, there's like have you ever been at like a bar when they play like these like weird like there'll be like like TikTok videos for lack of whatever, and you will just watch TikTok bars in New York?
No, not TikTok bars, not TikTok bars.
Okay, it's like there's like essentially a b-roll channel that they'll like sometimes play in public spaces, and you will just find yourself like being engaged with it.
Like there are certain things that just it works.
So Yeah, sorry, that's a completely off.
But but I wonder this quad code pill is very powerful for me.
I believe it will change how it all works.
They'll shift all of that over massively.
The the chart, but it's just really weird because if you didn't pay any like human cognition to get there, I don't think you're gonna be a great reviewer.
One of the reasons why you know what what makes that that human feet that loop well is because once upon a time you did that and you could make the thing like, yeah, yeah, idiot, you're not thinking about this problem in this way.
You're missing this this like you know, whatever.
You're not considering this 90%, you know, like the 10% tail, something like that.
Yeah, yeah.
And so it's like, yeah, I know you said this, but like, you know, the I I know G I know that I told you the valuation is the only thing that matters, but like it's also a fraud.
You can't do both, right?
Like, if you think about like the analysis stuff, you have to know when your own and personal embedded model is like, yeah, actually this one overwrites this one.
That that's through learned experience.
And I wonder if we're just reviewing, we won't be building and embedding those assumptions to understand judgment.
Right, right.
Because you're just checking for mistakes rather than trying to do original thought by just doing the work.
Yeah, yeah.
I think that's that is that is a danger.
Yeah, that's and that's what hygiene sounds like to me.
Like, hey, it's really addicting to be like, you know, whatever, press the button over and over and over.
But sometimes you do actually have to like think, you know.
So I think that that's it's gonna be really interesting.
I mean, have you tried like so?
So I mean, the this the way to model the sort of meta learning as as element is like once a night you do a batch job of like look over everything I've done, like I extract some learnings, you know, and open claw.
I I think one of the interesting things I really liked about it was this heartbeat on it.
Yeah, heartbeat beat.
And I'm like people aren't like excited enough about this because like, well, this is the first instance where like the agents are just always on, like always living, always reflecting.
Yes.
And like what is it sold.md too.
I don't really think it's much more for character and like whatever, but like yeah, hard is hard's the Quran.
Yeah.
I mean, I think yeah, that's a good way to put it.
Yeah, and so like the that's the powerful thing about all this stuff is that like okay, yes, we know that the context like it gets garbled.
We know that open open claw doesn't always do everything you asked to it ask it to do uh initially, but you can see the design patterns, like the heartbeat MD is a perfect example, can see the the design patterns where it's like, well, you know, is all of our tasks every single day actually us having this like genius thing, or do we like sit down in a single session, finish a single project, get up and get some coffee, then come back?
If it's that and you could just fuck you can make the heartbeat.md consider the like the session to session and like hey, meta learnings, all this stuff, and it's only specialized and focused on one form of doing something.
So it actually does have a context of all the like let me I'm thinking like uh customer service agent or something like that.
It does have the context in fact, it can look at every single time it's ever happened.
That's actually information and context no human could ever hold.
You're like, wait, that that feels like H.
Like that's effectively good enough to do a huge information test and have enough context and be able to fetch it and maybe like there'd be some verification to make sure it doesn't just totally mess it up.
But that to me feels like a design pattern that you can build something on.
And so that's the that's the vibe is that we've hit some capability that you can you can do you can build these much bigger blocks now.
And those bigger blocks are not just like this single line of code it might actually be a business.
It's kind of crazy.
Like I I wouldn't have put myself as AGI pill.
I think 4.5 is like actually I think my own timelines have moved up a lot.
Yeah.
Are you guys watching GDP Val?
I to the best I can but I'm feeling like I'm mostly just trying to no no to me when GDP Val came out so I I I'll just GDP Val is like an basically a like a a broader sweepbench let's call it where it's like applied by every discipl every profession that is white collar that you can model and it's above like something like two to five percent of GDP, something like that.
That's why it's called GDP Val.
And they they had human experts do the tasks and as well as GPTs and here's the results right like where 50% is parity with industry expert.
Yeah.
Yeah.
Coin flip exactly where so like you can see the the nice increase from 4.0 to opus 4.1 and since then obviously 5.2 and opus 4.5 have already exceeded that we're at 70 something now.
Yeah.
Which means models are consistently better than industry experts at these things.
Yeah.
So to me, like this is the AGI definition, isn't it?
Yeah.
Yeah.
This is this is and so like I think I think the problem though, yeah, I would say that that is the definition.
So the thing that's crazy is because there's like this ASI element that people are like really really focused on.
Yeah, we're moving the goalposts.
Yeah, we're moving the goalposts, but I'm like, bro, I the the goalposts, like I I I mean, we'll see if this is actually the machine god and Shogith will come and talk to us and vibrate on our same.
Okay.
I I I don't I I'm honest with you, I'm very open.
I will change my mind often.
I'm not this is not something I feel intuitively in my gut today.
Maybe it's the next max X thing.
But when it comes to like the the GDP valve version of this, yes.
Yeah, this is do white collar work, literally the white collar work, which is most of the most of the tampons.
Very boring, yeah.
Knowledge more, like like actually it's almost all not almost all, but it's a huge portion of all of work in the world.
Yeah, it's like now we just made like my favorite stat is like once upon a time, 90% of people were farming, right?
Now today less than one percent of people are farmers.
It's kind of like this crazy shift where technology is gonna massively change the relationship with all of that, and it's gonna be like this 99-1 thing.
I don't know if it'll be quite that drastic or whatever.
Maybe you know, everyone's just doing leisure.
So far, my experience is everyone just works harder in my experience.
But it's just it just feels like a massive moments happen.
Like the the steam engines invented, and you know, the the trains are here, and and everything's gonna change in knowledge work.
And it's kind of crazy.
This is a sort of economic cycle from my macro days.
That I'm I I can't remember the name, I can't look it up, but it's basically like there's this stages of economic development where like your your economy starts out majority agriculture, then it discovers like manufacturing then it discovers white collar work then it discovers that it builds like a very mature financial sector.
And like though these are like like a layer cake that all declining over time.
Yeah then then the new things increasing.
So I my my theory is like there's this like fifth layer that's like has to open up that starts to happen because I do fundamentally believe we'll just invent new work.
I do believe that percent like humans are very adaptable.
That's like my favorite thing I've learned you're able to adapt to God like coldest coldest place in the entire world the warmest place.
Humans are in every latitude that's in a physical sense but I think we're gonna find a way to make utilization go up but we'll we'll we'll invent more work for sure but I think the thing that's crazy is just like things change so quickly and that five to ten year period like 10 year gap can be drastic and crazy and that's just societally wild.
But yeah it's and it's happening in our lifetimes it's happening in our life like it's like happening like right now.
Like it's it's just it's really crazy.
It's like very and like so this is like a complete side task on I'm like really curious of when we start to see it in a much bigger way in the real economy.
That's like my my my pet.
Yeah, where why is it not showing out a GDP yet, right?
So there's gonna be, you know, some people are gonna be like, oh, you know, the facts the facts of the internet, same thing, information transfer, whatever.
I think I'm actually scared for a third worst thing, which is like now I'm now this is a complete crackpot theory.
Please don't hold me to this internet.
But what if AI is massive massively deflationary?
And and and also I think that one of the more interesting conversations I've had in a bit is like what was GDP was invented once upon a time as a way to figure out how much we could divert, you know, normal economy away just to war during World War like one or two or something like that.
Okay.
My spiciest take is I feel like GDP itself is gonna be very, very challenged by AI because information work, yeah.
So how we how we capture it effectively is all of an economic good and then the service, ours divided by ours.
Okay.
So there isn't like a widget to widget difference.
But in theory, if we could break all of information work down into units, we're gonna have a lot more information work for sure, like more work will be done.
I don't know what the value of that's gonna be.
Is it gonna be so much increase in supply it's deflationary?
That seems to be like a real concern.
It's possible.
Yeah.
And then we'll figure out how to use it.
But like there may be a Great Depression of AI, yeah, where like we figure it out.
Yeah.
Well, I I wrote this whole thing about railroad stuff because that's my favorite, okay.
My my favorite capital side is though.
Fab or it's on Fab, yeah.
Okay.
I can't remember.
It's like railroad, fab it's about all the railroad stuff over time, okay?
Pretty much because we're like everyone was first looking for the internet.
We've well massively passed the internet in terms of the absolute size of the build-out.
It's not even close.
Like we're what numbers are are you thinking of?
Like what could I think a trillion was a trillion all in, was essentially the real dollars versus version.
And I think we are well past like we like whatever this year, and it's cumulative, right?
We were well past that.
I think railroads, the reason why it's so interesting is because honestly it's way crazier.
But but probably part of the problem and craziness of it too is like railroad was literally like one of the first added layers of the layer cake, if you think about it.
Before it was agriculture, and railroad was like, okay, well, how do we move this agriculture around faster?
Yeah.
And then banking got I I I kid you not.
Like one of my big takeaways is banking effectively got invented by railroads.
Oh.
Because there's no need to finance it.
Finance it.
Yeah.
So much money was needed that like effectively 85% of all paper whatever was essentially just railroad debt.
Yeah.
One of my favorite anecdotes was before there was a federal bank, uh Federal Reserve, Andrew Carnegie was the Federal Reserve.
Yes.
Yeah, there were individuals.
Yes.
Yeah.
And so all this stuff.
So it's like the whole thing.
I kind of did some work on the Gilded Age, all this stuff, but like my takeaway is like that was a really interesting cycle because it was so big and took so long to deploy.
It actually was 45 years of like there's three cycles actually.
There's three boom busts.
Same.
I don't know if it'll be quite that long.
All the cycles kind of collapse.
Yeah.
Or that we can, you know, information, information.
It moves around faster.
Exactly.
Yeah.
And so you you have all this stuff where I think it's gonna happen faster, but like I would be really shocked if it was all in one go.
That's my vibe.
Yeah.
Where it's like it's all in one instantaneous up down.
I think it's gonna look like multi some multiple cycles.
So yeah.
Kind of just worried about railroads.
The there was like a baby railroad cycle, then there was a huge railroad cycle, the modern world was invented out of it.
That's like my favorite analogy for this because like I think it was like GDP percentage of CapEx each year were like high single digits for sustained for like 10 years.
Yeah.
But what's crazy is like that amount of spend is like we're we're like well on track for that.
It's did you do a percent of GDP?
Because I think that's I think the the way you'll make it convertible.
Stargate itself 2% of US GDP.
And I mean it's going to go up like yeah.
Yeah that's a yeah and it's not all going to be in one year right but it's okay so yeah so so total capex four it was 4.8% of GNP and 25% of total gross fixed capital investment.
Okay.
So 25% of investment every year and four five percent of GNP I think we're there.
You know Stargate plus and Five plus whatever yeah we're we're right there.
X AI.
Yeah so we're at the railroad build up.
But the thing is crazy we should exceed it like probably yeah yeah no no not probably like we should but okay like this is bigger.
Yeah.
Okay.
I I'm I'm like I I would like to say yeah sure.
I yeah we we will do it.
I I'm worrying.
I'm like, dude where we're gonna get all the money.
That's like such a like the pedestrian concern.
Yeah.
It's not a pedestrian.
I mean, this is what happens every capital center.
Like, we must hands in the Middle East, you'll flip the thing.
We must, we must the what what this happens every single time.
That's the reason why the like bubbles happen, right?
It's like we essentially get so big, where it's like this must be built, it doesn't matter the price.
And then all of a sudden we look at it, it's like, ooh, that was a steep ass price.
But I think I mean the thing I think about this is like how I think what the big picture is there's a demand curve and a supply curve, and we have no idea when they cross.
They will cross one day.
And every single year, the demand, then we're finding that demand curve, and then the supply curve, we're just like we're doing our best to deploy it.
And I think for me, like I don't know when that number is.
I'm not, I don't want to say number go up forever because I feel like that's like intellectually dishonest.
But quad code for me is the first time where I'm like, and we're bringing it all back together, where you're like demand go up so much.
I am now guzzling as an individual.
Like, for example, I'm we're we're off, I'm off max, it's not enough, it's not even anywhere near enough.
Like, I mean, some people buy like five maxes, and then yeah, they wrote yeah, so so I I so I'm on fast, I'm on fasted with one million on API, which is that is like an addiction level thing sets.
But yeah, I I I really think it's the first time we're like, okay, well, actually, how much is this worth to me on a yearly basis?
I think it's like 20 to 30,000 dollars easily.
Yeah, like if not more.
Like, I don't understand like what's the like I can't price it, I have no idea the elasticity.
Yeah, you pay for a perfectly compliant junior analyst, yeah, right.
And so what's able to that cost?
Like 90K.
That's able to work in parallel, yeah.
Like you can have a hundred of them, it's kind of crazy.
Yeah, so it's it's a skill issue if if you cannot manage a junior analyst that is 20k a year, yeah, 100%.
Which, like, I mean, okay, like, you know, skill issues, like it's your fault, but no, like we have to learn how to do this.
It's it's yeah, it's like it's at it's tweeted it's three months old.
Exactly.
It is two months that that's the correct way to put it.
It's like it's a it was definitely a skill issue that you didn't know how to get like your settings on your iPhone to work.
We know at one point one point in time, but like in the very first month of us having it, no one's gonna be like, yeah, you idiot, you rube, you don't know how to use your completely new technology that got birthed last month.
Yeah, I think it's just about a time it's a bit of time and it's like kind of interesting because you're like watching.
I mean, it's cool is that if you're like on this absolute bleeding edge, you get to see the design patterns like blossom in real time.
And like we have this like really old older guy who's like been through the history of technology since like forever back then.
He's like one of the most interesting intelligent people of semi-analysis.
And he talks about how who is it?
Tanch.
Oh, yeah, so like you said, this like we we we had the conversation one time, he was talking about like early internet, how like it wasn't actually sure if the browser was gonna win.
It was like a remote web file service.
Some people thought like, well, it's just I'm just gonna reach and play with someone else's web files remotely, right?
Who knows, right?
And that kind of you know, it kind of is a remote web file.
Who who the hell knows?
So they were design pattern searching back then, and I think we're at that again, where all the design patterns are open and it's like really interesting because there's many different ways this could go.
Yeah.
And we're gonna have to kind of collectively agree what's the best set of hygiene, set of design patterns, what's the level of abstraction, and then like all the rest of how much SaaS it will disrupt, all everything else, who the hell knows?
But you get to watch it like front row C right now.
Yeah, yeah.
I mean, m my biggest one, and I I'll I I do want to bring it to semis in in a little bit, but is the IDE.
Two months ago, they we had Steve Yegi from Gastown talk about how 2026 would be the year IDE died.
And I like two weeks ago, three weeks ago, I recently was like, shit, he's absolutely fucking right.
It's over.
Yeah, I'm really wondering too, because like ID, so so I think that same my like my the reason why I'm so excited about this is I get to like look, I never my daily driver was never an IDE, right?
My daily driver was like Bloomberg or Excel or something like that.
But I have a personal belief.
It's not happening yet because we're not quite there in the maturity curve, like software's just gonna be first.
But the year, like of you know, Excel is dead for finance.
Yeah, like it's it's it's A CL is the IDE for analysts.
Excel is the IDE for analysts, Bloomberg is the IDE for analysts.
Like, I believe every one of these IDEs are done.
It's dead over and going and dead.
I just think it's why, why?
It doesn't like just imagine the concept of you.
Like, I remember when I learned Bloomberg, I had to like watch videos to learn about all the random folders and keys, how to use this, how to use this, you know, the tactic knowledge of using this function versus that function.
That's like crazy to think about.
That is like that is like horse and buggy.
Okay.
The agent with the information that can perfectly retrieve and analyze stuff is gonna have the ability to pull that all together in a better UI than it was with no legacy, whatever.
I think all of that is dead.
And like this is why like my my spiciest take of all is like Microsoft is a lot to lose.
I think they have the most to lose of everyone.
Yeah.
Because Excel is a human IDE for information work that's generalizable.
So is PowerPoint, so is email.
Those are the base core level of abstraction that decided to be broadly generable.
But I I just don't think that matters anymore.
I think Claude Code or Cowork or whatever is gonna be the year that like it will destroy all of that.
All that information work that that where you sat every single year, it's it's over.
I think that that's the one that's like more shocking and scary that like people don't believe, like I believe in my stomach with conviction because I have already had that moment for me.
Yeah, I will never make a chart in Excel again.
I actually believe it's hard to let go because I I have so much like ingrained knowledge of of like manipulating things directly in Excel.
Bloomberg, I have so there's no way that you know this, but like my very first started was that an attempted Bloomberg killer.
Uh sent to K office.
I remember, I remember.
Oh, you remember?
Yeah, yeah, no, no, I oh yeah, yeah, you are one of the few.
You had an OD.
I was a sentio customer.
I was a sentio customer that I got rolled in, dude.
I remember how dare they acquire Centio.
I had a patent.
I we followed for a patent for similar tables.
Anyway, while my conclusions was like Bloomberg is just like three things.
It's it's it's Slack, and it's the journalism, which is amazing, and then it's the the data feeds, it's actually not really the UI.
Yeah, using the UI.
Yeah.
But I I think for the first time in my life where I just think that like, I just wonder if that like okay, if you can get uh obviously it's so you're telling me that the future, the undisputable future is just like it's IB and nothing else, and then like a terminal net like types in some stuff.
I think that if you are marginal and on the curious and not hyper interconnected, which I would argue that I am at some analysis, like for example, I'm trying my absolute best to just rip Bloomberg out.
We're going to fax an API.
Like I'll all in API with a cloud code is my belief of the future.
Hey, verifiable data source that you trust.
Yeah, yeah.
Hey, scale.
For you guys, you can do it.
Yeah, for traders, we're so Richard, no way.
I understand.
Like there's an information network that's like outside of this.
You do it deals in IB.
Yeah, which are tracked by the regulators.
100%.
Yeah, 100%.
It's totally I completely agree.
But as an analyst, yes.
For as an analyst, yeah.
And so, like, but I just think that like, okay, that doesn't really so you're right, the core cash flow cow thing will continue onward.
But like each iteration of this AI thing, I was like, Yeah, I'm still gonna be using Bloomberg, right?
This first time it's like, actually, no, I don't care anymore.
The IB is my unals of like marginal value from from IB is like now outweigh by how like clunky this is, and I want to just make some charts.
Right.
And so there's like immediately you save 10 to 20k.
Yeah, for switching down.
Yeah, there you go.
It's amazing.
Yeah, but by the way, what was your cloud code and of your prediction?
25?
Yeah.
I want you to know I sandbagged the ever living shit out of that.
Oh, okay.
I I just believe 25 is very like I like because like a the rate it's on is like whatever, 50 or something like that.
But I think I feel I wanted to give a 95 confidence interval.
I think 25 is within the 95 confidence and interval.
Sure.
So it's so it's between 25 and 50.
Something like that, yeah.
Yeah, it's it's just absurd.
But uh, you know, it's supposed to be codex code.
Are you that's a watching code?
Yeah, yeah, yeah.
So to be clear, I I'm actually even willing to comment on that because like I know we've done a lot of shit of being codex haters.
Yeah, I I think I I'm by the way, when I put COD code, codex, agent, whatever, all in percentage that we can publicly see.
I would argue that for the ratio outside of that's probably higher too, but whatever.
Yeah, I think together.
Yeah, we're watching Codex.
I actually think Codex is Codex is pretty good.
5.3, I think.
So we had the whole thing, I because like I wrote most of the articles, like, oh, token efficiency, the context rod all this is the same one, or yeah, yeah, it's in the bottom.
It's in the it's in the the paid section.
Okay.
Okay, so but like TLDR, I was like, well, you know, the reason why COD code is so good, Enthropic is so good, is because all of this token efficiency, the token efficiency is better than Chat GPT, all this stuff, blah blah blah blah.
And then like 5.3 codex came out, and it's like, yeah, that that completely doesn't matter anymore.
They're like they're they're so back.
I really think 5.3 codex is awesome in in coding, though, but you can watch it.
Like the reason why I like Opus 4.6 so much is because when I'm using it, I'm using it for like coding in is the way I interact with it, but I'm using it for broad generalized information work, right?
But I think the difference is codex wants to code because it's RL to be so good at coding to win on Sweetbench that like you're trying to use it for general information.
Like, hey, I'm trying to, can you go research and search all these websites and like I don't even think they have web search in it or whatever?
Maybe maybe you can give it an API or whatever.
But it's like great.
I'm I'm scrape I'm creating a piece of scraping software to go look at these websites.
I was like, no, no, no, no.
Just like just ingest tokens of what's on the website.
It's like, okay, great.
I'm still like it's so coding pilled on the RL that I think it isn't generalizable in the way that that 4.6 is where it's like, oh, I could have it, I could have it make some rubric or do some research or do something like that versus codex.
It's very, it's very code.
It coding codex is coding pill.
And so that's what but I I am very optimistic actually on codecs, and we do track quite a bit.
I I you can they have a meaningful amount of thing.
Share, you could see the Bloomberry, they have the the sh the chart Bloomberry.com.
So the cloud code definitely is in the lead, but I think the part of it too is like the the like to like comparison.
There's a ratio of codex that's not available because it doesn't sign off every commit.
It does sign off on pull requests.
That that ratio is much closer.
So all the open AI people, like Rune will tell, like, blah blah we're not accounting for it.
Yes, we didn't account for it.
But like I I think Codex is better.
I think there is some real problems and issues, but I bet you the second that they have a new pre-trained with the RL, because the RL stack on Codex 5.3 is amazing, like it's very coding code.
That's when that's when the it flips over.
And yeah, look at the other players here.
It's just like.
I mean, my favorite thing is that how GitHub Copilot is like number one.
And like I've never heard of like, do you know anyone who uses GitHub Copilot?
Yeah, look, look, okay, that that's that's a bubble talking, right?
Okay, that's a bubble.
Yeah, yeah, that's that's the that's the SF bubble.
Like, yeah, yeah, like you know, there's there's like all these Windows users, and you don't talk to them, right?
Like uh you do, but we don't in in San Francisco, and like that's just that's fine, but that that's definitely bubble talking, but yes, Copilot has a billion in ERR, I think at least.
Yeah.
What's crazy is Cloud Code is uh has a ratio, the their attribution of Cloud Code and ARR is 2.5.
Yes.
So that on this, the daily install counts, right?
Is an order which is by the way, just the VS Code extension.
Right.
So there's yeah, I know, I know.
That's not even a default way to use card code.
Yeah, you're right, you're right, you're right.
Uh CLI, MPM, DONLAS is another way to track it.
But I think they have like their own cust their own installer now.
Anyways, all in all, definitely heard, understand it's very hard for us to like actually track it, but like I I'm uh I'm not criticizing, I'm just like I I think Codex, a big thing I'm watching is well, is Codex back because they they reported like Jan to Feb the double users.
Yeah.
Okay.
So so I have some not skepticism just because they have such a big chat GPT portal that could be like try Atlas, like the modal that pops up can really move big users.
Like they're not quite a Google.com in terms of having so much ability to like siphon off users off.
But I I wonder like the like to like, butt but that's like my skepticism.
I have an answer for that.
Uh Alexander Umbericos was just on the Lenny pod saying that they actually haven't invested enough in the web experience.
So like I I I think I think the attribution for that is zero.
Okay, yeah.
I guess I just saw a modal be like, oh, try co try code.
I mean, but the but a modal is a minimum.
And then to be clear, codex in the in Mac is great.
I'm actually, I mean, like Yeah, yeah, it's the the they attributed the app.
They the app launch.
The the app launch is actually pretty good.
So so yeah, and and I think I'm pretty bullish that honestly, especially for coding, because it's like very coding code.
I just can't get it to work as well for non-coding stuff.
Then my, you know, I you use conductor?
No, I've not used conductor.
Oh, okay.
I I thought I heard you say on a podcast.
No, I've not used conductor.
So basically, like the the their argument for any la any first party app is that they're only going to prefer their own first party pieces.
Yes, 100%.
Which like they're ordered at play.
They're already doing it.
Like, like I feel like this is how they're gonna differentiate, right?
Like they're going to well then you have a conductor where you can use codecs and cloud code for different tasks as you see fit.
And so this is the the clean superset.
No?
In theory, yeah.
But but but I mean, this is like okay, so so then you can argue this is the clean superset.
It feels kind of like I guess my design pattern on that is really skeptical of building on top of something that is growing very quickly and has all the money and whatever.
Like I just think my favorite one is like platform as a service.
If you remember that one is like infrastructure as a service, platform as a service, SAS software as a service, and like, oh, this platform is a service.
And it's like it always just ends up being in the middle, so it just gets it even by one or the other.
I I think of that like middleware layer, unless if it's a really, really, really compelling case, often dies.
But that being said, in this moment, I agree.
I actually you I like to have them like review each other, like having them yell at each other is really great.
I might actually try this soon.
I haven't used I haven't used conductor personally.
Yeah.
I'm mostly just been, you know, going deeper into the psychosis.
Yeah, and this is as a former cloud analyst, very typical of like, do you want a multi-cloud or do you want to go all in one cloud?
And the the the classic argument for multi-cloud is well, then you can use the best of your exactly.
But if you go all in one cloud, you can exploit the sort of minor features of everything.
And you know, the it makes a market and you there's no right answer for everyone.
Exactly.
Yeah.
Yeah, I mean it's yeah.
That's what that's one of those things where I like even the really small percentages in AI still really matter because they're the huge and like yes or very happy, very productive, pay money.
Okay, it's good to be an analyst in the space because it's fun to keep up with it, right?
Like, I agree.
Like, I I think everything, everything we like the horse race.
Yeah, I like the horse.
Number one, number two.
Ooh, yeah, yeah.
But no, yeah, no, no, I know, but then you have to your brain also has to be like number two is really big too.
And then I I just think like for me, someone who likes the history of all this, like like likes history of innovation and competition and disruption and stuff, likes new technology.
It's like a very fun time to be following the stuff all together.
Tech during the like 2017 and 2020 years, so boring.
Yeah.
Yeah, at least for me, anyway.
I thought it was pretty boring too.
Yeah, yeah.
Sorry.
I was I interrupted you in mid time.
No, no, I I I remember what I was talking about, Roy.
Okay, it's just fun time.
It's a fun time to be.
It's okay.
Things are happening.
Okay, I wanted to transition to a little bit of a spicy thing where you were on TBPN, and their title that they chose for you was Douglas thinks Microsoft is out of AI.
And ooh.
Did you not see those?
Okay, so so so I wouldn't say out of AI.
No, I did s okay, so I didn't watch it.
I never rewatched these things.
Okay, so how I think about it is But like you s you said things like Microsoft is scaling back investment and so so it was the previous conversation I was talking about.
Yes.
How Microsoft has the most to lose.
They have the most to lose of everyone in the entire world.
If you're they're the they're they're they're the software company.
The horizontal software company's dead, but yeah, exactly.
They're the horizontal software company that humans use their software to do information work.
Okay.
No, like I cannot paint a bigger target, okay?
I cannot paint a bigger target.
And Source Wars.
Yeah.
Oh well so okay.
That's another two numbers.
Microsoft is uh automatically bigger source.
Yeah.
But the other thing too is they have this Azure business.
I don't think it's completely out of the race.
I'm like, you know, it's a really great clickbait title, but the the problem is the Azure business with OpenAI, right?
You're essentially renting barbarians at the gate.
You're you're like, you know, this is r ancient Rome, and you're like, hey, we need some extra guys, so we're gonna we're gonna pay money for these barbarians to build a big thing.
The golden army, exactly, through Game of Thrones.
Yeah, yeah.
The golden army, and the problem is like each year they become more powerful, and then and then at some point they're just like, you know, we could just like scale these these shitty walls.
So like the m the so that's the problem is the moat, the the wall and the moats every year are getting more dilapidated as they continue to rent GPUs to to the barbarians.
So it's just like Google Yahoo again.
Like yeah, yeah.
It it's exactly like that.
And so it's just like this weird process where that's a terrible setup too.
Because what happens in the history of that is you have to choose one or another.
Okay.
If you do either poorly, you're you're like you're like somehow in a third worst place.
You either all in become Azure, like maybe in the telecom era, right?
Because you're Team T guy, you become dumb pipes.
Okay.
That is the the Azure becomes co- what is it?
Charter, right?
Uh-huh.
Yeah.
But then oh, the other version of this is you say, no, no, screw these guys.
I have to like reinvest back in and like essentially steal copy their their features and build up my moat.
That means I need to stop investing in Azure for the stock.
That really sucks because the stock is very much weighed on out your Azure revenue.
And meanwhile, if we actually had to value Microsoft X Azure, the multiple would be really low right now.
I think about that all the time.
What would this trade X X Azure?
This is just Microsoft.
Yeah.
Eight times runnings, 10 times earnings.
Like it was trading like that before, actually.
Oh geez.
And like remember the 2010s era when it went all the way but down to like 10 times earnings in the Steve Ballmer era.
And then it inflected out where it as it did Oak with Azure.
Yeah, yeah.
Azure and O 365.
Yeah, there we go.
That's right.
Yeah.
Yeah.
Okay.
I don't think you have the answer, but I just like this is far the most bizarre.
I want to call it f but I don't know if it's a f or not, even.
Yeah.
Because it's a clear decision where they were the lead investors in OpenAI.
They had the deal and they consciously obviously stepped back.
They're still good partners, but like what happened?
Like so.
I I think the biggest blunder of all time, that the part that like is kind of crazy to me about that one is like, yeah, I I I definitely think there was a financial decision because when you look at it, it looks like a conversation of shareholders, ROIC, and how much are you willing to burn cash?
Because, like, you know, effectively you look at all the other peers, and Google, I would argue is going to free cash with zero.
I think Meta will go to free cash with zero.
Microsoft is still like, you know, Satya did not make the company.
He is a professional manager, and there is a board and there's a conversation.
Yeah, yeah, he's being responsible, right?
But the problem is that responsible of this is like an innovator's dilemma, right?
Like, do I maintain maximize shareholder value and cash flow today, or do I have a deep belief that AI will kill the hell out of my core business, and I need to all in investment you know, am I ready to bet the entire company on on a trend?
And it seems like Satya is not a believer.
You know, we've been talking about AGI, he is not ASI pill, okay?
He doesn't have any fear of the show death.
He thinks it's just like a new cool, it's a new loaded board, it's a you know Lotus and Excel came around, right?
Like it's just a new tool.
But I think at the same time, this this conflict between renting GPUs to the barbarians who will disrupt your business, or you know, your actual core business.
It's clear how they're feeling.
In the call of earnings.
They talked about they could grow a lot faster if they wanted to, but they're trying to reinvest back into the internal capabilities.
That to me sounds like we are not gonna hire as many barbarians.
We're gonna pull, you're gonna we're gonna reinvest in these walls, pull in together and try to defend the core mode, right?
Because the the dream of this, and in theory, you're like, oh, remember in twenty three when they did the first big deal, you're like, wow, Microsoft's gonna win it all.
Because they already have all the distribution and they're gonna have the perfect product and boom, they're gonna have this giant business that makes them, you know, whatever, a hundred billion, hundred trillion dollars.
Okay, whatever number you wanna say.
But reality is Claude for Excel, Claude for PowerPoint is literally exactly what it's supposed to be.
Microsoft should have built it.
Might Microsoft should have built it.
Yeah.
And so now you see the barbarians, and this isn't even your primary barbarian issue.
The guy who, you know, this is like this this is the the this is like the tribe over the hill.
Yeah, yeah.
Barbarian.
Yeah, this is the tribe over the hill, you know?
And though the tribe over the hill is like, like, you know, on an on a nightly raid, easily sack the hell out of your castle.
And you're like, dang, this is an issue.
So so Microsoft now is super stuck in the middle, and so how they're gonna have to do this is totally different.
I think they're gonna keep I think they're gonna keep pulling back in.
We're starting to see that, like they're gonna do internal training, they're gonna try to do more foundational models, they're gonna try to use the weights that they have access to.
This MEI, yeah, okay.
Yeah.
But I'm very skeptical because their execution has been kind of dismal.
Well, you know what maybe seen they they they're they do have inf you know, they are one of the big biggest companies companies in the world with all these resources.
Yeah.
I I I always wanna push back on the the sort of responsibility part.
Like, you know, so Oracle picked up the stack.
Yeah.
Is Oracle being irresponsible?
You know, I I'm actually if we're gonna talk about Oracle, I think so.
Let's talk specifically about Oracle, because this is where we're gonna go.
I think Oracle was irresponsible because of the magnitude of what they did.
Okay.
Like the thing is, like, I think the Slack they should have done it, but like the whole setup, in my opinion on Oracle is own goal.
They messed up the messaging, they messed up the fundraising.
And in my opinion, if they were not like one of the things that happened is they went so aggressive out the gate, did the quarter where they said like 400 billion dollars, right?
They they said R and prize the world, they promised the world.
Then they per proceeded to raise as much money as possible.
And like this is the first time they've ever done these giant build-outs.
And so now there's delays.
Everyone's like, whoa whoa, you did this much, right?
Capitalism is kind of like, hey, hey, pump the brakes.
And and seriously, I think that if they just tiered it out better, meaning that they didn't do it all in one period, played a little bit of expectations management.
This year's revenue from the deployed GPUs should partially help start to keep self-funding, and that's how you make this work in a glide path without going up, down, up, down, a big bang.
And so they I think what really happened is the big bang that really screwed them up was the debt side.
They they just offered so much debt.
It's kind of funny because in high yield TMT, it's such a big part of the entire index.
Like the issuance is so big, it's like debt index.
Yes, of the I I have zero familiarity of this stuff.
Hey, I'm I'm pulling some numbers up.
I did the numbers forever ago.
I'm like, I I co-sayed and whatever, forget all the precision.
Let's just say all of the investment grade at TMT is like 500 billion.
Okay.
I think Oracle is like 135 of it.
So that's like that's so big.
And so each time you have to you put up a huge new issuance, you have to give someone an incentive to go buy your debt instead of someone else's.
And so you just kind of like they're screwing up the liquidity because these issuance are so big, diluting the whole pie.
It makes all the terms a little better or more favorable for investors.
So it literally the entire index is selling off because it's like supply.
Yeah, it's a supply thing, right?
And that's the thing that's like crazy to me.
Is like so they they've massively overshot.
And I think that we're like a weird bottleneck.
I never ever, ever, ever, ever thought us thought would ever ever hit.
And I think you could appreciate this uniquely, is like one of the bottlenecks is like supply of debt into the market.
Like like capitalism cannot like absorb that much capital demand because the order of magnitude, it's totally different.
These hyperscaler businesses have been completely self-funded since the history of time, had never gone out and issued anything.
First time they want it, they turn around and they're like, hey, instead of like, can you give me a uh 10 billion dollar loan limit?
Like, we've never done that before, right?
So the absolute size is kind of screwing it up.
And I think that Oracle specifically was way, way, way too aggressive into a relatively illiquid market.
And so, like, you have to do this, like you have to kind of leg yourself into it if it's gonna be like that.
But they they would like super jolty did these big, huge incremental ads and kind of flipped the whole thing.
Oracle CDS, people all freaking out.
I think a lot of it's mechanical, specifically on how badly it was done from a supply-demand perspective.
And I think they can pay for it.
You want to what I'm hearing about all right?
Microsoft could have just internally funded this and like Microsoft could have internally funded this, it would have been totally fine.
100% agree.
And like this example where it's like, yeah, I think that that's a blunder.
That's a perfect example of blunder because Microsoft's cost of debt is the same as the United States government.
It's like the cheapest you'll get anywhere.
And like just from the like a like a P time, like the math perspective, no one else they they're better than Oracle.
They they just by their credit rating, they have a two percent more profitability at a capital basis.
That's that's like you can't beat that.
I don't know why they decided not to, but now they're in this weird thing where they're like they're they're they're kind of wavering.
Like, like to win, you have to be like really bold, right?
And they're kind of like doing this one thing over here, being really defensive with Copilot.
Satya is now you know the the product manager of Copilot, and then they're also pulling back from Azure.
Meanwhile, the competitors are pull are pushing in for the supply.
It's a really weird game.
I think Microsoft has to choose choose a direction.
We'll see, we'll see.
We'll see.
When it's okay, that's what's gonna make it fun.
I'm more than happy to change all of my opinions when new information comes around.
Yeah, and I'm sure it will it will it will have more information that emerges.
I wanted to touch on TPUs and then go into memory.
TPUs will hopefully I don't know, maybe maybe a short one, but like you know, for a long time you could not buy TPUs, at least like current gen TPUs externally, and now you can.
And Google's open as a as a as a supplier, I guess.
I think Sergei doesn't want to lose.
And I think the thing that happened was up until like you know, he wasn't no one was there.
A part of the whole deep mind story was we will hoard all the TPUs because we were first, you know, and and so like why so?
You know, why why why give anything to the topic?
I think it's because at least last year it became pre-gemini 3 it was like dude we have all these TPUs we're gonna hoard them all but like people aren't using our product anyways.
And and like hey what's all what what good is all these TPUs if we're getting our asses kicked in consumer.
I think it's an interesting thing too because the other thing I think about is there's a lot of different ways to to like break this down.
One we wrote about it in TP V8 like whatever we think Ruben will be much more competitive.
I think Ironwood V7 is the peak gap between on between TCO between Nvidia and TPU right.
So if you are at your absolute strongest point what do you do there's two ways you could do it you could try to maximize and like squeeze the juice and like make margins or you can you can gain market share I think the perspective of doing this externally with with anthropic is to gain market share because the biggest gap you have and and one of the reasons why there hasn't been a second merchant chip and also you can argue NVIDIA's most valuable company in the world what's the value of TPU in Google?
It's it's huge.
You must have done the math.
I've done the math.
It could be like it's like a trillion.
It's like a trillion.
Yeah.
It's like a trillion or something like that.
Assuming it gets like 30% market share or something like that.
Everyone has been trying to crack the merchant silicon mode, right?
And now they have the biggest absolute outperformance.
All a lot of the people who did the original TPU program are like now at OpenAI.
Some of them are Medics.
Some of them are, yeah, you're exactly some of them are they're all over, right?
Like the core team that did most of the engineering have like sense really dispersed.
And so I think the gap might might close over time.
And so at this absolute period of time, they're gonna they're gonna win the market share.
And then what happens is if you have an install base, you have an incentive to upgrade your install base.
That's like the hugest problem with AMD, for example.
No one wants to buy new AMD chips because it's not like they have old AMD chips.
No, they're not upgrading from anything.
And so when you have that number two place, you have to like you have to win definitively, and then also you have an opportunity to win win again next year.
I think the install base issue has been a kind of huge one.
And so TPU is at the point where the software ecosystem is mature enough.
The hardware is definitely mature, the networking's really mature.
You have a really good external customer who actually knows how to use your product.
If you want market share, now's the time.
Yeah, that would be insane if they if they actually sort of pump the pump the gas on that stuff.
Are you also hearing I I don't know if this is something that affects your analysis at all because I don't have any appreciation for the or the sizes that we're talking about here, that Jax is helping TPUs win.
Or or Jax is winning relatively the PyTorch, at least in like the academic arena, which is a leading indicator of what it's gonna be using in I do not have as I I don't have a special purview on that.
The thing I'm most excited about and like very much TBD we'll see is inference X will have TPUs eventually.
That's something we want to do longer term.
I think that that will really show in numbers.
What's in the benchmark?
Yeah.
Or how do you expect them to come in?
Pretty good on a price basis.
I mean, our expectation is like they're they're the best TCO by a meaningful amount right now.
Enthropic's very clear how they feel.
Like everyone is very clear.
I think even OpenAI would take.
I think everyone would eat as much TPU V7 as possible if you had it in a perfectly unconstrained world.
It would probably be at this exact moment, like you know, the the hottest kid on the block until Ruben comes out.
But the reality is supply chain really matters, and they're that just that's not available.
And so that TCO, that TCO advantage is at this absolute biggest aperture.
Then like Jensen essentially gets its it gets their stuff together, it's competitive and boom, it closes.
So this door only open right now.
Probably TSMC is the biggest blocker.
So there's no yeah, yeah, what what can you do?
It's this cascade, right?
They wish I think you've talked about yeah, like all the way it goes all the way back to the fabs.
Yeah.
Yeah, well, it's interesting because it's like even more than the fabs, like like on the optical.
Is there a link I should be pulling up?
Yeah, that that that's it.
That's it.
T E V7 and Google swinging.
Yeah, so so yeah, it all goes back to the fabs, it all goes to who's making the chips, and like I think one of the big differences too is just like the per it's just like a really good, cleverly designed system architecture, and it's relatively stable, and it's clear that they you can pre-train big models on it, which is like a huge, huge swipe at open air right now.
That being said, like I think OpenAI will get their their act together very quickly.
And so, yeah, that's kind of like the narrative.
I think it's gonna be a good story for probably like a year or two, but then the real question is the V8, we just don't think will be as competitive to Ruben, and that's when your special window starts to close.
What's the technical reason why?
HPM, HPM4 versus three.
And that's a secure is the strategic decision by NVIDIA?
Yeah, I think one so NVIDIA is always, if you think about NVIDIA, they're always trying to gas it as hard as they can.
Like they they're like it is a high performance chip.
It is a it is an F1, like it is as maxed out as possible.
TPU is kind of like this like replicatable pod in a very large with like very high stability, right?
Which if you know the history of Google, that's what that's what they do with it.
That's with Invra.
Yeah, that's what they do with infra, yeah.
But I think GB200 would have completely mogged, you know, V7 if it came out on time and stable.
It came out a little delayed and it wasn't stable.
And so I think there's a lot of different ways to kind of course correct that.
And the one thing that's important is like I think on the infrastructure side, or sorry, on the supply chain side, Barna and Nvidia is the best.
They own the entire supply chain.
They really do.
Like, you think all those HBM price increases, they're gonna come for TPU just like NVIDIA, but NVIDIA was literally in Asia.
You saw him drinking with everyone with the SK, with the everyone with all the Korean guys, all the TSMC, he's doing the shots with everyone.
Why do you think he's doing love shots with everyone?
Okay, it's because he he needs to get the chips, okay?
So yeah, this is Samsung's chairman.
Yeah, this is Samsung's chairman.
Yeah.
I know it was that it.
It was the other guy for it.
But let's put it this way that's that's a huge deal.
That's a huge, huge, huge deal.
Do you think do you think Google was out?
Yeah, he yeah.
Do you think Google was what what you know?
Do you think Sergey was out in Taiwan drinking to get supply?
No, 100%.
There's an opportunity here, but there's only so many TPUs that can be made because the because of all the bottlenecks, right?
And so NVIDIA has all the supply chain locked up, and so they're gonna have like so much of that kind of constrained there.
And so it's it's gonna be really interesting.
They're gonna they're gonna get the best, most performing HPM.
They're gonna be first on the roadmaps for even more rack density.
They're gonna have like the best connectors, the best, you know, the whole system will be once again turbo jammed again for as hard as it can be.
And the people who made V7, like they made the chip was done like three, four years ago.
Like the talent dispersion aspect where people who worked really hard on this team to make this great chip has really kind of gone all over.
That starts to get worse.
And so if that gets better, which takes some time, I I think our current read is that like the HBM specifically in the memory scale up is gonna really go in Rubin's favor.
And so that's the big difference.
And I think, as you know, that's what makes the context windows, that's when able to do bigger, bigger everything, everything, yeah.
And so they're gonna really jam it, and that's that's gonna be a huge advantage in performance.
One thing I love about your analysis is it's not actually just the context windows, it's not just the KV cache.
We also have to offload it to non-HBM.
Yeah, yeah.
Every other part of the memory.
It the the it's it's like such such an interesting cascade uh waterfall of like just like a uh short squeeze and everything.
It's not a short squeeze.
It's like a supply squeeze.
Yeah, it's a supply.
I mean, there's like I just wanna like I mean if I read the one ratio of like yeah, yeah.
Okay, so it's a three to one if I took a three to one to four to one ratio.
I think I think next year so so it's in the memory mania post that we just put out of like the four to one or the trade-off ratio.
Yeah, scroll down somewhere and you'll see.
Yeah, so so basically for for listeners, is the idea that like when you convert to HBM because there's a huge amount of HBM, it takes three times one HBM sort of units is like three times of the other sort of DDR or whatever, right?
Yeah, so some amount will always be lost in production because yield isn't perfect, and so effectively you're trading some like you're trading.
I I may I actually wrote a really funny piece, like I called it like super oil, but we let's say this is a better one.
But pretty much like you in order for this higher grade of jet fuel has been invented, and the only way to make it is to like actually get rid of all all your other fuel, and you have to like massively condense it and refine it.
Okay.
So now what happens is if there's any demand here, it's an instant shortage.
And so we we hilariously enough came out of like the biggest shortage ever in NAND and DRAM, like terrible, like catastrophic, the worst one ever.
Like the the last analysis I could put to is like 96 or something like that.
Seriously, it's like an a history one.
And then meanwhile, we have all this new demand, HBM specifically, the highest end, you need the most memory.
The the trade ratio is crazy.
So each, you know, each bit of of HBM is essentially a four X multiplier onto DRAM.
And then now, so we we've completely constrained, took all the DRAM capacity.
We just came out of this shortage, so no one invested in any clean rooms or capital equipment or anything like that.
People got like massively free cash flow negative, but no one's spending a cent.
Okay.
People could go bankrupt, you know?
Yeah.
So you they haven't invested in these three-year-long lead time items.
And then now there's the like more demand than God, and it also evaporates the middle layer because the KB cash offload, and then boom, you're just looking at the supply-demand.
You're like, yeah, this is not gonna catch up for like two years.
I think the thing that's like so interesting is the supply chain squeeze because these clean rooms take two years to make, man.
And effectively everyone paused, and how bad the last cycle was really forced everyone to completely pause altogether in terms of adding any new capacity.
Yeah.
And so now we're a few years later, and all the supply is gone.
So I mean, people are, I mean, it's just it's crazy.
We I our post, our conclusion is like we we could see DRAM prices like go up 100% again.
Like it's it's gonna be the point where, and this is like also example, like really interesting in the whole thing.
Another hundred percent, I think is demand destruction.
I think you will start to have demand destruction from what does that look like?
Where hyperscalers maybe purchase less or something like that on the margin, on the margin, right?
Because they're like, okay, well, what if I just really focus on this energy aspect instead?
And also, ironically, all the energy, so like every not every data center in America, but like many, many, many most of the data centers in America are delayed.
So you had this thing is supposed to come on in 12 months, it's coming on in 18.
Maybe what you can do is you can play trick in with memory prices and you can kind of push out.
Of course, everything you have in the pipeline, you you pull forward as hard as you can, okay?
You pull forward, you double triple order, and then the DRAM and HVM guys are like, oh my god, how look at all this demand.
And then at some point in time, what happens is you say, Well, we pulled this all forward, you know, we have the power's gonna constrain us anyways.
We're gonna like kind of chill out the orders.
And historically that's when the memory market that that's what causes the crisis, the the prices to drop.
Realistically, just looking at at the aggregate demand of how much we've purchased in term terms of power, it just seems like the gap is just huge, it's completely off to the point where the most obvious logical leg of the AI the AI trade is effectively investing in memory capacity.
Yeah, well not Nazis.
Oh yeah, it's you could say SK Heinnix and Samsung, Micron, and all the all the semi like SemiCap has been ripping.
Which is like, by the way, when I was in Ballyasney, we a majority of m a lot of money we made with just being long micron.
Yeah, in the last second.
Yeah, it's a good example.
Yeah, so like you have all the semicap stuff, right?
Like all the everything that is even remotely related to investing in capacity for memory, that is like the ultimate bottleneck right now.
And also for listeners, it's gonna affect like your phones.
Yeah.
Like Yeah, I Apple, I think Apple's moving.
Oh yeah, I used to buy an SD card for this thing.
Yeah.
It was a bucks.
Yeah.
That's that's nothing too.
That's and that because like that that's just the NAN side.
Dude, have you looked up like I want to say like 64 gigabytes of of DRAM.
Like I'm moving up, like I I need to refresh my iPhone.
I'm moving it up because I'm doing this research for like oh yeah, you need to do as soon as buy your iPhone now.
Yeah, yeah, you buy your iPhone now because what's gonna happen is when iPhones go into the spot market, it prices are gonna go up 100% on them.
That's insane.
And so they have to pass we're gonna be buying like old iPhones and then taking them out for them there's no that's actually there's a there's a whole super duper deep in the weeds there's this whole like technology that was very focused on cloud era called CXL which is memory expanders for CPUs in order to have like whatever just like elastic pools of compute of CPU and DRAM and whatever memory attached and whatever.
It never really took off because essentially HBM was like the way that really crushed it all high performance best of brief wins but this CXL technology that kind of never really took off is gonna take off just because what they're gonna do is they're gonna take DDR4.
They're gonna take the oldest every bit of spare memory they can find and they're gonna put them into racks and then they're gonna attach them via CXL.
So like this oh yeah exactly that yeah it's exactly that but the thing that's so crazy is like this dead technology is like having a shot on goal because of how bad the storage or how bad the memory constraint is yeah like yeah, that that you know I I was like a CXL bowl for once upon a time and then became very clear it was gonna die and now it's like it's back, but only because the entire express intent is to have these DD, like old chips, pull the old chips, attach it to something new.
That's what it's gonna be like.
The the memory shortage is just like it's crazy.
So yeah it's incredible.
Yeah.
So obviously this is lower level than I usually go to that which is which is why I'm having so much fun.
One thing I I do tell people about is like, well, you know, everyone, including Sam, by the way, is like predicting longer context windows.
We've been kind of effectively stuck at a million for two years now.
I've actually been thinking about that a lot.
And like this is not gonna go to a hundred million context windows.
It's not gonna go to trillion.
Like we're this is it.
Yeah, this is it for like five years, ten years, pretty much.
Okay, so the question is will yeah, I mean, yeah, probably actually.
Will capitalism work?
Will we will there be a way for supply to show up?
Probably, but on top of that, I wonder if there's gonna be like like I okay, his his history of compute, what happens is you have to like you have to like make a curve of the sub of the context windows.
Like, does free context windows go to like 1000?
Hey, you can use chat GPT free now, but you your context window is like a thousand tokens or something like that.
And then you just like somehow do a tiny like a tiny parcel for that, just so that you can then charge like you know a hundred X more for one million.
The one million context window is like a mansion, you know?
That's the real thing.
You would live in a mansion.
I live in a mansion right now, yeah.
Oh my god.
The word just context rationing just came to me.
I'm like, fuck, like we're gonna have like vouchers for like, okay, you can have this amount of context today.
Like it's like yeah, you're gonna have to you have to learn how to use it well because of the DRAM, yeah.
So I actually have a question.
I know because like okay, long context to me makes a lot of sense, right?
Hey, that's like like that's like the memory scale up version, like if you're thinking about chips, but in the AI world.
I just am like always been curious because it does feel like, at least in my stated experience, really long context, like you see in the papers, they kind of like drop off.
They like actually don't use all the context.
So that's like kind of the thing I've been most interested in is like does the hundred million context actually matter if if we if it's not possible to use it all.
They you know versions of the hundred million do exist today, they're they just suck in various ways.
They they're not actually applying full attention, right?
Yeah, you can you can use a state space models or even like a LSTM to like you know to to process 100 million tokens, but you're you're not paying full attention to those hundred million tokens.
And so I think like the way that we have context model today and those curves, they will improve over time, and they have they have been improving a lot, but we're we're just not we're never gonna use all of them, but we'll we'll improve like on the algorithm side.
Yeah, I think for me, what matters is you are you represent the physical constraints that us, the software side can never surmount because it's a physical constraint.
And well, I mean it just it just like physically we cannot double we can't even we can't even double in the much more says 10x.
Yeah, yeah, like what what's the point of talking about any of the things?
Yeah, what's what's the point?
Yeah, I was gonna say like we could we couldn't we can invent a lot of things.
Context rationing is pretty good.
I really like that one.
Context frugality or like budget or something.
I feel like everyone's gonna be like, whoa, you're running out of context with window today, you know.
Like maybe that's what happens next year.
We're we're we're we're charged on context window.
And then the the one of the more recent up sessions is recursive language models, which again is just reusing the same context window on it on an artist.
Yeah, I've been pretty interested in that, but like to be clear, I'm a total idiot.
I have no idea if Claude tells me what's you're this you're the semis guy, man.
Like you you're really good on your stuff.
One thing I wanted to spot check on was TALUS.
Uh I have not messed around with it.
Okay.
You don't have to mess around with it.
It's just the this general theory of custom basics burning the weights into the chip.
Yep.
So you don't need memory.
Yeah.
That's pretty good actually.
I think I think I think that that that makes sense to me.
This Kim just comes at the perfect time.
It does, but I guess okay so historically the question is how big does it scale, right?
But like I mean you know a lot of the models are kind of actually smaller than you think, right?
So like that's like sorry what do you mean?
A lot of the product the the production Yeah they they get to still to shit.
Yeah they get to still the shit.
So it's like the push and pull there is gonna be like okay can you just burn in a a s like enough efficient Pareto frontier in terms of performance to be burned in straight onto the silicon that doesn't memory and then boom you can scale this forever versus like you know the performance edge of the long thing.
It's pretty clear to me that like TALUS has a place because you're kind of seeing this market bifurcate a little bit.
You could argue the pre-fill decode disaggregation stuff like that is like the focus on performance inference serving is gonna be a subset of the market and then the training and the whatever and the big production like the you're we need to kind of break it in into smaller parts in order to be using the same grips.
Yeah it makes no sense.
Just in order for the compute to be even remotely okay.
It makes sense to you and I mean T BD on a sort of practical implementation, but otherwise burning the weights into the chip.
Why didn't etched or some of the other guys get there first?
Um it's just pretty interesting.
I don't know.
Yeah I'm I mean to speculate yeah I mean I'm not gonna like super speculate.
I mean the thing the thing is like their thing is like okay how do we have a a big systolic array right but like they they didn't burn weights into the chip that's a little different right?
I mean like look like I mean yeah I just think the way to speed things up is to never transfer anything.
Yeah.
Yeah that's that's the fastest way possible.
And so that's so but the thing is the bet on on on this really large systolic array is effectively everything is compute bound, right?
But like I don't think that that's really the case in in terms of like where we're actually seeing issues in production markets today.
It's like you're actually seeing all the issues in the memory right and so like I d I just don't know if that's like gonna be the perfect solution.
There is definitely a world and space and like a design space where they're gonna be very valuable and cool but like also the reason why my hit rate for every AI accelerator trip is like very like I just don't believe in them is because like where are they?
Until Cerebrus and Grok honestly they were all considered failures.
And even then we're like what are they gonna do with Grok?
What are they gonna use Truebris?
So is is Summonova out No I think Solanova was like a much more interesting one but I think there's like there's like all kinds of deal issues with that.
I haven't been keeping up with that one this much.
Yeah I I always I always try to mention them is as part of that cohort.
Yeah.
Yeah, because I kind of forget about them too.
But yeah, I honestly I was gonna say they were you know once once a year they show up.
Yeah, they they do, and they're not they're not so bad.
Yeah, yeah, yeah.
Yeah.
You mentioned actually some CPU shortage stuff or CPU outside of open.
What's what's going on there and I think it's I think it's okay.
So okay, I have one.
We'll start with the conspiracy theory that I think is really funny.
But have you been noticing just like I feel like web services have become really unstable.
It has been down a lot.
I I like and like me, okay.
This is pure, like you know, a schizophrenic hat brain, because I have a schizophrenic trend hat brain.
I'm wondering if it's two things shipping vibe code slot to prod.
That's number one.
That's definitely something if that's possible.
But it's happening to all the clouds at once.
I feel like it's not just uh AWS thing, it's not just a like a GitHub Azure thing.
We are kind of right at the exact five to six year period of the refresh cycle of COVID.
So COVID, we had this big 2020, 2021.
You bought like hundred billion dollars of CPUs and stuff like that, and so we're right at the natural end of life for these chips.
And so usually what you do is you have this big refresh of all these chips.
But what what's been happening instead is everyone has essentially scrounged all of their budget as hard as they can, but then like I feel like I've seen it in like Azure, like hey, last night my Amazon Prime thing doesn't work, and I was like, it'll probably work in them in the morning, babe.
Don't worry about it.
I think it's I think Azure's just are like AWS's piss tonight, you know, something like that.
But I think so.
We have this five-year thing, everyone scrounged every single dollar they could to essentially invest in as much as AIs and possible and just do maintenance gapics on CPU.
Ironically, at the same time, for all this cloud code stuff, is actually if you have this coding agent just generate you god knows how much computer, how much like software, where is the software gonna run on CPUs?
So I think we're gonna see some increasing utilization, as well as the fact that RL is like actually heavily used for like RL gyms.
You have to you have to simulate software and it uses a lot of uh CPUs.
So the or not quite like the orders of magnitude of the GPU stuff, but it's just such a big trend even when it steps slightly in a in a place mass amounts of demand.
I feel like we might actually be seeing a CPU shortage.
Partially because this refresh cycle but partially also because like I legitimately believe the cloud code cloud code is increasing software creation and then on top of that there is real DIN from RL.
Yeah yeah and just general production agents as well you know we just got yeah every like RLMs take compute and you know open cloud takes more compute and it's just it's just different slope but in the same sort of direction.
Cause everyone like how do it the same problem that happened massive underinvestment because they're like screw it we're doing maintenance only we're just all we're gonna do is mainten maintain the past we're not gonna have anything else and then all of a sudden just a little tiny slope on top of it you're like boom shortage.
Yeah amazing semi skies say semi numbers go up that's that's one way to put it yeah I I the thing that's crazy is we talked about the demand but it's like yeah it's like you're right like I mean, it's for sure.
Like you like, show me where I'm wrong with you where I'm I mean, I definitely not.
But the thing that's crazy is like memory prices are going to go up so much that we're gonna have to choose which which we go on.
That's the crazy part to me.
Historically, memory has never been a constraint like this where it said, actually, you're not gonna get your low end, you're not gonna get your low end phone, you're not gonna get a GPU this year for gaming, none of that stuff.
You're you can't do these things because your price out of the market.
That's what's crazy.
They didn't say that is the first thing that's happened in a long time.
That's gonna be really interesting to see where that shortage and how at how it's like digested and felt.
That's amazing.
That thank you for that breakdown.
I feel like I I really understood it like talking to you.
Yeah, it's really transition to a couple of personal things and the end.
How do you write?
Because you you write a fuck ton.
Yeah, I do.
I have been writing a little bit less these days now that I'm like in the semi-analysis mega mind.
I definitely write a lot.
And like you kept going with fab.
Okay, dude, to clear that was really so so look, I'm still trying to do fab because I I I I do feel deeply connected to writing.
Let's just specifically talk uh on this a little bit.
Yeah, just just like explain yourself.
You know, okay.
So the thing before L LMs came around, the thing I felt the strongest about my my number one information skill is I was able to read and synthesize and process at like really high speed, really high throughput, decently high comprehension, the adjustment is speed in terms of comprehension, almost anything.
Like when my like when my friend gets a PhD, I go read their paper.
I was like, Oh, I have a pretty good idea idea of what you're doing.
I was like, like, hey, when I was interested in semi-cupter book, I literally raw dogs some textbooks.
Whatever the comprehension was not very high, but like, hey, whose comprehension is?
You know, you know.
But I was able just to like push through these books and learn.
So I've always loved reading.
That's like my my number one original competitive skill set differentiator, and also something I like loved as a kid.
Crazy reader when I was a kid, always have been.
And then starting the Subsack, which has been really fun, actually, because I just really wanted to get my story out, like the things I cared about, close the loop for writing for me, because I love I love reading so much.
It makes a lot of sense that I love writing.
I think what really helped is I wrote every single week for like since October 21, like consecutive streak for a long time.
The streak has been a little broken as of late.
Semi-analysis plus fabricated knowledge is pretty hard to do.
Like all of 24, I think, like we're just talking just like every single day, every single week I will put something out, right?
Is it like a hard rule like one a week?
It was a hard rule one a week.
Uh at least an attempt to.
And so I think one of the best ways, all the people who write about who write about writing all say the same thing.
You need to just be writing more.
Yeah.
And so that's how I that's how I start.
I'm writing every week, they know.
Yeah, it really helps.
Well, I was gonna say, what's crazy is like it's it's kind of hard these days.
And I and LMs kind of have really, I don't know, I don't like LM writing.
I do like it for ideation, like making outlines.
Yeah, yeah.
We do here's my un organized thoughts, make it into an outline, and then like, you know, I'll even be like put bullet points in the outline, and I'll literally read the outline and then like ideate and write in parallel.
But yeah, that's that's how I feel about writing, I guess.
Write more.
I have a strong for nonfiction writing.
I really like this book caught on writing Well.
That's just a really good classic book.
It's actually summarized and synthesized into a r into a skill for me.
Oh, yeah.
Yeah, yeah, yeah.
So, hey, please edit this.
Use these, use this style guide, use the like learnings from this book.
So, yeah, stuff like that.
Yeah, okay.
And then do you like have a a topic idea list that you grew?
Like my I've put mine in the Apple notes now, but it's bro, it's no, never.
Never.
I'm one I'm just I'm just a one-shotter.
Whatever is on your head.
Yeah.
Usually I one-shot the idea all the way.
Yeah, yeah.
Usually I think about it for quite a bit.
So it's been bouncing around in my brain.
And then at some point in time, I've like condense enough information to make a really crappy outline.
And that's usually when I just one-shot go.
For me, like it's hard to one-shot and bounce because you will forget, right?
And sometimes you you have like really good stuff that you forget.
And sometimes it's actually so I call this mise en place writing, where you basically just have a store where you're just kind of writing or working with ideas in parallel.
And then every now and then you cook.
Yeah.
And so this is async and this is sync, right?
This is like passive, like, oh here's a data point.
Here's here's a quote.
Here's a thing.
I'll just slot it in the right thing.
And then and then I bake it.
Historically, okay.
So how that pre-writing actually works today is probably in the semi-analysis slack.
It's just like all the little search it up when you need to.
Yeah, it's search it up when I need it or something like that.
But like I do most of the pre-writing I think in my brain, and I have places that I put it out that I reference it later.
But my favorite thing too is like when it comes to the because like okay, well, once upon a time, much more on the beat.
Oh, hey, here's earnings, read every single one and put it all together.
But like my favorite skill or tip or whatever is like, hey, do the pre-writing, think about it, all that stuff, and go to sleep and wake up.
The next the fresh context window in the morning is my number one advice on writing.
Tell C D code yeah better.
It helps so much better.
Like literally, if I'm like, hey, I need to write something right now, I will do, I'll write it all down, I'll make outlines, I'll do all kinds of crap except for writing it, and then I'll be like, and I'll go to sleep and then wake up and the first thing I do I'll open up a new tab and I will w write it.
Go.
And then so usually I that will get me to 60 75% of something even if it's like an outline where I like have gotten all the ideas enough to know how to fill it out the rest of the way and then that's that's how I take it from there.
Cool amazing last thing hike.
Yeah so one bit of context for me is I I just I just I've never taken a break.
Never and I feel like you know if you take a break in this time you're like just gonna be so behind.
You're just gonna so miss out.
I just found out my friend from OpenAI took a break a year off to bike through Japan.
And he's like how how could you?
Like you're gonna miss it.
You're gonna miss everything but he's like I'm good you know like I'm I'm you know having kids whatever you did a sabbatical as well and like it was pre AI but it was interesting.
I I you did you did the Appalachian trail which one was so there's three big ones in the United States it's the Appalachian the Pacific Crest Trail and then there's a continental divide trail.
So I did the continental divide trail which is the longest and most remote of the three.
Okay.
Sometimes considered like the the the older bad whatever.
But like honestly, the PC, they're all different trails of like I'm I'm pretty steeped in hiking culture.
I think mile for mile AT is actually the hardest.
But I did the CDT as my first trail as my first through hike.
You know, you learned a little bit about the three when I was choosing which one I wanted to do, and the CDT was the one that scared me the most.
I was like, hey, this would be the hardest, biggest accomplishment I could possibly imagine.
And I thought, if I never have an opportunity to ever do this ever again, which so far seems to be pretty correct.
Which one I'm gonna do to feel the most like, hey, I did the thing that I really wanted to do, because I've always wanted to do a long distance hike.
And so I chose the content of the Video.
I did that in 2021, pre-AI and But after the GPC 3 essay.
After the GPC 3 essay, yeah.
I felt like I was missing out a lot, and there's like a huge it was a huge year for Substack.
I feel like I missed out like a very big year of like the big growth.
You're doing okay.
Yeah, I'm doing I'm doing fine.
But I I just think that for me is something I always deeply wanted to do from an intrinsic perspective.
I think something is like like life fulfillment.
Yeah, life fulfillment.
And and I would definitely do it again, but I probably to be clear for people.
It's like four months, five months, six months.
Six months, six months.
Six months, 2800 miles.
We'll s we'll call it on the route.
2850 or whatever the miles I went.
And like you meet people on the way, but you're mostly alone.
Mostly alone, did it alone, you get the trail name, it's a whole audiobooks.
I listened to audiobooks until I hated them, listened to music until I hated it, got bored as hell.
Like you just you just you go, you f you go through all of it, actually.
Yeah, yeah, it was awesome.
Six months.
I think the thing I think about is so far in most in in my life up until that point, you get kind of get kicked from situation to situation, right?
You create a f a view, a form of yourself, you think you know yourself, you have ideas of what motivates you, how do you react in situations, blah, blah, blah, blah.
I think the one, the CDT amount, like it's just like I like I like the outdoors, I like hiking, I'm like good at it, whatever.
It's just something I really appealed to me from an adventure perspective.
Like when in modern life you get to say, hey, I'm going on an adventure.
Never.
Like, and that's what it was.
It was a it was an adventure for me.
And one that I got to like really you you you know, it's like, oh, the journey is the destination or whatever.
You learn a lot about yourself, in fact.
I learned it didn't grow me up per se, but I feel like I am more well defined of my view of myself.
I understand how I react.
I actually know where my exact line, or it's like, you know, you're like, oh, I'll go do this.
It's like actually, no, I know my exact line where I'm like, I would not do that.
I know exactly where I'm not gonna that's too scary, too hard to, whatever.
Yeah.
I know my limits a little better.
I feel like I know just more about myself.
It is a very condensed version of a very intense life.
And yeah, I wouldn't give up that experience for anything in the entire world.
It was extremely personally meaningful to me.
I think it's very fun to go back to the lower part of the Maslov's hierarchy of needs.
Like all this crap where we're talking about today is so abstract, it's like totally fake.
And we were not born and built for it.
We were born to like, you know, our human evolution got us to like scrape a living in the mud, okay?
Hunt and get hunt and gather and just not die.
It's kind of interesting to go backwards and to see what feels like, dude.
I was so hungry, so scared, so alone, so like, but also like super low.
The the phrase is like lowest lows and highest highs.
These crazy lows where you're like, what am I doing?
What does it all mean?
Highest highs and me like holy crap, it's so good just to be alive.
All these things where it's like it's just so like the raw experience of life is so meaningful, and you don't get to experience it well doing it that way.
And so yeah, I wouldn't I highly recommend it.
It's very I would do it when you're younger.
I wish I did it after college.
Yeah.
Like right after college and said hey like whatever kick us out of here.
I think it's good to learn about yourself.
It's really important.
You're the your self-mastery is your most important tool use of all so yeah with that.
Yeah self-master is a most important tool use.
Yeah amazing well thank you for jumping on and like covering everything.
Yeah I feel like I got like got to go through the sort of quad code psychosis all the way to the semi usual all the way to the hiking.
Yeah thank you for having John yeah so this yeah great to catch up