# AI Infrastructure Strategy: Lessons from the Internet Boom

**Podcast:** a16z Podcast
**Published:** 2026-11-11

## Transcript

We live in a world where demand for AI capacity tokens is near infinite and supply is incredibly constrained.
So another way to put it is we all just live in Jensen's world.
You were doing back then M&A for the most valuable company on the planet.
The retro on the M&A process that I think worked well for Cisco was we were not afraid to fail.
And we did it with high enough frequency that we got a few outliers.
You bought this thing for 500 million.
and like it's doing a billion in earnings two years later.
When you see the AI build-out, how do you look back on your previous experiences?
If you look at the revenue ramp of Anthropic, OpenAI, are definitely experiencing growth rates that were far superior to what we saw at the time.
Do I think the build-out that's happening now is useful and will be fully utilized, if not more?
100%.
Will it be owned by the same people that own it today?
The last great infrastructure boom built the internet.
This one is building the infrastructure for intelligence.
What can the first tell us about the second?
In this episode, A16Z's RagoRagoRab and Guino Appenzeller sit down with Hanabi Capital general partner and former Cisco executive Mike Volpe.
They compare today's investment in GPUs and data centers with the fiber boom of the late 1990s and why Mike thinks the infrastructure will remain valuable even if a correction reshuffles who owns it.
They also discuss NVIDIA, custom AI chips, networking, and what Cisco's acquisition strategy can teach founders and investors today.
Mike's view?
The opportunity underneath this build-out could ultimately be even larger than the internet boom.
Great to have you here on the podcast, Mike.
And it's a very, very opportune time to have you because you were one of the architects of the great internet build-out when you were at Cisco.
And now, of course, we've got the great AI build-out happening.
So we and I in the audience would love to get your thoughts on then and now from all different dimensions, from the actual build-out, the capitalization of it, the technology differences, et cetera, et cetera.
So let's start, for those that don't know or remember, Mike was a senior exec at Cisco, overseeing a lot of the...
M&A and strategic moves that Cisco made in the late 90s.
And of course, that was the time the internet got built out.
So Mike was in the middle of all of it.
So clearly, let's start from there.
So you were there when the internet started taking off.
And then you guys rushed into a lot of great things.
And the industry rushed to build a lot of bandwidth, etc., etc., which for some time was not useful.
And then it turned out it was actually too little, right?
There are some loose similarities to what's going on now.
So when you see the AI build out, how do you look back on your previous experiences?
Well, first of all, it's an absolute privilege to be here with two legends here.
Both you, Raghu, and the case of Guido and I, we got the chance to work together at least once.
So it's fantastic to be here.
And as you said, an interesting time because history doesn't repeat itself, but it definitely rhymes.
Yes.
So there are some similarities.
Maybe useful to give.
some context on some of the observations of the late 90s build out and what some of those things might apply in today's world versus not.
Some of the kind of similar themes was that we were building out this technology called the internet and the web and all the things that wrote on the internet.
We had expectations that this would be massively transformative to the world.
And we imagined a wide range of applications.
that would ultimately massively benefit from this build-out.
Applications were different media types, right?
So first we just had pictures and text, and then we had audio, and we had video, and then we had high-quality video.
Applications like commerce, applications like social networking, all of these things were imagined in the 90s but didn't exist.
In parallel with that, there was a very big surge in demand.
At a minimum, there was a surge in demand for connectivity.
And on top of that, a reasonable surge in demand for bandwidth.
And in parallel with that, there was a great influx of capital of two shapes and forms, equity and debt.
So if you look back at sort of what happened, first of all, in the bandwidth world, and in particular in fiber optic bandwidth, the elements, the increments of increase of the amount of bandwidth are quite different than compute.
And that's because we invented this technology called WDM.
that essentially takes a piece of fiber and said, if you used to put one gigabit on there or 10 gigabits on there, by putting in more colors of light in that same piece of fiber, you can now have 40 times or 80 times that capacity.
So essentially, capacity increased quite dramatically with the advent of technological change.
That's actually different because compute ultimately is bound by something that resembles Moore's law.
So the pace of improvement.
is more moderated, hard to believe, but it is actually more moderated in compute than it was at the time.
And the other component that is, I think, different is that the practical applicability of AI, I believe today, is higher than the practical applicability at the time.
of network bandwidth.
And that represents itself by the revenue ramp of the providers.
So if you look at the revenue ramp of Anthropic, OpenAI, I don't know what the numbers look like within Google, even the providers of the open source models like Fireworks, Together, Modal, et cetera, et cetera, are definitely experiencing growth rates, maybe not quite at the same margins, but certainly growth rates that were far superior to what we saw at the time.
Now, Then what happens?
At some point, there is a shock that some of these businesses that are built on the internet are not good enough.
And then that ripples through the industry.
And ultimately, the players that were most saddled with debt went bankrupt.
Now, it's interesting to note that debt was buying assets, was fiber optic cables.
And those assets actually turned out to be quite useful in the future.
Exactly.
Except they were operated by somebody else because the guys that went bankrupt didn't have them anymore.
So the first thing to recognize is the underlying assets that were built at the time of this bandwidth, and I suspect this compute this time, it'll be useful.
There's no question that it'll be useful.
Whether it will be owned by the same companies or not is an interesting economic question.
So that's a long-winded answer to say.
Do I think the build-out that's happening now is useful and will be fully utilized, if not more?
100%.
Will it be owned by the same people that own it today?
A lot of interesting dependencies.
And a lot of things that, in fact, are outside of the control, because they're macroeconomic and geopolitical and so forth.
It's really outside of the control of the builders themselves.
And in particular, when I talk about the builders, Some of the balance sheets are pretty healthy, right?
Last count, when I was talking to folks at NVIDIA, there are 200 NeoClouds, and those balance sheets are not good.
Yes.
So I'll predict that there'll be some.
Not all of them, of course.
Yeah.
Some of them better than others, but broadly, I expect that there will be an economic correction in the market, but the underlying assets will be quite useful for a long period of time.
Isn't that one other big difference from how it was back then, which is that everything.
is now much, much larger.
I mean, back then we were investing a small fraction of a percent in infrastructure for the internet.
Today we're investing 2% or so of our GDP, right, in AI infrastructure.
Yeah.
Well, it is much bigger, but it's an interesting concept, right?
Because in the 100,000 feet abstraction, what is it that is being built, right?
Back then, it was communication that was being built.
Today we're building intelligence.
And The economic value of intelligence is probably higher than the economic value of communication.
Probably, yep.
Right?
So I think it's unsurprising.
I also think that, believe it or not, I mean, it's been nearly 30 years.
The markets are more efficient now, right?
Which is why so much more risk capital is willing to make its way into AI, which even back then was not able to.
Net is, I'm actually more optimistic about this cycle, but I also believe that there will be some correction and some cleaning up of the industry structure.
The role of IT has also massively increased, right?
Back then, I mean, in the late 90s, PCs were somewhat common, but not ubiquitous today.
So to speak, software has eaten the world, and it's sort of, for any company out there, it's a very central part of what they do.
Cost of distribution of intelligence.
Oh, for sure.
And not just any company, but 7 billion people have a supercomputer in their pocket.
Yeah, exactly.
So there's much more evenly spread consumption of the underlying benefit that's being produced.
Yeah.
Don't fully recall all the puts and takes in the 90s, early 2000s on the financial side.
Here, obviously, one of the recent shocks of the system was the rising interest rates and the scale of capital causing people to say, hey, This is going to cloud out everything else and we may not even have enough capital.
During the bandwidth build-out and the internet build-out, that was not the case, correct?
There was no sudden increase in interest rates, was there?
No, there wasn't a massive increase in interest rate.
There's a couple of big differences, right?
One is that I would say on a proportional basis, I believe more of the capital came from the balance sheet of the equipment vendors than today.
Today, it's a more capital markets-driven.
build out.
So, you know, it's private debt, it's public equities, et cetera, et cetera.
There is NVIDIA vendor, I don't think you're supposed to call it vendor financing, but it looks a lot like it.
So there's a little bit of that, and there's some corporate financing, but it's largely equity.
And it's not tied to specific equipment purchases and so forth.
So I'd say the capital structures are a little cleaner now.
The one thing that I think interestingly, though, favors the 90s is a lot of the money went into the ground in the form of fiber optic.
And when you put a fiber optic in the ground, it's good for a long, long time.
It's like it's a usable asset for 20, 30 years.
I think one of the deterministic factors is the compute you put in the ground.
What is its actual life?
And this is a fascinating phenomenon, right?
Because the dominant part of that compute.
is is is chips it's it's it's gpus 80 plus percent yeah it's the big chunk of it all right so i remember back when people were very excited about hopper h100s right you were you're they were like okay well maybe it has a five-year useful life and you're renting it out at a you know buck 82 dollars per gpu hour okay and that was four years ago So the chip should be defunct now, like near defunct.
Not at all defunct.
Not only that, it rents at a higher rate.
It's 250 or three.
So the kind of the interesting part is we thought this asset that have a five-year life, partly because of AI itself, is a useful asset for quite a bit longer.
And in that context, I mean, if you're in the business of financing an asset.
and that asset ends up having a much more longer useful life, then you're the beneficiary of that because your business models were constructed on a short.
So in some sense, I actually think this, the fact, an unintended consequence of AI being good at software is that the underlying asset life is getting longer, thereby helping it to be more financeable.
And that's an interesting phenomenon that I didn't quite expect.
when this thing got underway.
Although the investors are not, the dead investors have not broadly accepted the extended life of a GPU, but do your clouds clearly believe that?
I'm not sure I have either.
I mean, like, right now it seems what's mostly happening is that we have a massive scarcity of an asset, so even previous generations are still worth a lot, right?
If we ever get to a point where we have...
You know, an abundance of GPUs that might completely change.
You might go back to those numbers.
So I think the key metric is how much token revenue can you extract per watt, right?
Maybe, yes.
So the hopper is going to be a lot worse than Blackwell is going to be a lot worse than Vera Rubin, right?
They're all just more efficient at getting dollars out of a watt.
In theory, at some point, when you have sufficient Vera Rubens, it doesn't really make sense to run H100s anymore because, like, it's just the worst way of converting electricity into dollars.
Correct.
Right?
But the problem is right now that the demand is so high that even if the watt-to-dollar conversion is low, you're going to still use the asset.
So, and right now, the scarcity is so significant.
And ultimately...
you know, at some point in the conversation where we get there, but ultimately the scarcity is in the manufacturing of the semiconductor itself.
And that doesn't look like it's changing anytime soon.
Correct, yeah.
And so then scarcity persists, then the life of the asset continues to be long.
Yeah, yeah.
I think it's correct.
Yeah.
And the other big difference, there was a period of time where there was unused bandwidth.
Today there is no such thing as an unused GPU, right?
There's a bit of unused energy and like you could probably...
I don't know if we're going to talk about networking, but with better networking, you could get more flops out of a given GPU.
Yeah, I was just going to say that I'm a few spot of factor in certain use cases and how the people have built it, but certainly, generally not.
Yeah.
Back when you were at Cisco, were you still there when Cisco was the most valuable company on the planet?
Yes, indeed, I was there.
So you were doing, back then, M&A for the most valuable company on the planet, and it worked pretty well, I want to say, looking back, right?
I was very happy with it, yeah.
What would be your words of advice to NVIDIA in 2026?
Like whoever runs M&A there is in exactly the same spot as you are, correct?
Yeah, Vishal is in the same spot I am.
Look, the retro on the M&A process that I think worked well for Cisco was that fundamentally we were okay.
with making enough acquisitions that we got a statistically significant outcome on those acquisitions, meaning some worked really well, some worked so-so, some didn't work at all.
And, you know, if you kind of drew it out, one in each of those three sectors, a third of them worked really well, a third of them were whatever, and a third of them failed.
The benefit that we did is we were not afraid to fail, and we did it with high enough frequency.
that we got a few outliers, Andy Bechtel-Sheim's company, Mario Mazola's company, a bunch of these things that were sort of, you know, even for us, were venture-like returns.
So you bought this thing for 500 million, and like it's doing a billion in earnings two years later.
That's amazing.
And so it ended up that...
During the period that I was there, I joined in 94 and I was like the one biz dev hire.
In 2000, roughly, 50% of Cisco's revenue came from acquired sources.
So that was incredibly useful.
And we actually never really spent much more on a given acquisition than 4% or 5% of our market cap.
Most of the acquisitions were actually, quote, cheap relative to our market cap standards.
How much would that mean for NVIDIA?
I don't even want to think about that.
Yeah.
Well, my rationale at the time was like, if your stock trades more than the price you paid for the acquisition in one day, then you can do the acquisition as many as you want.
Nobody will care.
I don't know.
I'll call them.
Done with equity or was it?
All equity.
Almost everything was equity.
A handful of the small ones were cash.
And Cisco, I mean, they became major product lines.
So it was not like this was a little tuck-in that added a feature somewhere, but this was of a new product line that in some cases became the leading product line in the industry.
Now, you're a networking guy, so you'll appreciate this.
The one thing that's a little different about a networking acquisition versus other ones is that networking operates on open protocols, which means that I could buy a switch company made by somebody else, and it immediately talks to my switches or routers, whatever, freely because all the protocols are open.
So, you know, at Cisco, we had a couple of proprietary protocols that we enjoyed.
But by and large, all the OSPF, BGP, ISIS, Ethernet switching, 802.whatever, they all worked.
So you could immediately integrate.
the product into your sales team and have your sales team, like literally you just change the badge on the thing.
And so that allowed us to be like super high velocity.
Okay, how does that contrast to Jensen's world today?
One, they've made some really clever acquisitions.
You know, like Mellanox is the world's largest network.
That was amazing, yes.
Right?
It's like genius.
Now, they're making other acquisitions that...
Sometimes I scratch my head.
I'm not entirely sure what hugging face is all about, for example.
Like maybe there's a rationale.
Hard to me to see it.
I don't think it matters.
I think it's fine for them to do some of these and take risk.
And some of them will work.
And if they work as well as Mellanox, holy cow.
And some of them are not going to work.
And I think the market has granted them a moment of incredible optionality.
Right.
Basically, they get to do whatever they want, which is the optimal moment for them to take these risks.
So I think they're being very clever and you can always critique any given acquisition.
Trust me, I got critiqued endlessly at Cisco because I bought Stratacom and not Cascade.
And I, you know, bought an optical networking company from Vinod that went nowhere and whatever.
But in the end, you're like, all right, well, I did 70.
I think our team did 70 acquisitions.
I was there and those turned into half the company's revenue and we were doing 30 billion in revenue.
So it's 15 billion in revenue.
The strategy worked.
And I think that the team at NVIDIA is doing that.
I think that's interesting and wise.
The degree to which they are financing the industry.
We'll see.
Yeah.
Because that's different, right?
In that sense, NVIDIA is basically, I think they're basically trying to shape what an industry looks like.
Yes.
And I think industries are powerful and they find their own shape.
It's hard to reshape them.
Let me defend NVIDIA a little bit.
Having been at Intel and...
And having seen how the big hyperscalers essentially disintermediated Intel from the end customers, right?
And at the end, they could build better chips than Intel just because Intel had no idea anymore what the customer was doing with the chips, right?
I have some sympathy for wanting to have more than three customers to sell to or four, maybe.
As do I.
So, and you could argue, what is it, efficient capital allocation strategies that given it back to investors or could it be at this time?
which is probably once in a time in history, put it all to work on future innovation, right?
Well, I think both of you make very valid points.
The thing I would say, Greedo, is that an industry wants to look a certain way because when it does, it creates maximum utility, maximum economic value.
Why are there three hyperscalers?
Because it's a business where scale matters.
Software, yeah.
Right?
It's a business where, like, Scale matters, and therefore the industry wants to concentrate.
No matter how much you finance them, the little ones are always weaklings.
So, and obviously if you're NVIDIA, you don't like that.
You don't want three customers.
You want 50 customers.
So you fund them, but at some point they're going to die.
It's just because the industry, whenever it's a scale business, wants to consolidate.
You know, we could fund 100 airlines in America, but in the end, you're going to have three.
So my point is not that it's a bad idea.
It's just that it's futile.
So noble intentions, but...
Now, to your point, it could be a very good return on...
I don't know if they actually care about the return on capital.
They might.
And if they get out at the right time...
Yeah, it's not even about the return on capital.
I agree.
It is to some degree.
shaping industries, deciding the shape of the industry.
But on the other hand, it is enabling a lot of innovation as well because our startups and lots of other startups do get access to that capacity now.
Yeah, yeah, yeah.
No, no, look, overall, I get why they're doing it.
And, you know, many of our companies are beneficiaries of it.
So thank you, Jensen.
Exactly.
So, I mean, at VMware, we did probably not as many as you did, but maybe 50 plus acquisitions over the full lifetime of the company.
And the observation was whether it didn't matter whether you were acquiring a 20% company or a 2,000% company, the amount of post-acquisition work in terms of integration, culture, business model, sales team, et cetera, et cetera.
It was always a lot of work.
How did you guys pull it off at that time at Cisco?
Okay, structurally, the way we did it was that we had a very small, when I say small, less than 10 people deal team that was jobless to acquire.
And we had a somewhat larger integration project management team.
that had sort of rules and guidelines about how we did integrations.
And they had colleagues, those people that project managed it, had colleagues in the various business units and divisions and functional units within Cisco that collaborated with them.
So the important thing to recognize there is that businesses, companies, are good at processes.
they're very bad at one-time events.
And so essentially, if we were going to do a large number of acquisitions, we needed to have repeatable processes that were performed over and over again.
And we had a centralized group that was the administrator of those processes.
More often than not, and in the beginning at Cisco, we did this a lot, there was nobody that was formally assigned to this.
So it'd be like, Raghu, it's your job.
You go.
Have fun with that integration.
Let me know when it's done.
Right.
And you're like, where do I even start?
That was like the modality.
And we're like, OK, this is bad.
So we created this process and it worked pretty well.
Where it got challenging is we liked to buy small things and small things became too small to become significant to the overall business because the business got so big that like, you know, it just didn't make a difference anymore.
And then when they tried to.
This is a little bit after my time because I stopped doing the M&A stuff in 2000 and switched over to running a business unit.
But after my time, they started buying bigger things.
And then you get into the problem that you're talking about.
Big things are very, very hard to integrate.
They've been around a long time.
They have a long culture.
We bought Scientific Atlanta, which was a disaster.
We bought a bunch of things.
didn't work very well.
We also got into the mode of like, oh, we're not fully integrating them, we're like partially integrating them.
Yeah, it's terrible.
Yeah, it doesn't work very well.
So, in other words, the mitigation is to have a process, but then when you stop respecting the process, you do have big problems like you talked about.
Yeah.
And what about from the portfolio company side?
I mean, you got portfolio companies, you got portfolio companies that at least...
In recent memory, we have had some exits into being other large companies buying them.
Two questions there.
One is, if you're a portfolio company CEO, what would you watch out for, right?
And then sitting on the boards of these companies, how do you advise your portfolio companies these days about exits in general?
Look, when you're...
When you're in our business of being on boards and you're facing a situation where you need to sell a business, it's one of three outcomes.
One, your company's doing great and it's a really bad idea to try to sell it.
So then you try to convince the CEO, no, don't do it.
Two, the company's struggling.
We don't have a lot of finances.
It's a nice way to get out.
Everybody's happy.
Then you just help the deal happen.
You sort of manage a process.
Then you have the tweeners.
We are not totally sure.
And I think that there are a couple of important indicators that help you determine what to do in that moment.
The first is, does the CEO, the founder, the founding team have the fire?
Like, do they want to do this?
Do they believe they can get through this sort of intermediary period and bust open into the next phase, do they want it?
Because if you have a CEO or a leader that's kind of like, man, you know, maybe I should consider this.
I'm a little tired.
It's been a long haul.
Then let's just sell this thing and kind of move on.
And then the second factor is you have to be pretty cold and objective about like, is this going to be an interesting enough business?
Not like, will it double from here?
Because at the end of the day, if it's double, I'll take the money today.
but will it 10x from here?
And that, I would say, gets to be a really hard decision because it's a little bit emotional also for you.
Like you're, you know, you're not a founder, but, you know, you've been working with this company for four, five, six years.
Your heart's in it.
You've told your good friends to go work there.
And yet you have to step away from it and go like, objectively, this is not 10x from here.
And then.
you help steer the ship.
But I think the most important parameter is where's the founders head at?
Yeah.
Yeah.
And so there is a certain element of mystery or unknown for a lot of the portfolio company founders on how do these things actually happen, right?
People think one day somebody calls another company and say, hey, I want to buy you versus, and then there's other thing where people have built relationships for a long time.
How did you...
approach it at Cisco and how do you advise your portfolio companies even if they're expecting a 10x?
Yeah.
You just call them up, say, hi, I want to buy you?
No, well, obviously you do as a good a job as you can building a network and knowing what's going on.
It's basically cataloging the inventory is the way we used to think about it.
But every company in the market is a potential acquisition, so you catalog them.
And, you know, Where are they in their growth rate?
What kind of founders they have?
Who are the backers?
What are their aspirations?
Who are their customers?
Just like be informed.
So, and the, and the, you know, the business, the corporate development group had to do that, but I kind of considered it everybody's job at the company to be constantly informed about what's happening around them.
So, and then essentially.
At the company, we would try to weave in, at Cisco, we would try to weave in the inventory against a strategy statement that was there.
For example, we're a router company, but there are going to be more switches than routers.
Therefore, we need to have a switch.
What are the switching companies that are available, right?
Or optical networking is becoming important because of bandwidth, blah, blah, blah.
What are the companies that are available?
You often have the other way where companies come to you and say, hi, I'd like to get bought.
I think broadly a bad idea, right?
Because there's either an auction going on or you should have thought about it earlier if you were interested in it.
But basically, you almost never pay attention.
We never tried to pay attention to inbound.
Flip side, you're now a small company or a startup.
And how do you think about that?
I always encourage always the founders of our companies to talk to big companies, to talk to competitors, and to always be in dialogue.
Some people will say like, well, are you shopping yourself?
No, it's an inverse intelligence gathering mechanism.
I even encourage my CEOs to talk to their direct competitors.
Go to coffee.
Like, yeah, if you're clueless, you might leak some information towards them.
Hopefully you're not clueless and they might tell something to you that's interesting or they might shape your view on an industry or something.
So I encourage people to be out there always talking.
and seeing what's happening in the market.
And usually you kind of have a pretty good idea of like, okay, my company is struggling a little bit.
I think I'd like to sell.
I kind of already know then because I've been in conversations with who's where.
So I would say, you know, don't, a CEO of a startup shouldn't spend their entire day talking, but going on one or two coffees a week.
with, you know, the powers that be at NVIDIA, the powers that be at Amazon, OpenAI, et cetera, et cetera, when the right people are talking, do it.
It's fine.
The information exchange, you don't lose that much in that.
They're already thinking about it.
But you gain a lot by having that relationship.
I think this is really good advice.
I keep telling my portfolio CEOs the same thing.
If you don't have the cell phone number of the CEOs of each of your competitors, that's a problem.
There will be a day when you need it.
Absolutely.
So it's a network you need to build.
Yeah, yeah.
Now, enough of this M&A stuff.
Let's get back to actual technology.
You spent 30 plus years in infrastructure.
Very few people know more about infrastructure than what you do.
what you've done in the past.
Summarize the landscape for us right now.
When you think about infrastructure, opportunities versus where is the technology gaps that you see and what founders should be really looking at.
Well, super interesting topic.
You know, I would say...
You want to think about, I think, the current shape of this industry in two dimensions, right?
One is the vertical stack that starts with, you know, EUV equipment and goes all the way up to AI, applied AI and applications and everything that sits in between that sandwich.
And then you want to think about particularly the infrastructure side.
What's that?
What's sort of the value chain of it?
The first thing you would say is right now we live in a world where Demand for AI capacity, tokens, whatever, is near infinite.
And supply is incredibly constrained.
So another way to put it is we all just live in Jensen's world.
He's king of the hill, there's no question.
I think the shape of the industry will begin to change when either the characteristics of demand change.
or the characteristics of supply change.
And I think that my view is right now the supply side, because so much of the limitation, well, there are some power limitations for sure, but the dominant limitation is silicon.
And I would say that unless there's more fab capacity, that doesn't change, which means it's not likely to really change in the next two or three years, maybe five years out.
You could imagine that there would be more fab capacity.
You could put more wafers through the same machines or you can make more ASML machines and so forth.
But that's probably a governing factor of our industry, which means we live in a reasonably scarce world.
And those who have fab capacity are actually kind of kingmakers of the infrastructure.
On the demand side, I do think that the world is going to get smarter.
about the type of tokens they consume.
I think we've been in the kind of an era over the last couple of years that everybody just wanted the super high-end frontier token.
And I think the topic of conversation today is model routers and how do I move stuff to, you know, the economic difference between a top-end Anthropic token and a top-end Kimmy token is like five to one or seven to one or whatever.
It's a massive economic difference.
So people are going to get start smarter about consuming what?
And I do think that that will begin to at least put some price dependency on demand.
And that's more likely to happen in the near term.
So those are, first of all, I think the dynamics that are going to govern our industry and everybody that sits in between, you know, if you're a neocloud or you're an inference cloud or you're a model builder, you leave between those two poles of supply and demand.
Now, for the infra itself, I think right now we still live in a world where you need to think of the entire piece of infra as a single system, right?
So it's quite different, I think, than the networking days, for example, of old, where you would assume a very variable amount of traffic going from one to be types of traffic, times of day, lots and lots of variability.
much easier to statistically multiplex that workload.
Today, basically, you have lots of GPUs that are all working all the time, always connected with much more constant flows.
And because the interrelationship between the various components are largely still proprietary, you know, you're on NVLink or whatever, it's very hard to like, swap one thing out for the other.
And in fact, today I would argue that probably the best architecture is the Google TPU architecture because it is literally thought of as a single system.
And I think an NVIDIA style architecture is migrating in the direction of where the TPU architecture is, but it still has a few pieces that you can kind of swap out.
And then you have everybody else who's like, you know, I'm going to take a Cerebrus chip and throw an Arista switch on top of it and like all that good stuff.
And at least in today's world, those systems have inefficiencies because they're not constructed as a system.
They're more of a patchwork.
I think that industry will evolve.
But today, people that can think of the whole thing as a single system and optimize the architecture of it through that have a competitive advantage.
Yeah.
I mean, it seems we're moving from a card as the minimum unit to a rack as the minimum unit, or even a few racks in some cases.
A few racks.
Yeah.
That's true.
The media part that they announced recently, that is the minimum unit, possibly.
Yeah.
Well, but if you look at how Google does this thing with these, like, toroidal, pass-through optical MEMS switches, it's literally like the whole fucking data center is the minimum unit.
So the row is the minimum unit.
Yeah, exactly.
Yeah.
But my sense is that's where NVIDIA wants to go also.
So the two dominant providers are all going to that systems approach versus...
I mean, even the smaller startups are heading that direction.
But tell me, how do you see AI chips evolve?
And right now, Jensen, king of the hill, incredibly dominant, right?
Google TPU, very successful, currently mostly sold by Google.
It's expected to change.
Not enough TPUs.
But not enough TPUs, exactly.
How is the future looking like?
Will we see more competitive silicon?
I think for sure.
We all want that.
Yeah, we all want it.
And we all want to invest in it.
But I would say, first of all, what I find interesting is that model architecture is becoming reasonably diverse.
Right?
So you have...
Some people are using FP4, FP8.
Some people are using mixture of experts with lots and lots of experts.
Some have fewer experts.
Some are jamming up the parameter count.
Some not so much.
And I actually think that there is an interrelationship between your chip design and the model that's optimized around it.
How to software co-design.
In some sense, to your point, is there going to be more diversity in chips?
And I think yes.
because there's going to be more diversity in model designs.
And depending on what kind of model you want to do, what kind of things you want that model to do, you can certainly select a different chip architecture for that.
Now, reasoning intensity has dominated to date what chip architecture should be because reasoning intensity equals big parameter counts.
and big KV cache.
And that means that you need to talk a lot between memory and chip.
And, you know, when the only way you can talk between chips are the edges of it, it's just less connections than if you can use the top and bottom of the chip.
So I'm a big fan of 3D DRAM architectures.
And some of the new stuff that's coming up with hybrid copper bonding is even more interesting because you can have even higher density.
So I think that architecture has huge promise.
So long as we continue to see these KVCache, in particular KVCache intensive use cases, and it seems like the most value added labor that occurs, labor or AI that occurs.
is reasoning-driven labor, right?
And therefore, that should favor architectures that have very good memory bandwidth.
We're investors in Cerebris.
Cerebris did very well to go fast, but if the model doesn't fit in the SRAM, you know.
You need a lot of them.
Yeah.
Well, yeah, you need a lot.
Well, because once you get off board of Cerebris, it's not going to go super fast.
And also, because the chip, the wafer is so large, you're actually at a significant disadvantage in terms of the memory bandwidth once you go off chip.
So, now, they run Sol on the Cerebris chip, so that's a big model.
So they've figured out how to fit it in.
But my guess is OpenAI had to do some hard work to get that model to fit in the chip.
Yeah, no, we are fans of 3D architectures as well, but...
The manufacturability of it is a lot more complex with those effects and so on and so forth.
I'm a big fan.
I'm a mechanical engineer that studied thermodynamics.
There you go.
Heat is my friend.
That makes you one of the very few people who would say that.
Yeah.
So all the packaging divisions of TSMC and everybody else are calling you right now.
Yeah, yeah, no, I agree.
The packaging of those chips are going to be complex, especially, obviously, the thermal attributes of it.
And then when we get to a world where you have both a logic chip and a memory chip in the same package, the different coefficients of, well, they have the same coefficient of thermal expansion, but logic chips get a lot harder.
So, you know, the ability to put that all into the same package will be non-trivial, shall we say.
Chips are hot again and they're getting hotter.
The other possibility, of course, is to keep them still separate but have optical interconnectivity between them.
Are you believing that?
Yeah.
I think yes.
Optics, though, are very finicky things that need to be experimented with a lot.
So, you know, when you heat up, optics, they don't like it.
Photons don't like getting hot.
And I mean, in theory, you would have a substrate that had waveguides in it or something, but the minute the logic chip starts getting hotter and you get sort of outside of the comfortable temperature range, you're going to have a variety of connectivity issues and so forth.
So I think optics are interesting.
If you can do it with copper, I'd do it with copper, is my view.
But if you...
Really?
Yeah.
Even for scale-up racks, not just interconnect now, but even for...
No, no, I mean, obviously for scale-up racks, that's a different story.
That just simplifies the cabling and so forth.
I'm talking about it at the...
Inside the pack.
Inside the server.
Yeah, yeah, yeah.
When you start going top of rack and all that...
Yeah, you need it.
Definitely, no question.
You just can't, you don't have enough bandwidth.
So, Cisco back then, most valuable company on the planet.
early hardware leader in networking, which sort of powered all of the internet, at some point wasn't the most valuable company on the planet anymore.
What triggered that?
What can we learn from that?
What tends to happen when you have a bubble in demand, in perceived demand, is that customers over-order because they know they're going to get cut back.
And so every part of our product line was out of capacity.
We couldn't make enough switches.
We had orders on top of orders, on top of orders, on top of orders.
And then we were, of course, putting orders on all of our supply chain.
And because the demand felt so robust, we were taking the financial risk on those orders.
In other words, we were like, here's the money.
Keep the money.
Just get us this stuff as soon as possible.
That would possibly go wrong with that.
Yeah.
So then...
When demand dries up, it's not like people stop buying.
They cancel all their orders.
They're like, yeah, I ordered twice as much as I wanted to anyway.
So I need half of half of that.
So the orders went to like one quarter of what they were.
And then A, you've taken the risk on the supply chain.
And B, your demand falls through the roof.
And so I think.
We ended up in the December quarter, and this is kind of memorable because it happened in December quarter, so we were one month off.
So November, December, January, and usually what tends to happen with businesses that are a month off, you know this, right, like December soft.
And then the first week of January, you start seeing it coming back.
And then by the end of January, you've made it up.
So we go through December.
It's soft.
And it's like, no big deal.
December is always soft.
And then week one, January, nothing happens.
Week two, January, nothing happens.
Week three, January, nothing happens.
You're like, oh, shit.
So the quarter came in.
I don't know.
Maybe we missed by 35, 40 percent.
Wow.
Wow.
Yeah.
Massive.
For a company, by the way, that publicly traded for a decade, never missed a quarter.
Yeah.
And yeah, and then that's how it all corrected.
And it was difficult.
Stock went from 80 some dollars a share to nine dollars a share, I want to say.
So it was like literally one tenth the value over a period of six months.
Very tough.
Yeah.
So look, every time when the workload changes enough at the top layer, the underlying infrastructure, software and hardware needs to change with it.
With AI, we've seen this.
We talked about compute.
Right.
What's going to happen in networking?
We've seen big changes in networking.
We no longer have one network.
We don't have suddenly the second back of network.
Where did this come from?
Did you just say Ethernet is dead?
There we go.
Ethernet is dead.
No, no, no.
I mean, Ethernet is a framing mechanism.
It's coming back.
It was dead, right?
Here I think that the core tenets, the world of telecom started with circuit switching.
Right?
Like we made the phone and the idea with the phone is that you had Starting point A, starting point B, constant bandwidth between two, you connect the line and off it goes.
And then we found out, well, this is like reasonably unreliable because one line cut and you lose it.
And obviously it doesn't scale very well in a world where you have uneven traffic patterns.
So then we decided, let's go to packets.
And packets are very convenient because they allow statmoxing, which is the big, big benefit of it.
David and Andy back then, right?
They're cheerleading it.
So that and, you know, variable length packets and all of these, you know, IP packets, essentially.
So essentially, the world grew up building networking equipment, assuming that you had uneven arrivals and departures into a device that spread out the workload and you would under provision the device.
to connect more bandwidth because not everybody all communicates at the same time through the thing and the outputs of that inbound are highly variable.
Okay, that's the internet, that's the workloads that we live with.
It was almost dogma, right, for a long time.
Okay, so fast forward today.
What does an AI data center look like?
It does not look like that.
You don't have random arrival.
Links are consistently fully utilized.
GPUs want to be fully utilized.
Traffic patterns, are actually much more predictable than they ever were before.
Particularly in training, they're incredibly predictable.
They're less predictable in inference, but you can see the traffic pattern, especially if you do good batching.
Good chunk of it is still very predictable.
Yeah, especially if you do big batching.
You can definitely make it more predictable.
So the fundamental tenet of the workload has changed.
Yet, if you look at the dominant networking architecture today, which is the Broadcom...
what's it, not Trident.
Tomahawk.
Tomahawk, yeah.
Which is the Dune acquisition that they did, is a statmuxed packet switching architecture.
Yes.
So I think that at some point for data center networking, we're going to have a new kind of architecture that assumes a high degree of predictability and a full utilization as opposed to an unpredictable, a stochastic statmuxed approach.
So essentially back to circuits is what you're saying.
Yeah.
And optical circuit switches.
Yeah.
Or, I mean, time division multiplexing works as well, right?
Or combinations of those.
Yeah.
You know, you can have many modalities of multiplexing, but they're not stochastic, right?
They're constantly timed somehow.
Like you can do TDM, you can do whatever else you want, but basically.
So, and I think these are...
The issue is the switches you buy today have the correct amount of bandwidth and the incorrect architecture, not incorrect, but less adapted architecture.
And I think the data center switches of the future at some point will look more like that.
Will look more like.
You know, potentially you could do it with MEMS.
So bring ATM back into the data center.
But even ATM had the statmux, stochastic cells, right?
Straight up circuits.
Yeah.
So now networking vendors, including my former employer, this is bad news because when things don't change, you can't charge a lot of money for them, right?
Like the whole point of a switch is that it's a complicated animal.
to have a statistically multiplexed traffic flow.
It's a hard thing to have, oh, my one packet came in here.
I have to throw it into memory.
I have to do a lookup, do the lookup.
I got to send it out of this gate.
That's like hard to do.
So you make the thing expensive.
But in a world where it's relatively static, you don't need that much intelligence.
You don't need buffers anymore.
Bam, send it right through, right?
So I think that there's an opportunity for somebody to sort of rethink network architecture.
given this workload and make it way cheaper.
It's like classic innovators dilemma because no matter what vendor you are today, this is not a good trend.
You're not going to want to do that.
But if a newcomer and you say, you know what, forget all that stat muxing shit, like the world's different.
I'm going to go with my thing.
Actually, you might be able to build something interesting.
Now, it could be optical.
It could be electrical.
It could be a combination of the two.
But I think we're going to, what will emerge, and I think you can see clues of that from how Google does its network.
I think Amin has been the pioneer of specifically optical circuit switching.
They were the first ones.
That's for like their toroidal thingamaging.
I don't understand why it's so slow, by the way.
It takes them like a minute to switch.
So that one I don't totally understand.
I think to some degree it's because of the use case that right now you use it to set up fairly static.
I mean, when it is static, then you can.
Fair.
I mean, for availability, if you want to.
Give me a few milliseconds, not a second.
Yeah, but you probably could, honestly, right?
I mean, the technology, I think, is there.
It's just the use case is not there.
Yeah.
Yeah.
So, yeah, no, I've been looking at a bunch of these, whether it's MEMS or...
is Max Zander thingies, the MZIs and whatnot to see if there's something in there.
But, you know, somebody has to emerge with an architecture.
And then the issue is going to be who embraces it because you can't just build a switch without the rest of the system coming together.
The universe dilemma extends to the user's inertia as well.
Exactly.
Well, at this point, the network to some degree, coming back to the, you know, ship me a row concept of infrastructure today, the network has become an integral component of your architecture.
So this innovation to some degree almost has to come out of the GPU vendors.
Yeah, but it's interesting, right?
Because there's only NVIDIA, maybe AMD will get there at some point, but only NVIDIA and Google right now have the ability to ship you a rack system.
That's right, yeah.
So today they would have to adopt it.
It'd be interesting to see like, you know, as more chip vendors evolve, is there, you know, is there an alternative architecture that they can all agree on?
Sort of the shit that, you know, we did.
Back in the days.
Back in the day, right?
So like, the thing is like, you have two dominant vendors.
The ITF 2.0.
Well, they have two vendors, they're dominant.
They have their own approach of doing things.
They're not likely to shift off of that.
And can you have a collection of everybody else that says, no, actually, this is a standardized network architecture that we're all going to agree upon and that's what we're going to build.
The Rebel Alliance.
I like it.
Yeah, the buyers are going to agree.
That's the case.
Yeah.
So, Mike, you've seen a lot of these cycles now, some bigger, some larger.
You've seen the last really big one.
How does this compare?
As you probably know, I've started a new venture fund called Anabi Capital.
Technically speaking, I retired in 2023 and I was supposed to get a yacht and play golf.
That didn't work out very well, did it?
Why?
I actually think this is the most fascinating time to be doing our job and to be in our industry.
Not just because the change is big and we all know how transformative the change is, but because it requires more intellectual exertion to understand how, what next.
It's like, you don't have a crystal ball.
It's not obvious what the next move is.
Yeah, it's hard.
And so you have to keep thinking and you have to think at a systems level.
Like, I don't think I've ever been in an industry before that I had to think about how light gets concentrated on a Zeiss mirror.
To figure out whether my chatbot works well.
And like, it's all, it's this amazing interconnection of things where there's bottlenecks and this changes and this changes and that change.
So I find it just the most intellectually fascinating time.
And because the end prize is so significant, by figuring out how these different parts unlock, you can create.
a company here and a company there, and this one can become worth a trillion dollars, and this one can become 500 billion, and so on and so forth.
Honestly, I did not expect to encounter another time like this in my career, and by far, I find this the most dynamic and interesting time to be doing what we're doing, and I think we should all feel quite fortunate about it.
Amazing.
This tops 99.
Oh, yeah.
Easily.
Totally, totally agree.
And we are all in the same business for very much the same reason.
This has been a fascinating hour.
Thank you so much.
And we could be continuing this conversation endlessly.
Perhaps we'll do it another time.
This was super fun.
Thank you, guys.
Thanks for coming, Mike.
We appreciate it.
Thanks for listening to this episode of the A16Z Podcast.
If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.
For more episodes, go to YouTube, Apple Podcasts, and Spotify.
Follow us on X at A16Z.
and subscribe to our Substack at a16z.substack.com.
Thanks again for listening, and I'll see you in the next episode.
As a reminder, the content here is for informational purposes only.
It should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.
Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
For more details, including a link to our investments, please see a16z.com forward slash disclosures.