# AI Infrastructure Bottlenecks and the Machine Age Fund

**Podcast:** a16z Podcast
**Published:** 2026-08-28

## Transcript

We have a whole new technology that's the most important technology ever, and you need a whole new infrastructure.
Normally when we talk about the infrastructure world, we're talking about the servers and the storage and the network.
Here it goes all the way down to the mines of copper mines.
That's how widespread this thing is going to be.
It used to be when you built something, it was an engineering problem.
And here it feels like it really is a resource limitation.
So whether it's tokens or not, we're pouring a ton of money into systems and then those systems are producing a result.
And right now we're bottlenecked on those systems' ability to actually match the resources we're pouring into them.
The leading memory runner said the demand they have today will take them three years of capacity to supply.
If this fund does what we think it will do, how do we see the world in five to ten years?
If America wins in the infrastructure dig, damn, that would be awesome.
Today, A16Z is announcing the Machine Age Fund, a new fund dedicated to the infrastructure powering the next era of AI.
I'm joined by Ben Horowitz, Ragul Ragulram, and Martin Casado to explain why we're launching it now and why we believe the next major bottleneck in AI isn't necessarily the model.
It's everything underneath it.
Chips, memory, networking, power, cooling, and data centers.
are all being pushed beyond what they were originally designed to handle.
At the same time, AI is changing an old rule of technology.
Throwing money at an engineering problem didn't necessarily make it move faster.
Increasingly, capital can be converted directly into compute, and compute into more capable intelligence.
We unpack what that shift means, where new infrastructure companies can break through, and why a new generation of founders is returning to some of the hardest problems in computing.
Ben, Martin, Raghu.
Thank you.
All right.
Thank you.
I want to start with a Mark quote to introduce this new fund.
This is the biggest technological revolution of my lifetime.
This is clearly bigger than the internet.
The comps on this are the microprocessor, the steam engine, and electricity, or maybe the wheel.
Guys, the Machine Age Fund, please introduce it.
Ben, start us off.
Well, basically, what's happened is we have a whole new technology that's the most important technology ever.
What happens every time there's a dramatic new way of using all of the things that we love, infrastructure, you need a whole new infrastructure.
And never has it been more high impact as it is on this one.
So not only do we need new chips, new system software, we need new ways of doing power.
we need to replace copper.
I mean, like, it's absolutely everything.
So it's a very exciting time.
So particularly for the kind of hardware aspects of this new era, we needed a new approach.
Yeah, I would agree.
Normally, at least in the computing, when we talk about the infrastructure world, we're talking about the servers and the storage and the network.
Here it goes all the way down to the copper mine.
That's how widespread this thing is going to be.
And that's number one.
And number two, I think what we have seen over the last three years is the steady increase of the capabilities of the models, where the model is no longer the bottleneck.
And in fact, using AI, these models are getting better and faster and faster and faster.
Now the bottleneck is all what I call south of the model.
And so that's why we need to work on that.
The only thing I'd add very quickly is we tend to follow founders.
And we've been watching over the last couple of years as the number of very strong teams going after complex hardware problems has increased.
I don't know the actual numbers, but I was trying to estimate it over the weekend.
So I think we'd get maybe 5% of the deals from top founders would come in, would be hard before.
Now it's the north of 20% or 30% right now.
So it's like the founder community, which tends to be much smarter than the VC community, has identified this as a very active area for innovation, and they're responding.
I think 5% is probably generous.
Yeah, it was very low.
It was very low.
Yeah, 3%, yeah.
And explain some of the macro conditions that have led to this change in terms of the surplus of five years pursuing these ideas.
Like, what are they seeing that's enabled?
Well, I mean, the obvious is the demand for AI is basically infinite.
And as a result of that, every part of the supply chain is under duress.
I mean, everything, including, like, materials used to make things, like memory.
It's also very interesting.
There's something unique about AI.
Which, because the demand is infinite and growth is infinite, what you tend to worry about is the margin of companies, which is how efficient it is.
Like, normally you worry about growth.
Can I just get people to buy this stuff?
You don't have to worry about that here.
The question, can you do this in a way that's profitable?
And a lot of the efficiencies are actually strictly a physical limitation of hardware.
And so even the business model of the AI wave is really putting a lot of stress on the existing systems because they weren't built for AI.
They weren't built for those workloads.
And I think there's just this...
This global observation that we actually need to change the core components to get that efficiency to help drive the growth and to drive the value of the businesses.
And how did we know that demand is actually outpacing supply here rather than this being another hype cycle?
Yeah, there are any number of cities.
Firstly, it is that some of the smartest judges of demand are cutting huge purchasers.
I mean, if you look at the hyperscalers, right?
Their CapEx spend has been exploding.
Next year, supposedly, it's going to reach a trillion dollars collectively across the bay.
Hyperscalers this year, it's about $700 billion, right?
And if you think about the hyperscalers' position in the industry, they see demand from everywhere, right?
They see, obviously, the frontier labs wanting their compute.
They see the AI-native companies.
They see the enterprise.
They see the U.S.
geography, the international geography.
So if anybody has visibility, it is down.
And they've been jacking up their capex, like it's never been seen before, right?
So that's a clear, clear sign.
And secondly, if you look at the companies that we see on a day-to-day basis, they are all ripping.
All the application companies, the growth is insane.
The frontier labs, the growth is insane.
It's been documented.
I would say on the demand side, the signals have never been clear that this is not a hybrid and prices are going up.
Like, we've never seen prices go, like, up on chips.
The GDP prices went down.
They always go down.
It always goes down.
If you look at the price curve, it went like this and then back to this.
I mean, only like 5%, 10% of the others are marking the stock today.
I mean, if you look at the supply across the board, it's basically all booked out to 2028.
I mean, it's so bad.
We've actually seen multi-day auctions for a few thousand GPUs.
The other side of that, of course, is demand.
And as Ragu said, we've seen the fastest growing companies we've seen in the history of the industry.
And also the unit of work that AI can do.
The value of that unit of work keeps increasing.
But underneath the colors, the number of tokens that are consumed is going by orders of magnitude, right?
If it's 100 tokens for chat or agent, it's...
thousands of tokens, right?
So you've got expansion on both sides of demand.
One is the unit of work is becoming more and more consumptive of tokens.
And then secondly, a number of people that are for that are going to be benefited.
It's not just the developers.
It's going to be all knowledge workers and all of beyond that.
So that's what we see.
You said the key components in supply are sold out to 2027, maybe 2028.
What does it mean for an entire industry to be sold out that far?
I don't know if this has ever happened before.
Do you guys recall?
I mean, remember in the internet days when we were doing massive build-out, the majority that was actually being put in the ground was speculative and was dark.
Remember the dark fiber in here?
Basically, every GPU that's being created is already pre-sold.
Yeah, we weren't quite there.
I mean, there was a lack of bandwidth like in the 98, 99 timeframe, but...
There wasn't that much real demand for it because there just weren't that many people on the Internet.
So it was a two-sided thing, and the companies were all rushing there and needed more bandwidth theoretically, but there weren't the users on the other side to consume it necessarily.
And then to really consume a lot of bandwidth, you have to do high-bandwidth things like video, which weren't really viable for a number of reasons that had nothing to do with how much bandwidth was in the data center.
It smelled similar, but it wasn't this.
This is like we're flat out and people are reselling GPUs for four times what they bought them for and this kind of thing.
And then we're also out of power and cooling.
And then on top of that, it's really hard to build because there's these incredible political headwinds going into it.
So it's really unprecedented in my career.
that we've had anything like this.
I want to give you a quick anecdote.
So I was talking to a CFO of a large company, a large public company, who had historically been very resistant about going into the cloud.
So they had a lot of servers.
And they were doing an inventory check, and they realized that the memory in their servers had increased so much it could fund the entire migration to the cloud.
So I just feel like we're in a very unusual situation.
That's right.
We're out of many things.
Power, cooling, memory.
GPUs, like you name it, we're out of it.
So the flagship conference for the industry is the one called Hot Chips is going on in Stanford.
And the leading memory runner said the demand they have today will take them three years of capacity to supply.
It's just today.
It's not even future demand.
So in terms of being about everything simultaneously, Is it because people just underestimated how good models would be, how useful they would be?
They just couldn't have seen the demand?
Well, I don't even think it's that.
I mean, this stuff came out of nowhere, right?
We're only four years into this.
So even if we had a perfect Oracle once it started working, I don't think we could have built the capacity.
We could have built the capacity.
There's no way.
And we're talking about...
Like chip cycles, which tend to be three to four years.
We're talking about breaking ground and building data centers, which is four to five years.
And connecting, breaking down and building them and having a power source.
So like you either have to build your own power or like usually both.
You've got to build your own power and have a power source, which is not easy.
Yeah, we have an industry that's used, if it's growing at 20, 30%, it's a great growth rate, right?
And it's being connected to...
And the AI software industry, it's typically just as the base.
So you can see the disconnect, right?
So it's just widening.
And so why didn't this fund exist, you know, five years ago or seven years ago?
Or why was it not a great category to invest in the same way, Brad?
Well, I would say we're probably, I'd like to think we're just in time, but, you know, we probably would have been well suited to have at least a couple of years ago.
I mean, I will say you could actually point on basically every epoch to an independent company that came up, right?
Clearly, they moved from a mainframe to the client server.
We saw a bunch of companies come up.
They moved to the internet.
That's why we've got Cisco and Juniper.
Even in the mega data centers, which, by the way, was largely driven by the incumbent cloud providers verticalizing, you saw the arising of Arista.
So there has been the ability to invest in...
you know, silicon on hardware, but it's been relatively minor because the change has been relatively minor, like one chip company, one switch company, where here everything is.
And so I agree with Ben.
We probably, you know, we probably could have started a little bit earlier, but the amount of change is so high now that it's just an obvious thing to do.
And the other thing is the demand for intelligence is so vertical with really no end in sight.
I mean, Because every company that's adopted it is growing very fast in its usage.
And then most companies haven't adopted it to a high degree.
And then consumers are just getting started.
And so it's going to probably, the demand for tokens is probably going to grow close to 1,000% a year, which you cannot grow supply that fat.
Like we're not.
Like the amount of just work we're going to have to do across the board to get to the point where we can grow like infra at that kind of rate is pretty vast.
So I think there's a lot of investing opportunity on the way.
And by the way, the other thing is like all the architectures of the hardware systems were built for a whole different era of computing.
And so more than...
Just we need more capacity.
We need capacity to build.
There's lots of opportunities to build different kinds of infrastructure.
Yeah.
They're all reaching the physics limits for what they were designed, right?
Like what Ben was talking about, copper and so on and so forth.
And you could go across every one of these categories and you could find, okay, this is the limit of this type of technology.
So now you got to...
some technical breakthroughs to get to the next one.
Yeah.
I want to dive deeper on the demand side for a second.
As we've moved from chatbots to reasoning to agents to multi-agents, each step has multiplied the number of tokens a single task takes up by increasing orders.
Nobody likes to use AI more than AI.
Why does that keep happening instead of leveling off?
And do you just see that happening indefinitely, just continuing to?
Well, so there's a couple of aspects of that.
The first one is, for sure right now, if you look at the way we're achieving scaling, the way we're doing it is through a lot of inference.
So through a lot of token, right?
If you think about what RL is, it's a lot of inference.
If you think about chain of thought, it's a lot of inference.
If you think about long-running agents, of course, it's a lot of inference.
And so that's just basically been one of the approaches that we've been using to scaling.
I think if you want to step back and say, kind of, what is the macro trend here?
It used to be when you built something, it was an engineering problem.
You could throw a bunch of engineers at it, and that doesn't scale, and that would have a natural law of engineering physics, which is what the Mythical Man Month came from.
And here it feels like it really is a resource limitation.
So whether it's tokens or not, we're pouring a ton of money into systems, and then those systems are producing a result.
And right now we're bottlenecked on those systems' ability to actually...
match the resources we're pouring into them.
And so I think like tokens right now is probably where we're on the scaling curve, but we don't have a natural regulator like engineering like we did before.
So I think we should expect this to continue and we have to build a supply to support it.
Yeah, like the similar way to think about it is any problem that you have can be solved with enough infrastructure, GPUs and power and money.
And so until we run out of problems, We're not going to run out of demand.
And that's the challenge.
AI's answer to getting better and better is to use more AI, right?
Inference is one basic building block path that it keeps using over and over and over again.
And so that's why these tokens multiply.
Yeah, even the autocatalytic effect.
So even the idea of using AI to create more AI, like creating a GPU kernel, of course, is just using more AI.
as part of the process.
So, again, one way that we think about it is in the past, money would come in, you have an engineering problem, we know that it takes two years, normally fails, you know, it's a national governor, and then you get the product on the other end.
There's nothing between the money going in and then the hardware, you know, creating intelligence.
And so now we're just limited by our ability to create supply.
It's a very, very different dynamic.
GPUs are now sold as gold.
Yeah, that's right.
As long as you have the money, the GPUs, and the data, you know, for the foreseeable future, you'll be able to scale these things.
And it's fascinating because, you know, over the last decade, it feels like there are so many, you know, people and the pervasive sentiment was there's too much money going to startups.
We're overfunding these startups.
There's too much money in venture capital.
Say more, Ben, about what that means because there used to be this...
sort of skepticism that the more money you put into the industry that, you know, there would be bigger outcomes.
And now, you know, we're saying at the off-site that there's, to some degree, the market is as big as we collectively contribute to it.
Yeah, so this is, look, the one thing we all knew in startup world is that if I have a two-year lead on you and you try and cast me by hiring a thousand engineers, you're going to wreck your company.
Like, that never works.
It's a mythical man month.
Nine women can't have a baby in a month.
Like, that's it.
Like, that never works.
Okay, now that works.
But it's not hiring 100,000 engineers.
It's taking $3 billion and, like, lighting up a magnificent cluster.
And then all of a sudden, you know, whatever, Grok can come out of nowhere and like, oh, all of a sudden it's real or Kimmy or what have you.
It's just like these leads, you can throw money at the problem and you can throw money at almost any problem and that works.
And so that is just completely different than anything we've ever lived through.
So we're, by the way, we're all psychologically adjusting to this.
The ChatGPT app has a billion weekly actives.
There's about 30 million developers who are using relatively a big portion of compute demands.
How do we think about compute demand needs now and in the future in light of what people are actually doing with that?
That's the progression, right?
So Chagik meeting was a casual act and it's coding for professionals, right?
Now, using coding, you now have built amazing tools for knowledge workers.
So that's the next frontier.
And now there are over a billion knowledge workers in Hawaii, right?
And with that, it's a long ways to go for that demand.
And by the way, the work that they do, all this work, grow in automation and so on, and then they get to the back office, which is all the agents.
So progressively, each of these things unlocks, I would say, an order of magnitude more than that.
And we're just at the start of this.
Well, and now you have GrokBot, which is kind of, you know, what happened with coding is kind of happening with all use of computer via GrokBot.
So we're in a whole nother wave of demand, and most certainly there's going to be more to come.
So it does seem quite unlimited at the moment.
And we haven't even gotten into embodied AI or robots, which are going to be another source of demand.
Mark is an expert, but my understanding is that Glockbot uses computer use, which is just like...
You don't have been sitting inside the computer type of way.
I literally used it over the weekend to update my credit card with a bunch of services that I'd been, like, lazy to do and cancel a bunch of subscriptions.
I mean, this is not coding or whatever.
This is true computer use.
All of a sudden, you create, like, lots of billion knowledge workplaces, except they're all sitting inside of the computer.
Doing what?
I do think that Mark Andreessen is right.
It's like the right analog here is like the steam engine or electricity.
In the following way, like, we've...
We've introduced this new thing that you can turn to work.
And there's some very obvious applications now.
But there's probably 30, 40 years of throwing computer problems.
I didn't think with a clear reward signal.
And we're just starting.
Like, we've got language and code.
That's it.
And just starting computer use.
But, like, what else are we looking at?
We're looking at in terms of science, materials, biology.
I mean, of course, creativity is a massive use.
And so, listen, we're on the very, very early part of a very long journey.
And we've...
remove this key bottleneck, which is, you know, traditional software engineering.
Now, of course, you know, bottlenecks will move and there'll be kind of more complexity elsewhere.
But I think we're very early in a very long run of throwing computer problems.
So let's expect, you know, this compute need to persist for decades.
Because we mentioned it, Marty, talk about GragBot, because we were talking at the offsite about how, you know, what struck you about it.
Obviously, we're involved in every possible way.
You could be involved.
But what...
What did you find so interesting about it?
So I think we've, as an industry, gone through kind of multiple realizations for how AI enters our lives, right?
And very early on, we're like, okay, well, you add AI to a product and it's like, whatever, it's like a search bar.
And then you kind of, you know, you do chat with it and it chats back.
So that's kind of the traditional way to do it.
And then OpenClock kind of showed up, and that was earlier in the year.
And with OpenClock, I said, okay, well, maybe like it just being like Google, but better.
Maybe that's not the full embodiment of it.
How about we'll have it be a standalone thing, but it'll be an extension of you, and it'll share your keys, and it'll know your passwords, and it'll just kind of do stuff that you would do, right?
So it's kind of an extension of you, but it's more like a human extension of you.
And then what I think GrokBot got really right is, no, how about it just actually an employee?
So now you have this thing that's an entity.
And it doesn't have, like, special access to your keys or whatever.
It has its own computer, and it has its own browser, and because these are the smartest models in the world, it can do whatever an employee can do.
And it's kind of interesting, because now, actually, if I want something done, my first thing I think is, like, well, can Grog Grog do it for me?
And often the answer is yes, even if it's something you wouldn't, you know, expect it.
So the obvious ones are, like, whatever.
manage my calendar, you know, like book a meeting.
But there's also non-obvious ones as well.
Like, so for example, I'll have it read through my email and do triage.
And I don't tell it how to do that, but it will know to check with me before actually doing the triage.
So like these things are sophisticated enough that you can give it relatively high level tasks and it'll do kind of, you know, like sophisticated things as a result.
Ben, I know you're thinking a lot about this and how this works in the organization.
You think a lot about culture, of course.
What are your thoughts here?
Well, I mean, I think if you just look at us, you know, it's like having a new kind of employee and there's going to be a lot of them.
And we have to just like, you know, with our, we spent many, many, many years figuring out how to work with our kind of regular human employees.
And now we've got these other kinds of employees.
You know, there is a learning curve with them.
So they can burn a lot of tokens and spend a lot of money and get nothing productive done.
They can forget stuff.
They can make stuff up.
You know, they can have good behavior.
They can have bad behavior.
Like humans.
They can create security problems.
So, like, there's all those aspects to it, but, like, they can also be, like, super-duper productive.
So I think figuring out how to integrate them in, have them work nicely with the people that they're working with, the actual humans, is all something that we're learning how to do.
I mean, I don't want to sit up here and say, I've cracked the code.
We've got this marvelous loop and the whole firm is just completely automated now and I'm going to slowly get rid of all the humans because I can.
Like, that's not at all where we are.
We're much more going like, okay, how do we make all our humans superhuman?
without wrecking the place because the bots got out of control.
Yeah, and it's interesting because we've tried a couple of different ways.
It's how best to get agents into the system, if you will.
And eventually, it was Martin's insight, just treat them as people and get it done.
And that's what we're doing.
That's turned out to be the most durable way of getting this thing going inside of an organization.
I want to go back to the supply side and go deeper into the bottlenecks.
We were talking about how, you know, in terms of the data centers, the chip architecture, system software, facilities themselves, that none of them were designed with AI in mind.
What would it look like for them to be designed with AI?
Like, what is sort of the mental model for thinking about what that could mean?
Yeah, so, I mean, if you start with a statement that you just said, hey, original model of infrastructure on any of these models, has to change.
You can go category by calling and see varied breaks, right?
And then you start unlocking the bottlenecks and the tonal of these things.
So eventually you have to get to a system where if you look at what an inference engine does, right?
It takes up a lot of memory.
It generates new tokens along with the computer.
And so you can just think about how do I optimize all of this?
What does the memory need to be?
What does a computer need to be?
How do they need to talk to each other?
How much power does each of them need?
And if they need all of this power, how do you code each of these, right?
And then how do you put the collections of these things together?
That is the exercise that's underway in the industry right now with a lot of the founders.
So they're breaking down the problem into its fundamental comports and saying, what is the exact nature of the computer that's getting done?
Okay, how's it going to be matrix multiplications?
How do I optimize my computer and that kind of a...
scenario.
They all need memory progressively to generate these tokens.
What is the best way of hierarchically arranging this memory, right?
And then how does the power consume?
I mean, then you've got to connect it together.
What are the ways of connecting it on the same chip but across chips and across data centers?
How much power does each of these data transmission take?
So you have to progressively break it all down and rebuild it from these fundamental building blocks.
And that's what we see underway and that's where we see that generally.
Let me give you an interesting mental model to think about how the landscapes change.
So today to build a frontier model costs, let's say, $3 to $5 billion, right?
And that's to train it.
And so the inference has to pay back at least that, of course, right?
You know, in order for any of this stuff to be viable, let's say two times that.
So let's say that now inference has to make $10 billion.
If you can save 20% of efficiency on that, that's $2 billion.
And you can easily build an ASIC for $2 billion, right?
So we've actually gotten to this interesting point in the industry where it actually makes sense to build an ASIC per model just because the amount of capital investment in that model.
And then unlike traditional software, traditional software has a lot of state and a lot of, you know, it's very dynamic.
These models are fixed.
The model weights are fixed.
And so...
We don't know if the world goes to per model ASICs, but it gives you a great mental model of how you would evolve the architecture to be far more bespoke for these massive capital investments we're doing.
I don't think in the history of the industry we've ever created a digital artifact with something like $5 billion that went directly into that artifact.
And so, you know, like this, I think, is going to put the greatest demands on hardware that we've ever seen.
To that end, rack power requirements are moving from roughly 5 to 10 kilowatts to 100 to 250 kilowatts.
Compute density is climbing something like 70x.
Cooling is moving from air to liquid as a requirement.
What are the investment opportunities as a result of this?
Well, first of all, when you get to that level of power per rack, AC power doesn't work anymore.
So, like, that's pretty wild.
thing.
So now you're into DC power, which, by the way, also requires its own cooling.
And is, like, by the way, super fucking dangerous.
Which is kind of ironic, because this was Edison promoted DC power by claiming how dangerous AC power was and demonstrating it by, like, electrocuting animals and things.
The horse, yeah.
So...
But he was right, but around his own kind of power, which is extremely powerful.
It's the good news.
So, you know, just starting with power, yeah, that's going to be, like, very, very different.
I think with cooling, and this gets into, so, yes, we're going air cooling to liquid cooling.
I think we're already at liquid cooling for any state-of-the-art data center.
Like, that's already kind of a done thing.
But it gets into...
okay, you know, given the political environment and so forth, you, like, liquid cooling isn't enough.
It's got to be eco-friendly liquid cooling.
And, you know, kind of, DC power is not enough.
It's got to be power that contributes to the power of society, not takes away from it.
And so you have data centers who have been behaving badly.
small percentage, actually, probably 10%, wasting a lot of water.
You know, not as much as pistachios or almonds and so forth as people demonstrate on the internet, but, like, they could be a lot more efficient with that.
And then there are ones that, you know, kind of are parasites of power and don't contribute power back.
I think all that's going to end.
It's going to have to end just because, like, we've kind of gone through a one-way door on that.
So that requires like a level of engineering that, you know, many haven't invested in yet.
So that's coming.
And then, you know, like if racks are that dense, there are other things that like the way the floors are designed have to support that kind of weight.
You know, that kind of thing is actually for real.
And I think that you just need a lot of everything.
And so there's going to be—also, the things are really loud.
So you have to build the data center with thicker walls or you're going to disturb the peace in the neighborhood, which is not going to be acceptable.
Like, I don't think any state's going to allow that.
And so a lot of the ways people have architected and designed the buildings themselves are already completely obsolete.
Like, once we get to Feynman— a much smaller percentage of the data centers that we have today work.
In fact, I mean, everybody talks about memory prices, but one of the fastest areas where prices are increasing is reinforced concrete.
The other thing that happens when these data centers are sending 800 walls to the WAC is it's become so dangerous, number one.
But secondly, we don't have enough.
electrical contractors that have the expertise to deal with 800 volts inside the data center because this is high voltage.
Only 2% of electrical electricians in the U.S.
have been certified on DC power.
So like that gives you an idea.
Now, Meta's got a whole program to train people up and so forth, which is great.
It's like a new job core where they train people for free to do this job.
You know, it's funny, AI is taking all the jobs.
AI is going to create a lot of new electricians.
Yeah, I think we're just doing something in space too.
The guys that own the big cloud data centers, they all are furiously experimenting with robots, right?
To do the work of assembling or putting servers into the data center, etc.
And so you will see that increasing as a result of the evolution in AI.
By the way, to be clear on the actual fund that we're raising, our focus is on computer science infrastructure.
So anything a model runs on, that's computer science, right?
So think, you know, chips, network, interconnect, storage, all the way down probably to the electricity.
Yeah.
And say more about the robotics arm in terms of what we'll be doing versus maybe American Dynamism or how to think about that.
Yeah, for sure.
You know, again, we think that any platform that AI will run on, like one of the great breakthroughs that AI does is it allows computers to interact with the physical world, right?
It can see, it can hear, it can talk, right?
And this means new platforms, right?
And the simplest way, people will say edge device, but that doesn't really mean anything, right?
I mean, it could be a mobile device, it could be a CDN, it could be a laptop, but it also could be an embodied, you know, device that goes around.
And so, again, we, as...
you know, as infrastructure-focused investors, don't do heavy regulated industries or more verticalized industries, but any sort of computer science platform that's going to push AI further out, we're quite interested in.
Yeah.
Going back to the data centers, by 2028, new data centers are going to need something like 44 gigawatts of additional power against maybe 25 gigawatts of expected grid additions.
Hold on, hold on.
We use that word gigawatt.
No, it's like, oh, we'll have 100 gigawatts.
Martin, what's a gigawatt?
I mean, how big is it?
It's multiple football fields.
I mean, it's massive.
It's 50,000 people.
And what is it power?
What do you mean, what is it power?
Like...
The equivalent is like 50,000 houses.
50,000 homes.
50,000 homes.
It's like I grew up in Flagstaff, Arizona, which is a town of 40,000 to 60,000 people, depending on the universities.
We have less than a gigawatt of power consumption.
So you can basically light up and air condition your entire town for a gigawatt.
Yeah, I mean, this is enormous.
He's just throwing them around.
No, but by the way, everybody talks about the gigawatt.
There's very few gigawatt data centers that are actually up.
I mean, we've got a long way to go.
Why can't utilities and hyperscalers just build faster?
Oh, there's so many things.
Well, there's, first of all, right now, you need humans to build them.
So there's just like the regular construction.
But much more than that, you need permits.
You need access to power that you can...
So you're either, you're doing a combination of, you've got to get access to power, which is a massive kind of regulatory bidding struggle.
There's very limited kind of amounts and things you can tap into in terms of natural gas, power grids, what have you.
But then you also have to build your own power.
And guess what?
We've got shortages of transformers and turbines and everything that goes into that.
It's just, you know, like you've got to get all that stuff.
This is not a software problem.
It's not just like a bunch of engineers, can't you like work weekends and that type of stuff.
Not that that works anyway, but there are real bottlenecks in this.
And these lead times are not that easy to compress.
And look, we have the best minds in the world trying to figure out how to compress them.
It's not easy.
It's not easy.
And the demand is not slowing down.
So we're already behind.
The demand is growing, you know, 10x a year right now.
And the supply just can't grow that fast.
By the way, it is so bad that right now, if we have new companies going for GPUs, it's often in Mexico or Australia or another country just because it is so difficult in the United States.
Yeah, we're creating huge, both job and long-term economic opportunity in other countries by banning data centers here.
I think, look, the right answer would be to set a standard where a data center contributes back to the community.
Like that power gets better, there's no noise, there's no water issue.
and it's adding jobs.
Like, that ought to be the standard.
And then everybody ought to be just held to that standard.
By the way, like, there are data centers that do that now.
Like, that's not a, you know, like a futuristic dream or something.
Energy rates have gone down, like, every year they're there.
And the reason is they provide their own power.
They give power to the state during the day.
And then at night...
They borrow power from the state when the state doesn't need it because the way power plants work is you're always generating peak capacity.
And since a data center has steady capacity during day and night and the city goes way up in the day and way down at night, that's a symbiotic relationship.
Zooming out, why do we think the, you know, we were batting around the name for a little bit.
Why do we think machine age?
is a compelling term for what we're doing here.
Well, listen, let me take it.
So the first one is, I think Ben's absolutely right.
Artificial intelligence was the wrong word.
Like, we shouldn't have called it.
It's machine intelligence.
Say more about that.
Why is that?
Because it's not how humans think necessarily, right?
I mean, it is a cache of how humans thought.
It is a collection of humans' thoughts.
But, like, to date, we don't know how to take a...
an AI with no knowledge and put it out in the world and have it reconstruct language, right?
Like, that's not what we've done, right?
We've built something that can learn off of everything we've already learned and then use that in a productive way.
And listen, AI is a general term that goes back 70 years in computer science formally that applies to many different things.
And of course, it's got a lot of baggage, either from science fiction or from, you know, Nick Bostrom who wrote about it or whatever.
And so...
So the first one is just an acknowledgement, like this really is machine intelligence.
And then you want to emphasize the machine part of it.
I mean, there's kind of this deep irony, and this is from the software's eating the world people, that you've really come to a place where you pour money into something and then you're limited by the actual machines below it.
And so I think it is kind of a nod to like the hardware component is so significant in this wave, and we want to acknowledge that.
Yeah, I think that's what's going to create the next breakthroughs.
the quality of the machines are not made.
So that's basically losing to the name.
It's also a cool name.
It sounds good.
Futuristic.
Given how much has been spent on AI infrastructure to date and how capex intensive these businesses can be, are we past the point where new companies can break in at sort of material levels?
Why not incumbents like NVIDIA, CoreEV, etc.
just take the lion's share of these markets?
They're all doing.
Well, there's no question about it, right?
But to our discussion earlier, when you need fundamentally new innovations to keep the growth continuing or the pace of improvement continuing, whether it's tokens per second, per dollar, or tokens per watt, tokens per rack, right?
Or power.
You take any metric.
If you want to have a 10x on those metrics, you got to have new innovation.
And new innovation traditionally comes from Boozian founders thinking about solving the problem from first principles in a different way, right?
And that's what's needed here for the next jump in innovation.
I mean, this is the law of markets, right?
I mean, let's assume that the existing silicon incumbents are multi-trillion dollars in market cap, which is absolutely the case.
Even 5% of that is a massive private company.
massive private company, right?
We're talking, you know, annual.
And you could say, well, but NVIDIA could do that.
They could, but why would they if they're focused on things that are in the 90%, which is also driving the same amount of growth?
And you always ask these questions.
We ask these questions during the cloud days, right?
Like, well, wouldn't Amazon did this?
You would ask these questions during the Microsoft days.
Why wouldn't Microsoft do this?
There's a very natural law of markets is once you get to a certain scale, there's tremendous opportunity for innovation at the margins.
Yeah, there's a funny quote from our partner, Alex Rampell.
He had this startup called TrialPay.
He was trying to sell it to or sell his services to Meta and then Facebook.
And Dan Rose, who was the head of CorpDev at the time, said, Alex, that's great.
It sounds like you can collect a lot of silver bricks, but I'm like, I have so many gold bricks, I can't even pick them all up.
So the last thing I'm doing is looking at a silver brick, and I think NVIDIA is in that position.
100%.
Yeah.
We were talking about, as it relates to the model providers, that if you're, you know, in the sweet spot of what OpenAI Aranthropic can do, you know, one of their main sort of interest areas, that might be a tough place to be, but anything outside of those maybe, you know, three to five areas might, you know.
As markets expand, they fragment, right?
And it happens all the time.
And remember, in the early days of Ford, there was the 1913, there was the Rouge River plant.
Literally, this was like made cars.
It went like water, coal, and rubber trees, and out came cars.
By the way, he bought a whole rubber tree plantation in the Amazon jungle.
And there's a great book called Fordlandia, because he wanted to own the complete vertical thing, where he created this city called Fordlandia in the Amazon jungle, which was all Americanized, bandstands and ice cream and all this kind of stuff.
And it actually worked for a while until he made people show up to things on time, and then they were like, screw this, get the fuck out of here.
So now if you look at the car industry, of course, there's multiple levels of supplier and there's a bunch of companies.
And this always happens.
So, you know, as markets expand, they fragment.
And then once that...
growth slows down, they tend to consolidate.
The consolidation can either be acquisition or it can be like new challengers rise up.
And that is the everlasting cycle of private markets.
Yeah, because the use cases are multiplying.
And there's no way, like if you're the biggest company, you can get to the biggest use cases.
But there's so many use cases.
And all, as Martin was saying, very valuable use cases that it's just very hard to get to in a great way.
Yeah, even in France, it's just one simple architecture.
Right.
No longer.
It's just like, it's so complex now.
So it's inevitable that you can optimize things in a different way.
By the way, here's a very interesting thing.
Like people don't, often don't understand that like margins kind of fell out of the standard way of doing the technology with software.
Right.
Like it wasn't really a technology problem.
Like once you got the business working, you tended to have pretty good margins because that's just kind of how software works.
Certainly when you shift it, but even as a service.
And that's not necessarily the case with AI.
So we may actually be entering an era where the optimization in the hardware is absolutely meaningful to the upside of the business in a way that we haven't seen in the past.
So there's a lot of opportunity here.
Let's get deeper in talking about the types of companies we'll be investing in.
Maybe we could start by either illustrating the subsectors or if we can talk about a few or a couple investments that we've made.
And there's some that haven't been announced yet.
Ravi, do you want to take this down?
Yeah.
I mean, the subsectors, as we've been talking about a while, is every one of these categories, right?
The obvious ones are compute chips.
But these days, it's not enough to build a chip.
You need to build a full system, right?
And then, therefore, what goes into the system?
There's potentially memory innovation.
There is potentially networking innovation.
There is potentially power chips.
And so on and so forth.
So each one of these categories are categories where...
you can see public company-style companies emerging.
And those are all the things that we are looking into.
And then once you put it all together, there's a lot of software around it to automate all of these things, to manage these fleets, and so on and so forth.
So that is another important area.
So these things keep building on each other.
Every one of these categories is important.
Talk about what's different about these kinds of companies from the usual companies.
I mean, one thing you can tell from the companies we announced is their first rounds have been massive.
you know, hundreds of millions.
Is it a different kind of founder?
What else is different as we think about just the practice of, you know, building and investing in these kinds of businesses relative to our traditional software?
Well, I think the big thing is you hit on one of the big things, which is a lot of money goes in before they get to a product.
And that's just kind of the nature of it.
Now, that's true on big models too, but I would say that's a little more of a known path, whereas this has got...
a little more risk and a little more money than some of the other things that we've done.
And, you know, look, a lot of the chip founders are here from the past.
You know, like the guys who know how to make memory, they're not young.
So it's, you know, that part is different too, but it's kind of exciting, you know.
Yeah, the other thing about these founders, They've all got to be systems founders.
So what I mean by that is you can't just be a researcher or a great computer scientist, right?
You've got to be able to architect and design the chip or the system, whatever it is.
Then you've got to think about how is this thing going to actually get manufactured, right?
Who's going to be supplying this?
And a whole bunch of these downstream things, which normally if you're building software, you don't have to think about all of these things.
So really, the best founders, and of course, Jensen is the Michael Jordan of this, right?
They think the entire ecosystem right from the get-go, before they start designing the chain, right?
Because of the nature of the bottlenecks and all these things that have to come together.
So that's a big characteristic that is different.
There's two environmental factors that are important to you.
The first one is the labs are so desperate.
that they will engage with startups.
And so, like, we actually have quite a bit of signal early on because they're, you know, the labs are inking deals with companies before they actually have hardware available.
And that's a big, big shift than, you know, five years ago, right?
Like, you just didn't go and, you know, sell your kind of janky hardware thing to Google or whatever.
So that's a shift.
The second one is the capital availability has loosened up a lot.
I think there's general consensus that...
that, you know, it is the time to reshape this stuff.
And so follow-on rounds, there's a lot of capital available, which, you know, of course, you want to be investing in two areas where there's capital available.
And so the atmospherics are also just different.
Patrick Carlson, you know, remarked a few years ago, said, hey, it feels like there's less younger founders today in the way that Zuck, you know, in college, building an ex-Facebook or Gates, you know, in the same way with Microsoft.
And of course, you know, the Michael Trulls of the world, there's still some young founders building iconic companies, but it does seem, you know, to your point, that there's more older founders building these companies or less 20-year-olds.
I'm curious if it resonates and why.
Well, I think it's Ragu's point that if you're building something that has like a very complicated supply chain, has to manufacture things, and is technically complicated, that, you know, some experience helps.
And, you know, if you look at Elon or Travis Kalanick, their companies when they were young were software companies.
It wasn't until they got like a lot, even those guys, the best guys, needed some experience in building a company, building technology and so forth to kind of graduate to the much more kind of complicated or...
I would say, elaborate domains.
There's just much more, there are many more moving parts in these things.
And so, look, when you're learning how to build a company, it's hard enough if you completely understand the product.
If you don't completely understand the product and have to learn it while you build the company, that's just such a steep learning curve for a brand new entrepreneur.
So I think that what we're seeing is you see, Michael, on the one hand, who is a very young guy, brilliant, but what he built was kind of a pure software AI thing.
And then on the other end, you have like an Elon or a Travis who can, who's got enough experience.
I think Michael could probably do that, you know, 10 years from now, but today that would have been hard.
It's important to remember, like it's been defocused by the entire industry and academia for the last 20 years, right?
It just, it just hasn't been the same opportunity.
Like it's been there.
But like it's never been a growth area.
The growth areas have been, you know, software, networking, things like that.
And so I also think we just have a positive people coming out of the universities or having experience at large companies that have done this.
I mean, there's just not that many.
Like you don't go intern and like build a chip.
So, but a lot of that's changing now.
Like, listen, we're going to create a whole generation of...
of founders that come from these new companies that will know how to do this and they'll be hired in much more junior.
I would say actually one of the greatest legacies of Elon towards this is, of course, he's created these great companies, but the amount of entrepreneurs that have come out of SpaceX that are changing the entire industrial complex may be even greater legacy than the companies themselves.
And I think we're going to see the same thing for computer science and hardware.
As a matter of fact, one of our investments was told about it.
two founders in their communities, but if you go walk through their offices, you see the experienced people as well.
So it's ideal combination here.
Yeah, yeah.
It doesn't necessarily have to be the founder with experience, but that founder better be able to tap into that experience in a real way.
Get the people with it, yeah.
Yeah, yeah.
Well, and then be able to work with them and they have to be good and all these kinds of things.
It's complicated.
Speaking of experience, this is a...
a big new fund we're launching and there's no new GPs.
We're sort of collecting.
It's because you guys have a lot of experience and the rest of the group, you know, in this field that has been kind of latent and dormant.
Well, it's kind of funny.
I think we almost had to be warned against it almost just because like our backgrounds are from us hard.
And I think the reason that we needed a reminder is because all of us have spent so much in our careers, systems and hardware, we're kind of drawn to that.
And so, listen, we've been clearly invested.
in Harvard other years, right?
We're in SpaceX, we're in Adderall.
These are very early checks.
We're in astronauts, we're in Waymo.
You know, so even early on, we did a number of those investments.
But like, you know, this is because it's so much in our DNA.
And so I don't think this is necessary to increase the team of competencies just for us.
If this fund does what we think it will do, how do we see the world changing or looking like in five to 10 years?
Well, you know, hopefully...
America wins in the infrastructure game.
And we have lots of, like, super eco-friendly, efficient data centers out there and lots and lots and abundance of chips and abundance of memory and abundance of power.
And, you know, that would be awesome.
And I think, like, we, it goes back to, like, we really think America is a special place.
We're important not only to everybody here, but anybody in the world who wants to kind of make a contribution and do something bigger than themselves.
It's kind of the best place to come with nothing and do something profound.
So we'd like to keep that going.
And I think that doesn't continue to go if we lose our lead in technology.
I think we'll be in another era and it'll be another country.
And maybe they have a different set of values around that.
Thanks, guys.
Nice to wrap.
Great.
Appreciate you.
It was fun.
Our team, Ben, our group.
Thank you.
Ben, too.
Thank you.
Thanks for listening to this episode of the A16Z podcast.
If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review, and share it with your friends and family.
For more episodes, go to YouTube, Apple Podcasts, and Spotify.
Follow us on X at A16Z and subscribe to our Substack at A16Z.substack.com.
Thanks again for listening.
and I'll see you in the next episode.
As a reminder, the content here is for informational purposes only.
Should not be taken as legal business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund.
Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast.
For more details, including a link to our investments, please see a16z.com forward slash disclosures.