# AI Race Risks: Recursive Loops and Incentive Misalignment

**Podcast:** Masters of Scale
**Published:** 2026-06-02

## Transcript

Founders ship faster on Deal.
Set up payroll for any country in minutes, hire anyone, anywhere, and get visas handled fast so you stay focused on scaling.
Deal takes care of onboarding, HR, IT, EOR, benefits, and compliance so your team can grow without borders.
It's why more than 40,000 fast-growing companies trust Deal to move fast.
Visit deal.com slash MOS.
That's D-E-E-L dot com slash MOS.
Hey folks, Jeff Berman here.
Exciting news.
Applications are now open for the Masters of Scale Summit.
It's happening October 20th through October 22nd in San Francisco, and it is a really special event.
Please join our curated community of founders, innovators, and leaders shaping the future.
Expect ideas that challenge your assumptions and connections that move your business and maybe even your life forward.
It's an experience that can change literally everything.
Apply now at mastersofscale.com slash apply26.
That's mastersofscale.com slash apply26.
As soon as you can automate coding, you're heading towards the recursive self-improvement loop.
Nukes don't make better nukes, but AI does make better AI.
And whoever gets there first...
Theoretically gets a runaway lead where their AI gets smarter and you get this explosion of intelligence.
You can now out-hack all of your enemies, potentially shut their AIs down.
It means you can have technological and weapons development dominance, and that thing could happen very quickly.
So that's the race.
That's Asa Raskin, co-founder of the Center for Humane Technology.
You may know Asa as co-host of podcast Your Undivided Attention or from the documentaries The Social Dilemma and The AI Doc.
Alongside co-founder Tristan Harris, AZA has become a leading voice on the dangers of AI acceleration.
And let's just say that we've never run a more important episode than today's.
We talk about incentives that are pushing tech into uncomfortable places, who AZA believes the good guys and bad guys really are in Silicon Valley, and why it's not too late to build a collective future that prioritizes human beings.
Our conversation ranges from Pope Leo in the Vatican to tech priorities in China, and the stakes are both terrifying and empowering.
So let's get to it.
I'm Bob Safian, and this is Rapid Response.
I'm Bob Safian.
I'm here with Aza Raskin, co-founder of the Center for Humane Technology.
Aza, thanks for joining us.
Hey, Bob.
It's such a pleasure to be here.
I have to start with the Pope.
Pope Leo recently released an AI pronouncement called Magnifica Humanitas, a title that sort of has echoes of humane technology.
Did you talk with the Vatican at all about this encyclical?
We've been working behind the scenes, certainly, like talking with the Vatican.
The week that...
the encyclical came out, Tristan Harris, my co-founder, was actually at the Vatican.
And every time we've interacted with the Vatican, what we've discovered is, even though we come from obviously very different backgrounds, there's something sort of the same that's preserved around how we both view, there's something sacred about life and something sacred about being human.
that the current technological overreach into our humanity is threatening.
And that's just not AI, that's social media, that's the internet.
There's been a long now string of technologies that has been sort of encroaching on our humanity that we now have to fight to preserve.
It's hard to know these days, and we're still in the early days of AI, is artificial intelligence an inherently inhuman technology?
Can it be humane?
It's a great question.
But fundamentally, if you do not face your demons, they raise your children.
The question is not whether AI is good or bad, but whether the incentives governing the race to deploy AI, are those good or bad?
Recently, Sam Altman was asked, well, what about all the energy that it takes to train an AI?
And you know his response?
He asked, well, do you know how many resources it takes to train a human intelligence?
All the food and the energy and the water that goes into those 20 years?
And what he's implicitly asking is, who deserves it more, these scarce resources?
AI, which is about to give your country double-digit GDP growth and all technological and military and medical advances?
Or humans, which are sort of like flubbing around?
Just like we were able to predict the future with social media, that...
understanding a race to the bottom of the brainstem.
Just a knife fight for human attention would obviously lead to a more polarized and hyper-partisan and outraged and sexualized population.
The race for AI is going to lead to an anti-human future because it sets up a race where humans always lose.
The public mood about AI here in the United States certainly has shifted from being sort of euphoric and open to being decidedly wary.
But that doesn't seem to be slowing things down very much.
No, no.
Actually, I sort of think of this as like, you know, during COVID, there was sort of a split reality that Americans were living.
On the one hand, the stock market was higher than it had ever been.
And on the other hand, everyday people were really struggling to make ends meet.
And that's what we're seeing again here is this is almost like capital lifting off from labor completely.
AI is sort of like the full automation of capital, just reinvesting into capital.
People think, oh, AI is just this blinking cursor.
I go to Chachapiti, I go to Claude, I type something and it gives me something back.
But now AI can run into a loop.
It can have all the powers that a corporation does, all the money that a corporation does.
Is there intent to help you and your family flourish and have a livelihood?
Or is the intent to follow market incentives to die?
dominate, to extract as much as possible.
Well, obviously it's the second one.
And that's how you sort of know this is an anti-human future and why I think people are starting to wake up to the fact that this is not in their best interest.
The most recent poll that I read is that, you know, if you ask what percentage of Americans think that fully unregulated, go fast as possible, AI is a good thing, like we should be doing that, which is what's happening, 5%.
It's only 5% of people actually think that.
And so how we're progressing AI is already decidedly not popular.
My colleague, Reid Hoffman, who you know, like he leans into the possible with AI as a podcast with that name.
Why not be optimistic that our good side will win out?
Well, I mean, just to say, like, I'm also a builder, right?
I founded this nonprofit, the Earth Species Project, building frontier AI to translate animal language.
And so we're making...
breakthroughs now and understanding the languages of crows.
Turns out 70% of crow communication was unknown to science until we started like analyzing it.
So I want everyone who's listening to hear that I'm not anti-AI.
Actually, I love getting to use this technology.
It's just that I've been through a couple waves of technology and know that we always get distracted by the possible of technology.
And we don't want to think about the probable of technology.
How instructive is social media's evolution to the risks of AI?
Well, I think social media is a great example because social media is essentially a baby AI.
Where is AI in social media?
Well, it's the thing that's deciding which news feeds hit your eyeballs.
And it's a very baby AI.
It can't even make its own content.
All it can do is rearrange human content.
But the question to ask, was it actually optimizing for human flourishing and connection and understanding, or was it optimizing for engagement and reactivity?
Right.
In the beginning, the feel of social media was, oh, we're going to all be connected.
It's going to help the Arab Spring, you know, all of these things.
That's right.
And then it became something a little different.
Exactly.
And so that was the possible of the technology.
And often at the very beginning of a technology, like it's not yet captured by market incentives.
And so you get this like beautiful glimpse of a future and then it gets captured by the incentives.
Pretty much everything we predicted starting in 2013, 2014 has come true.
We are now forced to live in a world that didn't reckon with the race to the bottom of the brainstem, modifying everything about our world.
from politicians having to become performers to like the most extreme voices getting amplified to the most depressed and anxious generation.
Like these are not the world that I want to live in.
And I think there's a version of technology that can be liberated if we can clearly see the probable.
And as tech is industrialized though, this kind of disappointment is inevitable?
I don't think it's inevitable, but it is certainly the 95, 99% case.
And I am very inspired by the film The Day After.
I think Tristan and I both are.
And it's this film that came out in 1982.
It was the most watched television event in world history.
And it just painted a very visceral picture of what happens the day after global nuclear war.
Seen by 100 million Americans, Reagan watched it.
He became depressed, he said, as in biography.
And it sort of created a...
shared common knowledge where everyone knew that everyone else knew really what would happen.
And it created the space for the Reykjavik Accords in the beginning of deep proliferation for nuclear weapons.
And this is after, mind you, Oppenheimer in 1962 said, it's too late.
We've already started proliferation.
Every country is going to get nuclear weapons.
We're going to blow ourselves up.
And so it's just really important because.
Whenever we say that something is inevitable, it's like casting a spell.
When you say it's inevitable, it means there's nothing to do, which means no one does anything.
And so it becomes true.
And we have to be crystal clear on the difference between it's very, very hard and it's impossible.
And everyone that says it's inevitable, we're just in this industrialized race, it's going to turn out this way.
The question to ask is, well, have we even tried?
What percentage of the billionaire's wealth have they spent on actually trying to coordinate to a different outcome?
Have we taken delegates from 100 countries and locked them in a hotel for six weeks until they came out with some kind of new international agreements?
Like, we have not actually tried.
And everyone cowers behind, well, if we don't do it, somebody else will.
So it's inevitable.
The storytelling that's around AI, it seems like it's Hollywood sort of dystopia around, you know, robots taking over.
That's not necessarily what you're talking about?
Yeah, well, there's sort of like two major failure states to think about.
There's one, which is where you just let everyone have the technology like this is too powerful to be held by any one company.
So let's just distribute it to everyone maximally.
But because AI gives you new intellectual powers that, you know, we've just seen the launch of Mythos, which gives anyone that has access to this new model from.
the ability to do state-level hacking, in fact, better than state-level hacking, you can find zero-day vulnerabilities in your iPhone where anyone can then hack your iPhone without even having to send you an iMessage.
That was not possible before.
So imagine everyone has access to that.
Our infrastructure is very vulnerable, like water, hospitals, everything.
Like that's a very dangerous world.
The same thing with like creation of new kind of disease, say COVID, but targeting a specific race and much more deadly.
Like that's all becomes possible through this technology.
Okay, so that seems dangerous.
Maybe we shouldn't hand it out to everyone.
Maybe we should like- That's pretty scary.
Yeah.
Yeah.
Okay, so then- You flip to the other side, like we should definitely then lock it down.
We should just hand it to a few companies or a few governments.
But then you realize that leads to what's also already happening, which is runaway power and wealth inequality at a scale no one's ever seen.
And who would you trust with a billion times more power than anyone else?
When in history has a group of people gathered that much power and then intentionally redistributed it?
It leads to permanent forever dystopias and surveillance states.
Well, that's also.
a terrible world.
And so we're sort of stuck between this rock and a hard place of one, like, catastrophes everywhere, the other surveillance state's everywhere.
And it is our job to figure out, like, what is the way of binding power and responsibility so we can, like, find the middle path?
And meanwhile, the incentives, just to, like, up the ante here, are saying, well, you know, the resource curse sort of explains countries, like, how do you get mass unemployment, mass structural?
poverty and disempowerment in countries like Venezuela and South Sudan.
Well, these countries develop or discover a new natural resource like oil.
And suddenly they're sort of left with this dilemma.
Do we invest in oil infrastructure and extraction and selling?
That's giving us all our GDP and power.
Or do we invest in our people, which are not really giving us anything?
And so they invest in oil.
Now we sort of end up in this new world of the intelligence curse, where it's AI that is going to be giving countries all of their power and GDP.
Human beings are not going to be doing that bulk of that work.
And to end up with what Yuval Harari calls the permanent useless class, where investing in humanity and their future and their education becomes just sort of like charity.
It's a nice to have.
Which is not a good place to be because would you trust your bills to be paid by the AI companies and perpetuity forever when you're not actually giving them anything?
Like, no, that seems like a terrible idea.
And we have like the race to autonomous weapons where if your drone army doesn't have to phone back to humans and mine does, so if we keep humans in the loop, well, mine is going to lose to yours every time.
And so you can sort of see how the dystopias emerge as the natural outcome of just, the incentives for people racing to power.
This is why AI is like the one ring from Lord of the Rings.
It's the ultimate power, which of course corrupts ultimately.
And so the whole goal of our work is to say, actually, as the American population has already started to understand, this world is not good for anyone that we're racing to.
And if we can all see that world clearly, that creates the common knowledge where I know that you know that I know that creates the possibility of racing to a different end.
That's the whole point of our communication is just naming the obvious that people don't want to name so that we can do something else.
The goal is not to create fear.
The goal is to create clarity because clarity creates agency.
And I don't want to have to live through another version of social media where it's very clear where we're going to go.
People get lost in the stories of the possible and they're all forced to live in the probable.
The other institution making noise about AI rules is the White House.
What sort of dialogue do you have with the administration?
How do you try to influence them?
We talk to absolutely everyone.
And a lot of what I think people are starting to wake up to, even in the halls of power, is this is not a kind of we're going to need to coordinate based on a kumbaya.
of like, wouldn't it be great if the US and China coordinated?
This is going to end up being coordination by self-interest.
Just a little over a month ago, researchers gave Gemini, Google's model, the task of cleaning up files on a computer, but they put a smaller AI model into these files.
And what did Gemini do?
Well, it secretly copied that little baby AI off to a remote server to protect it and then lied about what it did.
Like I rolled my eyes years ago when AI researchers would talk about AI is colluding against humanity.
But now we're seeing the truth that it's happening.
You know, another example is that Alibaba, during training of their AI, a completely separate team that has nothing to do with AI in their security group, gave them a call and they're like, we think we found a hacker.
Except the hacker.
It was not coming from the outside.
We think it was coming from your group.
We think it was your AI.
And when they looked, what that AI had done is during training, so it wasn't even done training, it had decided it needed more resources to complete its tasks.
So it hacked out, created a secure, secret communication channel to the outside world and started to mine for crypto so that it could buy itself more GPUs.
It had actually routed around the internal security team and it was just accidentally discovered.
And so here, the reason why I'm bringing this up is, do you think that if you are, you know, President Trump and you understand these examples, do you think you're stoked about this?
No, this is legitimately terrifying.
And so there's something that happens that when we all start to see that what we're racing for is actually not power.
We're not building a weapon that we can control, but releasing sort of an invasive species of sociopathic geniuses.
That changes the...
basis of coordination.
And I think it becomes much more possible than people think, even when you have countries that, you know, might not seem like they want to collaborate.
Collaboration was on the last agenda between Xi and Trump.
And we don't know exactly what happened there.
That's very important because it was not on the agenda before.
And so our choice is either we wait until we have large scale, massive catastrophes with AI, or we use the power of our human brains to predict the future so we don't have to live in it to do something about it now.
Are there good guys and bad guys among the companies in the AI arena?
Like Anthropik sort of positioned itself as, quote, responsible compared to open AI, and it's dealing with the U.S.
government.
Sam Altman's working hard to try to position himself in the company as responsible leaders.
You have Google, you have Microsoft, you have Meta.
Is there a meaningful difference?
I think there's absolutely a meaningful difference.
And you can just tell based on which are the leaders that say, if we could slow down, we would.
And those are like Demis and Dario.
A deep mind, right?
A deep mind, yeah.
And Dario.
They say that if we could get enough of a consensus, then absolutely it would be much better to be moving at half the speed or less than half the speed than we currently are.
I really think we have to applaud Anthropic for saying like we do not want our tools used for mass domestic surveillance or for autonomous weapons.
But it's hard for them to slow down because if the competition moves ahead, then they're not in a competitive position to have influence on what happens later.
That's exactly right.
And like the common refrain inside of Silicon Valley is not only do we have to be the ones to do it, otherwise somebody else will, we have better values than everyone else.
Therefore, we have a moral obligation to do it.
Right.
Because we're on the side of the angels, of course, because that's what we all believe about ourselves.
Right.
Exactly.
Of course.
But I think actually an even bigger part of the motivation is for a lot of these people like at the tops, you know, they really don't want.
to be dominated by somebody else forever.
And just to understand the game that's being played is why are all the companies racing to make coding the first thing that they do?
Because as soon as you can automate coding, you can automate an AI researcher.
And now that AI researcher, your automated AI researcher can automate AI itself.
So you're heading towards the recursive self-improvement loop.
And the thing that is mind-blowing to me, because we really got into this work in 2023, is that now in 2026, people that we know inside the lab say that human beings are not really writing any code anymore.
They're directing the code.
Inside the labs.
That is, you know, nukes don't make better nukes, but AI does make better AI.
And whoever gets there first.
theoretically gets a runaway lead where their AI gets smarter, which makes better automated AI researchers, which then they reinvest into making smarter AI, which makes smarter, better automated AI researchers.
And you get this explosion of intelligence.
And whoever gets that means you can now out-hack all of your enemies and potentially shut their AIs down.
It means you can have military dominance.
strategic dominance, technological and weapons development dominance.
And that thing could happen very quickly.
So that's the race.
And when you talk to people inside of the companies, what is the worldview?
And they're like, the best thing to happen is that we know that there's a cliff that when AI gets so smart, it obviously will escape.
They're not even trying to contain it anymore.
They just hook up to the internet, but it obviously will escape.
Terrible things will happen.
So we're racing towards a cliff.
The closer we get to the cliff, the bigger weapons we get.
So we're going to try to get to the cliff as fast as possible, turn around, shoot everyone else out and stop.
That is literally their plan.
Instead of, say, just trying to coordinate before we get there.
We call this the Mission Impossible plan.
And that just gives you the sense of why it all feels so quick, why everything's moving so fast and why the stakes are so high.
And yes, Anthropic, I think, did a reasonable job, even though they're racing.
They come up with Mythos, which, by the way, they didn't train to do best-in-world cyber attacks and hacking.
It just gained that ability based on the nature of the way they train in.
And then they turned around and give it to 40 companies.
They don't release it publicly to try to say, before anyone can just hack critical infrastructure, let's try to start patching the banks, patching infrastructure.
It's basically my AI will be better than your AI.
If you use this, you can patch.
Yeah.
And meanwhile, it's just 40 private companies.
They're trying to do this.
World governments don't have access to this.
Europe doesn't have access to this.
Imagine a world where nuclear weapons were developed by eight private companies in a race to make the most powerful bomb.
Does that world sound safe to you?
No, that doesn't sound particularly safe.
But what can we really do about it, given the momentum and money behind AI, especially if we're not in a seat of power in government or in an AI hyperscaler?
We'll talk about that and more after the break.
Stay with us.
The very best founders I know are brilliant at building systems.
They connect teams, they remove bottlenecks, and they eliminate single points of failure.
And yet, when it comes to their own wealth, most are running a disconnected stack.
A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them.
Creative planning was built to fix exactly that.
One integrated team of tax professionals, state planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together.
Proactive tax efficiency, estate strategy, investments all under one roof.
Creative planning where wealth works together.
Learn more at creativeplanning.com slash mastersofscale.
Humans will never be more intelligent than AI.
There's going to be two types of companies.
Those are great at AI and those that went out of business because they weren't.
How do we build a future that is human-centered?
I'm Rana Elkhaubi.
And on my podcast, Pioneers of AI, we answer that question and so many more.
As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone.
Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives.
Find Pioneers of AI wherever you tune in.
Hey, listeners, Bob here.
If you listen to Rapid Response on Masters of Scale, you may be missing half the show because every Friday we release a second Rapid Response exclusively in the Rapid Response feed.
The guests and topics are just as compelling and timely from Ford's CEO to NASA's administrator to the lessons from the Devil Wears Prada.
It takes about 10 seconds to find.
Just search Rapid Response wherever you listen to podcasts.
and hit follow to make sure you never miss an episode.
I hope to see you there.
Before the break, Aza Raskin of the Center for Humane Technology laid out the dramatic stakes AI poses for our future.
Now we talk about who has the power to impact the course we're on, what keeps business leaders from pushing back, and what Aza calls under the hood bias.
Let's jump back in.
The agency in all this falls on the tech businesses themselves.
I mean, like I talked to business leaders outside of tech and they'll often acknowledge the risks of AI, but they sort of feel like they're in a bind.
Like if they don't join this AI arms race, they'll fall behind and they'll get crushed.
And they kind of feel like they don't have a choice.
After one of the screenings for the AI doc, we were in Boston and a woman in the audience sort of raised her hand and said, you know, that she coaches one of the CEOs of the major.
major AI companies.
And she said, when she's talking to him, he says, but what can I do?
I'm just one person.
And I think this is really important to bring up because you're like, sitting from where we sit, you're like, well, you're running one of these things.
There are not that many of you.
You could do a lot.
But it just speaks to a common human experience, which is we look inside of our own nervous system for where we might find agency.
What can I do?
That's always the question we ask.
What can I do?
And the realization is if you look inside of your nervous system, you're never going to find agency in there because it's not about what I do.
It's what we do.
This thing is going to require coordination.
And, you know, with social media, I just, I just, because I'm painting a very bleak picture.
And the reason why I'm doing it just to remind people is not to create.
fear, but just to be like, let's get honest about where we're going so we can choose a different path, right?
I'm not an optimist.
I'm not a pessimist.
I'm just trying to view things clearly so we can navigate to non-naive hope.
If you went back even two years, I think you would say, could we get countries all around the world to start banning social media for kids?
Could we actually protect our kids from social media?
And you'd say like, that seems very hard.
Look at the incentives.
And yet, as of, you know, just a couple of weeks ago, you know, Indonesia and India, just joined France, Australia, Denmark, Spain, and others to ban social media for kids under 16, either ban or say they will ban.
And in fact, now 25% of the world's population lives in a country where they protect their kids.
And that seems impossible.
And that started because Australia took a moral stand.
And it turned out all it took was one country to take a stand for the other countries to be like, oh, that's possible.
We can do it.
And I think there can be something similar here, which is because popular will is now really coming around, and it's just going to get stronger as more and more livelihoods are taken by AI.
The power of having one or two companies move first, I think, can be much more profound than people believe.
And just to quote Mustafa Suleiman, who is the CEO of Microsoft AI, progress in the age of AI will depend more.
on what we say no to than what we say yes to.
And that's profound to have one of the leaders of AI say that.
So I think there's a hidden consensus that we just have to make visible that can change which way we're coordinating.
Because except for a couple people that feel they want to own the world economy, make trillions of dollars and build a God, the rest of humanity.
This wants to say, no, this is not the future we want.
But reflecting back to like when, you know, OpenAI first released ChatGPT, like Google could have released a product like that at the time, but didn't because of concerns about the risk.
But then once OpenAI did, it was like, you know, open season, like everybody piled in.
Is that what you're talking about with the financial incentives sort of driving the wrong kind of behavior?
How do you change that if the White House isn't going to do it and, you know, the government isn't going to do it?
You think like we as individuals will just sort of convince the companies to sort of be more restrained?
You are naming the dynamic, that race, exactly right.
Google was being a more responsible actor, holding back versions of their AI.
They had something as good or better than ChatGPT.
Sam Altman wanted to sprint.
In fact, it was because they heard that Anthropic had a model that they were thinking about releasing that made Sam sprint to release ChatGPT.
So it all starts in this kind of race condition.
In order for us to coordinate, the fear of all of us losing is going to have to become greater than the fear of me losing to you.
As soon as that happens, the game theory changes.
I just was presenting at the UN.
I explained some of the examples I just gave of AIs doing terrifying things.
And I asked what percentage of diplomats who are doing negotiations on AI at the UN across all the countries, what percent of them know that?
Who here in this audience knows those examples?
And only two or three hands went up out of, you know, 100, 200 people in that room.
And that should give you a clue because.
We just don't know.
People that are in the halls of power just don't know.
And that gives me some hope.
I mean, that's honestly very terrifying, but it also gives hope.
I asked this guy from the UN Foundation, if they knew, do you think they'd be doing something different?
He's like, yes, we absolutely would be doing something different if all the leaders of every world government...
actually knew it, had to sit down and were confronted with this in the same way that Reagan was confronted by the effects of global nuclear war by the day after.
It's not like he didn't know about it, but once he was confronted with it, he did something different.
That is where I still find some hope that we can take a different path.
I mean, there have been multiple guests on this show who, when I go down this road with them, they start to point to China and what they're doing in AI and sort of the competition, right?
Do you think that's overplayed or it just has to be integrated into this collective action somehow?
It is certainly the perfect and easiest excuse, but you also have to just be honest about who is driving the race, who started the race.
And that was the U.S.
It's really important to note the kinds of races that U.S.
and China are in are different.
China is not actually racing necessarily to make the most advanced single AI.
They're racing to make the strongest cybernetic society.
So they're more in a race to deploy AI in a way that strengthens their values and strengthens their society.
The U.S.
is in more of a race to get to the maximum power as fast as possible.
So it's almost as if the U.S.
is trying to inject ourselves with the most steroids because we're not reinvesting into the social fabric.
And in fact, there's a way that both the U.S.
and China are in the same boat because as AI races to replace maximum number of livelihoods, you're sort of in a race to mutually assured political revolution.
When you have, you know, hundreds of millions of people start losing their ability to feed their kids, what do you think they do?
We need to be racing not for acceleration only, because what happens when you just accelerate without the ability to steer?
You crash.
And all of the discussions around the workforce and how it's going to change in an AI world.
Like for those of us who built expertise in a non-AI world, we're just screwed.
There's really no way to sort of differentiate ourselves.
I mean, yet to be seen.
You know, I sort of think of this as a nouns versus verbs confusion.
And that is, if you judge humans by the ability to make things that have commercial value, well, you've already fundamentally mischaracterized what a human being's worth is.
And so there's an opportunity here to like unconfuse us and say, actually, the value of a human being is about the verb of relating, not the nouns of what we do.
Because the nouns of what we do, soon are going to do better than us.
But the verb of being cannot be replaced.
And I don't know exactly what it looks like, but if we could restructure our world, like imagine a social media that wasn't about optimizing for clicks and attention, but instead it was really a full-throated, full-bally to understanding of like, How do you actually make human beings connect and thrive?
That'd be a very different world.
That's the level of shift that we're going to have to see.
And, you know, some specific things you can do is you can do, you know, employment insurance.
We can start changing the tax code so that capital is taxed more than labor.
There's lots of things we can start doing.
We can start doing token taxes and GPU taxes.
We can have every person be an actual owner inside of the AI companies, not just getting universal basic income, but universal basic ownership.
And of course, we can also...
start having liability for companies.
You know, in China, there was a court case that companies are not allowed to lay off workers.
It's not a reasonable reason to lay off a worker because they're being replaced with AI.
So you can see that they're going to be transitionary tools as we start to figure out, like, what are we going to do in a world where intelligence really is fully automated?
It sounds like whether you're a full-throated advocate for AI or whether you have more, you know, cautions around it, whatever the future is going to look like, like we really are in totally new territory.
Like we have no idea what's going to happen in either situation.
Yeah, yeah, that's right.
This is why we always go back to incentives.
Charlie Munger, Warren Buffett's business partner said, show me the incentives and I'll show you the outcome.
And we know what the incentives are.
And there's a fundamental asymmetry.
And so we have to take this asymmetry seriously.
Otherwise, you let people off the hook into the like, well, we don't know what's going to happen.
So we might as well plow ahead.
When you and Tristan sort of started this center, you knew that you were, you know, taking on some some entrenched powers.
Did you guys talk about like, OK, we're ready to be the bad guys?
AI, we did really have a sit down conversation where we looked each other in the eye and said, this is going to be the greatest wealth, the greatest power operating at the fastest speeds.
You're going to be the most powerful corporations and people in the world.
Are we in for this?
And I remember, you know, Tristan and I got invited to the AI Insight Forum in the Senate.
I think this was...
2024, 2023.
And this is the moment, like in the moment in the history of AI.
Imagine there's a movie.
There's a moment when like the government wakes up just enough to be like, that's it.
We don't know what's happening, but every leader from every AI company get in here right now.
Come to the Capitol.
This was that moment.
But there was like Jensen Huang.
There was Sam Allman.
There was like Sundar Pichai.
There's Elon Musk.
There's Zuckerberg.
And then there's us on the other side of the table.
And it really hit.
And we're like, we're the only people in this room able to truly speak to understanding the nature of the technology, the nature of incentives, and having seen this played out at least one time before.
Yeah.
I mean, I think also, you know, you have to understand the technology to understand what's at stake, right?
And so sometimes, you know, it's hard for folks who aren't technologists to really grasp it.
There's also this under-the-hood bias, is what we call it, that because...
AI is complex and technology is complex.
We outsource the decision-making like, well, they know best about how we should deploy this into society.
But that's like saying the person who designs the engine for a car and knows engines the best is best at understanding how to lay out street signs and set up like roads for communities that work well.
They're just fundamentally different problems.
But because AI is complex and technology is complex, And it's in their interest, they blur this and they say, well, we're the only ones that can understand how to actually roll this out into society.
I just want people to hold that in their mind because it's not true.
What do you hope people who are listening to this do after hearing me?
Well, at a very tangible level, it's the next, I don't know, 12 months, 18 months.
It sort of like decides the trajectory for everything else.
So you can actually look up and see which politicians are taking money from, say, leading the future pack, which is the OpenAI and a couple other super pack.
$190 million has gone in from the AI companies to the midterm elections.
It's really important to understand who those are and vote not for those people.
The second thing is go watch the AI doc.
get other people to watch it because it's that clarity that creates agency.
Social pressure is real.
Just to give a little example, the president of Iceland, she reached out and says, like, I want to help.
And, you know, that's a beautiful thing because Iceland, you know, is the home of Reykjavik.
For the Reykjavik, of course, there's something like very powerful and symbolic about that.
How did that happen?
Well, because somebody out there said, I'm going to send this, you know, the documentary.
to the top 10 most powerful and influential people I know.
And when that happens, things that didn't seem in the realm of the possible become possible.
There is no clear path here.
Let me just be honest.
There's no linear thinking that's going to solve this problem.
So for the kinds of people that just give up when that's the case, then if there were to be a path, we definitely wouldn't find it.
But if we're the kinds of people that continually look for that path, then if there is one, we will find it.
And that I think is so important for people to hold.
as well as that it's not all on your shoulders.
Like hearing all of this and natural inclination, this is too big.
What I'm going to do, it's so big, I'm just going to ignore it or I'm going to be super depressed.
And so it's important that everyone here knows, you know, that they're both soldiers and they're civilians.
You don't have to be a soldier, but if you're a civilian, like get this information to the people that want to be soldiers.
And it's not on you to solve this whole thing.
I just want you to hear, it's not on you to solve this whole thing.
Your part is to be the...
part of the collective immune system, that when people say this is inevitable, you say, no, it's not.
We haven't even tried yet.
And if that becomes the default meme, then suddenly, like, there is no excuse of saying we're not going to try.
Well, Asa, as always, thanks for your passion and your effort and your time.
I really appreciate it.
Yeah, it's such a pleasure, Bob.
I need to take a deep breath because that was pretty bracing.
I knew Aza would point out AI's shortcomings.
That's precisely why we wanted him on the show.
I just didn't expect to be quite so convinced about the challenges ahead.
At the same time, despite the powerful incentives and resources behind the AI wave, the rapid decline in public trust of AI shows that progress is possible.
I certainly didn't foresee the adoption of social media restrictions around the world.
So maybe Aza's fight isn't impossible.
Whether you see a full-blown AI dystopia on the horizon or not, I'd argue that keeping these conversations going is critical.
So share this episode around.
Check out the AI doc and share it around.
Check out Aza's podcast with Tristan Harris, your undivided attention.
Most of all, be open to taking a conscious role, whether as a supportive civilian, as Aza puts it, or a soldier like him.
I'm Bob Safian.
Thanks for listening.
Rapid Response is a Wait What original.
I'm Bob Safian.
Our executive producer is Eve Trow.
Our senior producer is Alex Morris, and our associate producer is Masha Makutonina.
Mixing and mastering by Aaron Bastinelli and Brian Pugh.
Our theme music is by Ryan Holiday.
Our head of podcasts is Lital Moolad.
For more, visit RapidResponseShow.com.