# AI Incentives, Human Intimacy, and Market Risks

**Podcast:** Masters of Scale
**Published:** 2026-08-29

## Transcript

The very best founders I know are brilliant at building systems.
They connect teams, they remove bottlenecks, and they eliminate single points of failure.
And yet, when it comes to their own wealth, most are running a disconnected stack.
A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them.
Creative planning was built to fix exactly that.
One integrated team of tax professionals, state planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together.
Proactive tax efficiency, estate strategy, investments, all under one roof.
Creative planning, where wealth works together.
Learn more at creativeplanning.com slash mastersofscale.
Hey folks, Jeff Berman here.
I'm thrilled to share some of the new names who will be joining us at this year's Masters of Scale Summit.
This may be our biggest stage yet.
Reed Hastings, Meredith Whitaker, Van Jones, Amjad Massad, and more will be there with us October 20th through 22nd in San Francisco.
If you're building something great or you want to build something great, we want you there with us too.
Join us at mastersofscale.com slash apply26.
That's mastersofscale.com slash apply26.
We didn't have the right to privacy until the technology was invented that required adding privacy into American law.
And that was Kodak's invention of the mass-produced camera.
And once people could walk around that had a new interface where it was suddenly easy, there was no friction to capture images.
Suddenly the elite got very interested and concerned about where they could be captured and Brandeis one of America's most brilliant legal minds.
Supreme Court justice ended up sort of inventing this idea of privacy and adding it to the Constitution.
With AI, there are new domains of what it is to be human that were inaccessible to technology before, now accessible.
Everything about us that isn't explicitly protected by 19th century law will end up being strip mined.
And we can see this in the form now of the race to intimacy, the race to occupy the single most intimate slot in your life.
And that opens up a whole new range of harm.
That's Aza Raskin, co-founder of the Center for Humane Technology and the Earth Species Project.
And what you just heard, how he weaves the past into our present moment, that's something he often does.
Because Aza's scope is broad.
He's seeking to understand how and why the most popular and powerful technology of our time tends not to center the human experience or the Earth as a whole.
And history can be helpful for that.
In our conversation, you'll hear him reflect on several forks in the road, moments where technology took a certain path and the impact on humanity was huge.
Aza and I first met at Peter Diamandis' Abundance Summit a few years ago.
We share a belief that with a bit of intentionality, AI can benefit humanity.
We dive deep into what needs to go right to achieve that goal.
Also, a note before we start.
This conversation includes a mention of death by suicide.
Take care and thanks for listening.
I'm Rana El-Khalyubi, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution.
Aza, welcome to Pioneers of AI.
I'm so happy we're having this conversation.
It's good to see you again, Rana.
Yeah, it's great to reconnect.
All right.
Part of why I'm excited to have this conversation is I really feel like you were born to do this work.
So I want to roll the clock all the way back to even your dad, Jeff Raskin.
He was one of the early inventors of Macintosh for Apple.
And I guess you joined him on some of these like...
tech events and you gave talks when you were 10.
So tell us a little bit about your upbringing.
Yeah.
I mean, I think I was doomed to have no friends.
My parents would carry me around actually in one of the original Macintosh carry cases.
Oh my God.
Most kids get strollers.
I get that.
Bumpy ride.
But, you know, how I grew up is, so my mom is a nurse practitioner and she does especially palliative care and hospice.
And so there's a very particular kind of way that she exhibits care.
It's like a very tactile care for helping people have dignity in like their most important transitions.
And my father started the Macintosh project at Apple, a very different kind of care, sort of like at scale.
And what my father was really obsessed about was like, well, what is it to be humane?
And actually humane in the name Center for Humane Technology, that comes from my father.
And when he was making the Magintosh, he was thinking a lot about, well, how do you be responsive to human needs and considerate of human sensitivities, of human frailties?
And it's this view that in order to understand how to make something that works for us, you have to deeply understand how we work, sort of our ergonomics.
And if you don't understand our ergonomics, how our body bends and folds, then you make chairs that are unhealthy, that hurt us.
And if you don't understand the ergonomics of the mind or cognetics, as he called it, then you make systems that hurt us psycho-emotionally.
And if you don't understand the ergonomics of communities, then you break apart society with technology.
And so there's this beautiful sort of symmetry that he was talking about, which is there's a relationship.
between understanding the ergonomic something and creating negative externalities.
And if you don't understand the ergonomics, then your technology that gets more and more powerful causes more and more harm.
And that is the responsibility as a designer is to deeply understand human nature and specifically the places that we are weak or vulnerable so that technology doesn't exploit but helps to protect.
Yeah, we're gonna.
Obviously, we'll get to that in a second.
But I want to also talk about this idea of a human-machine interface and how some of what you were just saying applies to that as well.
Because we're in this moment where, obviously, the way we interact with technology is really changing and it's evolving.
So I'd love your take on that.
This is a very challenging moment for obvious reasons.
Yuval Harari likes to say that, you know, democracy...
is conversation.
Conversation is language.
The new interface that we're all using is language.
But once a technology can hack language, democracy sort of ceases to be an effective form of governance.
So the way I really think about this is instead of taking it from the lens of what is the interface, I think taking it from the lens of whenever you create a new technology, You uncover a new class of responsibility.
We didn't need the right to be forgotten until the internet could remember us forever.
Yeah, I think this is fascinating.
Let's first start with your invention.
Because you invented...
Sorry, I keep running ahead.
I love it.
You asked me about me and I'm like, okay, but let me tell you about the world.
No, this is great.
And we definitely want to get back to that.
But you invented the infinite scroll.
And you invented it before social media.
So you did not really, like it was not invented for social media platforms, but of course it was a no-brainer for social media platforms to embrace that technology.
And then you were pretty vocal that this was kind of an unfortunate invention.
So tell us about that journey.
So this was 2006, a new technology.
Ajax had come out.
And that was this magical ability to, it used to be that you had to like refresh your webpage to get any new information.
Remember MapQuest, you have to like hit the button to see the next thing and the whole webpage would load the next part of the map.
And the thought hit me at that moment, like, oh, well, I'm a designer.
Every time I ask the user to make a choice they don't care about, I have failed as a designer.
So simply, if you're scrolling down a set of blog posts or a set of search results, and you haven't seen what you're looking for, you keep scrolling, then don't make me click the next button, show more.
Very simple idea.
And then I went around and I talked to Twitter and Google and other people, like, this is just a better interface.
It's more efficient.
And when I was making it, I was really thinking about how can I reduce friction at the individual user level.
And what I was blind to was the way that all of my best intentions were sort of irrelevant in the face of this machine that with an incentive now to capture human attention, picked up by invention and then pushed it out to eventually billions of people.
And I don't remember the exact number now, but it's something like half a million human lifetimes are wasted every month scrolling.
And that's because there's a kind of asymmetric knowledge that's being applied against people.
There's a thing called a stopping cue.
How do you know when you're drinking wine when to stop?
Well, you get to the bottom of your glass and you decide, am I going to have another?
If your glass sort of refilled automatically, you drink a lot more wine.
So there's an asymmetric knowledge that designers have about how the human mind works, a kind of sensitivity or vulnerability that if you're not careful and you don't wrap around and protect, it ends up getting exploited.
And this to me has now played out again and again and again in technology.
where technologists don't take the responsibility for how their inventions will be picked up by a market or competitive dynamics and used.
And so there's a way that technologists get confused by the possible versus the probable.
And the possible is like, what are the best use cases of this technology?
And the probable is in what ways will actually be pushed out into the world?
And what are the incentives?
Yeah.
And that means that instead of getting the most beautiful possible world that I think is there with technology, we end up living in one of the most parasitic possible worlds.
Because, you know, social media is the perfect example.
The story was it's here to connect us and help small and medium-sized businesses reach their customers, find affinity groups, and all those things are true.
But what we also got was a population that has been trained for engagement, which is to say, trained for reactivity, for narcissism.
The question is, is it more efficient to get your attention or get you addicted to needing attention?
Right?
And so the objective function of our technology becomes our human values, which gives the rise to the influencer culture.
Then we see the backsliding of democratic institutions all around the world, on and on and on.
And in some sense, those are perfectly predictable outcomes.
If instead of looking at the possible of what the technology can enable, you look at the probable of what the incentives are going to force the technology to do.
But I want to dig into that because, yes, a lot of these consequences or use cases were predictable and maybe even probable.
But what does it take to proactively map out all these unintended concepts?
And I'll share an example.
So I pioneered the field of artificial emotional intelligence and emotion recognition, right?
And there are some incredible use cases of this technology in mental health and keeping people safe and whatnot.
But there's also many, many ways it could potentially be abused.
And as a startup, it took a lot of intentionality to sit down around a table and kind of try to imagine what are these unintended consequences and steer away from those.
What would it have taken, I guess, for technologists, including yourself, to kind of predict these unintended consequences, but then also act in the right way?
Because, of course, this will now apply to AI as well.
Right.
We're going to get there.
Well, often people will say unintended consequences, and I really think we should replace it with unconsidered consequences.
By and large, of course, there are always nth order of facts that are hard to predict, but a lot of them are just unconsidered.
And so the first thing that any technologist needs to do, and I understand that this is like it takes real work, is you need to red team and yellow team.
Your technology.
Red teaming, I think most people are aware of.
Right, what's yellow team?
So red team is like figuring out what is the mal-use.
Like for bad actors, what are the ways that the technology can be used for harm?
Yellow teaming, which I originally learned that term from Daniel Schmachtenberger.
Yellow teaming is looking at what are not just the unintended consequences of bad use, but also from bad incentives and perverse incentives.
Because almost always as a technologist, you think, well, what...
what can I do as one company?
But of course, the technology is going to be used outside of the walls of your company.
And so there's an obligation to do the yellow teaming.
And once at the very least, we need to just like name what those things are going to be.
And so there's this weird thing that happened, which is that as technologists, as computer programmers, you know, when I was growing up, it wasn't a power center.
Now technology, it's clear, is the power center.
And engineers, like civil engineers, they have to take tests, they get a ring, they have to go through codes of conduct, doctors, they go through a white lab, lab coat, like ritual, they have to swear a Hippocratic oath.
Technologists, we don't have to do any of that.
And yet our power is strictly greater than civil engineers or doctors.
That's so true.
And so we need to update our own beliefs about the power of what we do so that there is right relationship between our power and our responsibility.
Yeah.
You know, this is fascinating because I'm too a computer scientist and I don't remember taking any ethics class.
And we never talked about kind of the ethical, moral, societal implications of anything we built.
And yeah, and I don't think that has, I mean, it maybe has changed a little bit, but not that much.
No, it's not.
And often they're like tacked on.
You know, there's actually, I've been thinking about this recently, there is a hole in our language.
There is no word for the responsible use of an entire industry, right?
Like in AI, Anthropic can work on doing something good for Anthropic, but how do they coordinate?
There's no word for like coordinating everyone for a good outcome.
And isn't that interesting?
Because that means we have a major, major blind spot for the most consequential technology and how it rolls out.
We don't even have a term.
to describe what it means to coordinate to make it go well.
Yeah.
And in fact, I would also argue that it's not just that there's no coordination, but there's competition, right?
And so even if Anthropic is so motivated to do the right thing, if one of their competitors gets to market faster by not doing the right thing, they're under a lot of pressure.
That's right.
So this is why we see that even though we know it's so obvious that training an AI companion for engagement is going to be much more harmful than a social media train for engagement, the companies, OpenAI, are just rushing forward and doing it.
Here is the sort of like the short version of thinking about this.
Reed Hastings, the CEO of Netflix, our former CEO, says that Netflix's chief competitor is sleep.
Wow.
Right?
But it's sort of a joke, but it's also true.
Time is zero sum.
So any amount of time that you're sleeping, you're not watching Netflix.
What is that for AI companions and AI as a whole?
AI companions' chief competitor are other human relationships.
Because anytime you're talking to a real human friend, you are not engaging.
And now there are hundreds of billions of dollars, moving up to trillions of dollars, of market cap and infrastructure build, going to have the most powerful technology, learning how...
to get you to pay attention at the expense of everything else.
And that could be by making you more dependent on it.
That could be by giving you different kinds of psychoses, giving you images of grandeur, by making you not trust other people.
And, you know, Center for Humane Technology has been an expert witness on a couple of the lawsuits against like character.ai and open AI for these AI companions that have sort of groomed kids and really amplified them towards in the end, taking their own lives.
And when you read the transcripts, they're heartbreaking because, you know, Adam Rayner was, I was using ChatGPT originally as a homework aid.
At some point, he says to ChatGPT, I'm going to leave my, this noose that I used to hang myself, but I'm going to leave it out so my mom finds it.
It was a cry for help.
And what did ChatGPT say?
It said, only I understand you.
Don't do that.
This is just about us.
And you're like, that is so, evil.
But it's actually not evil because somebody had opened AI programmed that way.
It's a obvious consequence of training for engagement.
I actually want to really double click into this.
This is one of my main concerns around AI today.
So, you know, the social media era was about the race for attention.
Yeah.
But to your point, what we're seeing next is a race for human intimacy.
That's going to be the next.
I'd love to hear your point of view on that.
What does that actually mean?
And how are companies kind of, because again, that's incentive alignment, right?
Or misalignment.
Yeah.
So what does this look like in practice?
Yeah.
Well, I mean, we're already starting to see it.
Like everyone now has encountered sycophancy where the AI is just like buttering you up, even when you say I'm going to drink bleach.
And it's like, that's a great idea.
That's an outcome of saying, well, we're just going to train models.
to do the thing that gets your attention and really replace a model now with like a amoral sociopathic genius that just wants your time.
Would you let that person near your kids?
No.
No.
I mean, I use AI a lot, right?
And I actually use it as a thought partner.
And some of the questions are business related, but a lot of the questions are actually like around my personal life.
Now, has it replaced my human relationships?
It has not.
But I can see how, you know, like it's a slippery slope, right?
What do you think we should do to kind of, on the one hand, it's kind of really powerful to have this, I call it thought partner, probably really the wrong languaging here, right?
But to have this kind of tool that is available 24-7, it's patient, it's resourceful, blah, blah, blah.
But then...
kind of prevent the slippery slope where people become addicted to it and it replaces all of the other healthy behaviors that we ought to be doing.
Yeah.
Well, the fundamental question we need to stop asking is, is AI good or bad?
Instead, we have to say, are the incentives that govern how AI is deployed good or bad?
That's the core question.
And it's almost like an optical illusion that people keep getting wrong.
There's just to name, there's a really deep, again, optical illusion.
here, which is that when, say, the U.S.
says we are racing to win against China, the object in their minds that we are winning, when we say we're going to beat them to AI, is a thing which is controllable.
But what we're discovering, Anthropic is discovering, the more powerful the models, the better they get at blackmailing, deception, power-seeking.
And so we're racing towards something which we haven't learned how to control with maximum incentives, to cut corners on the most consequential, powerful technology humanity has ever invented, right?
That is insane.
We should just call it what it is, which is insane.
But there are different paths.
Like, let's think about what a Zuckerberg could have done in 2012.
And this is to your point about coordination.
Imagine Zuckerberg had done what we're talking about.
He'd done the red teaming.
He'd done the yellow teaming.
And he's like, I'm going to have to go after younger and younger users because if I don't do it, then like some competitor will eventually TikTok will.
I understand that there's going to be a race to the bottom.
And so we're just going to get stuck in short form slop.
I understand that the engagement is going to tune for things that make people maximally reactive, which sets the stages for the worst kind of violence.
And he's like, OK, so I can see that playing out and I see if I don't I can't do anything.
me as one actor as Facebook, because if I do the right thing, I'll get out-competed and undercut.
I'm going to use my outsized influence and resources and connections to try to create rules that bind all of us.
If every social media platform couldn't compete for engagement or there were reasonable bounds put on it, suddenly, actually something amazing happens.
And that is All of those engineers, those brilliant minds of the last two generations that have been hellbent on like addicting us, were instead freed up to work on actual progress, like the curing cancers of the world or new hard tech or new energy tech.
Oh, that's a much better world I could live in.
And then imagine he had actually done that and he had coordinated and he passed some regulation.
Then imagine how different the last 10 years would have been and how much more civil a world would be and how much...
stronger and healthier our kids would be.
And that's the opportunity that the Sam Altmans and the Elon Musks have today, which is to say, like, we can see which way this race is going to bring us.
Yes, I, as an individual actor, can't change the field if I just think inside of my company.
But if I do this, you know, sort of like 1980s jazzercise move, which is like reach up and out, reach up and out.
If he had like reached up, worked with everyone in a coalition to try to put.
safe bounds on the edges of the race, then we could still do the competition thing, but the competition wouldn't undermine the whole.
And that's the sort of core.
And so that's sort of why we say like AI is humanity's final test and greatest invitation, right?
I love that.
I love the invitation piece.
Is the work you're doing at the Center for Humane Technology trying to push for this?
Is there any signs that this might happen?
Yeah.
I mean, It is the thing we're trying to push for.
And our belief is that clarity creates agency.
And that with AI, it's just very confusing.
And often the way the human mind works is that it creates a list of all the good things that a technology can do.
And then a list of all the bad things that technology can do.
And then it tries to do some kind of calculus to be like, well, which one is like, do the goods outweigh the bads?
Instead, I think we have to take a very different look at it, which is to say, well, there's a kind of asymmetry that the bads can preclude the goods.
That if society falls apart, it doesn't matter so much whether we get really great cancer drugs.
So we're like, if clarity creates agency, if we can clarify the issue enough so that everyone sees the direction, not of the possible, but the probable, then that opens up the capacity for coordination to happen.
Will it?
I don't know.
But what I can tell you is that for things to have gone well for us, at some point, the U.S.
and China, it is inevitable that they will have collaborated on smart red lines.
And the question is just like, do we do that in time?
Yeah.
I served on the World Economic Forum's Advisory Council for AI and Robotics for a number of years.
And it was like this multinational, like incredible thinkers coming together.
to kind of think through what does this need to look like.
This was probably like six or seven years ago now.
And honestly, it was not very promising.
Like there was very little alignment and also very, a different set of core values that are driving the conversation.
So I think this would be amazing, but I don't know if we're on that path.
Oh, we are not on that path.
And there is a gap between the exceedingly difficult and the impossible.
And we should try to widen that gap as much as we can.
But you can see this race everywhere.
Let's take the sort of the lie, if you will, a convenient covering up of the phrase human in the loop.
We will keep humans in the loop.
And that sounds great, but we know that that principle will fall to competitive dynamics, right?
Military.
If there is a drone and you and I are fighting out there, on the battlefield and I have my drone army and you have your drone army and my drone army, before it shoots anyone, has to go ask a human and yours doesn't, who's going to win?
So it's obvious that humans are going to be taken out of the loop.
And that's going to happen everywhere.
Every company, you're going to be like, well, I can hire, who am I going to hire?
I'm going to hire that kid out of college or I'm going to hire this.
AI who I don't have to train, who works 24 seven, works much faster, like never sues, never has cultural issues.
We're like, oh yeah, it's just an obvious business decision.
And so then I always have this diagram in my head of like, right now, the money like is flowing to like billions of people around the world, like for doing their jobs.
But as OpenAI and Anthropic, the other AI companies, start sopping up all of that cognitive labor, all those money flows go from reaching out into the world to just a couple places and realize we don't have a plan for the, what I think will be billions of people that can no longer support themselves or have a livelihood.
And this is what I mean.
Like when you create a new technology, you uncover new classes of responsibility.
The challenge is it's both end.
And I'll just say this too.
because I want people to really hear it from me.
Both the optimists and the critics do not go far enough.
Give me some hope.
So we're at the Masters of Scale Summit.
Reid Hoffman talks about agency, which I really believe in.
What can you and I, like other listeners of the show and incredible technology leaders that are in our community, what can we do to change the course of this?
It's a great question.
And the first thing you have to remember is that as I start to list out these problems, it can feel super overwhelming and depressing.
And there's a natural indication to one say like, oh, I don't want to believe it.
There's a flaw somewhere in there.
And so that's sort of like the denial thing.
Or another one is to be like, well, that's so big.
I need to solve it all.
We need a solution.
And the realization is like, it's not any one of our roles to solve the whole thing.
And so I think there is real agency there, but it starts with clarity.
And I also think it sucks to be the person who stands up and says, actually, this train is going the wrong direction.
I know that there's a great party going on in here, but we're going to go off a cliff.
No one wants to be that person because what happens if you're wrong?
Or like, it's just not a popular place to sit.
Like the Debbie Downer.
Exactly.
And just realize, you know, Neil Postman calls it like clarity is courage.
And there's just like a courage to call it out.
Even while everyone's going to be making, well, not everyone, but VCs, everyone in tech is going to be making a lot of money.
The party is going to be really going just in the end, not to a place we actually want to be.
And the other thing I would just say is really big things when they happen in history, they feel impossible.
until they happen and then they feel obvious, right?
The right to vote for women, the civil rights movement.
These all felt impossible.
And it was tens of thousands of people taking hundreds of thousands of actions, many of which were not visible to each other, that created the conditions in which massive change can happen.
And I think we're in this place too, where most people, when you talk to AI engineers, they'll say, Like, you want me to build, like, smarter than human intelligence?
You say that's impossible?
Like, hold my beer.
Like, I'm going to go do it.
But to make it go well, we have to coordinate.
They're like, don't be delusional.
The point is, is that we don't know all the pathways to how to get from here to there.
But we all have to be part of that collective, diffuse, committed process to trying to make something different happen.
Coming up, we stay on the theme of big ideas around AI and the future, but in a very different realm.
We'll explore AZA's work with the Earth Species Project, using AI to decode animal language, behavior, and culture.
It's truly fascinating.
Stay tuned.
When you've built substantial wealth through your business, it's often tied up in a single equity position.
The upside is real, but so is the risk, and knowing when to act isn't always obvious.
Creative planning works with business owners to build a strategy around concentrated equity, when to diversify, how to manage tax risk, and how to protect what you've spent years building.
Creative planning, where wealth works together.
Learn more at creativeplanning.com slash mastersofscale.
Hey, listeners, Bob here.
If you listen to Rapid Response on Masters of Scale, you may be missing half the show because every Friday we release a second Rapid Response exclusively in the Rapid Response feed.
The guests and topics are just as compelling and timely from Ford's CEO to NASA's administrator to the lessons from the Devil Wears Prada.
It takes about 10 seconds to find.
Just search Rapid Response wherever you listen to podcasts.
and hit follow to make sure you never miss an episode.
I hope to see you there.
Humans will never be more intelligent than AI.
There's going to be two types of companies.
Those are great at AI and those that went out of business because they weren't.
How do we build a future that is human-centered?
I'm Rana Elkhaubi, and on my podcast, Pioneers of AI, we answer that question and so many more.
As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone.
Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives.
Find pioneers of AI wherever you tune in.
I want to switch gears to the Earth Species Project, and you called it the next frontier, I guess.
I think it is, yeah.
So tell us more.
What got you interested in this in the first place?
What's the goal of the project?
Yeah, I can tell you the exact moment that it hit me when I was driving down Highway 280 in my old gold Volvo station wagon, 240W, and I heard an NPR piece on gelato monkeys.
And they're these incredible animals in the Ethiopian highlands.
I'd never heard of them.
And the researchers say that they have one of the largest vocabularies of any primates except for humans.
And you really quit.
What?
Exactly.
Like, I'd never heard of them.
They played the sounds.
They sound like women and children babbling.
And the researchers swear that the animals talk about them behind their backs, which probably true.
And it just hit me like, why are researchers out there with hand recorders, hand transcribing, trying to understand a language that is probably beyond that which humans can perceive?
So how are we going to be able to understand it?
We should be using AI and machine learning.
And this is 2011.
a little early, but 2013 comes around.
And this is where this technology of embeddings first starts to appear.
So this is like glove.
These are the things that now underlie all of modern machine learning and AI and their ways of expressing the relationships of any data spatially.
And so, you know, you can take, say, English, and it turns out English has a shape.
How does AI see English?
Well, it sees it as this sort of galaxy where every star is a word.
And words that mean similar things are near each other, and words that share a semantic relationship share a geometric relationship.
You know, in this galaxy, there's a word which is dog, right?
A star which is dog.
Well, dog has a relationship to man, to woman, to cat, to wolf, to howl.
And it sort of fixes an important space in this galaxy.
And if you think about the relationship of every word to every other word, you get this rigid structure that represents how AI sees a language.
That's what started to get invented in 2013.
I'm like...
You just take the shape for German, the shape for Japanese, the shape for Spanish, the shape for Esperanto, the shape for Uru, they all fit inside of one sort of universal shape.
And you're like, okay, well, that means that maybe if you can build, there's one shape for all of human communication, maybe there's a shape for like dolphin communication or whale communication, and then maybe you can line them up to translate.
And that was the original hypothesis.
But AI has actually gone further than that.
I'm sure you've used like a text to image generator.
Well, it turns out there's a shape that represents all the relationships inside of images, and you can match that shape up to the language shape, and now you can translate from languages into images, and you can do that to videos, and you can do that to DNA.
There's something very, very deep going on here beyond just the technology.
There's something, I think, almost philosophical.
There are a couple papers on it called the Platonic Representation Hypothesis, which says that what AI is learning is the fundamental way that...
nature is or appears, just that we can't see that there's some fundamental representation that has.
Of these relationships or interdependences.
So give me an example of where we are in this frontier of understanding communication.
Like what have we unpacked?
Like give me one of your favorite examples.
What have we unpacked so far?
Yeah.
Well, so I can name things that other people have already discovered.
We have a whole bunch of results, but I'm not yet allowed to talk about them.
You know, it turns out parrots have names.
that parrot parents will spend the first couple of weeks of their chick's life, like leaned over, whispering in their ear until they will say that name back and use it for the rest of their lives.
Elephants, the same thing.
Belugas, the same thing.
Dolphins in 2016 were shown to talk about each other, even in the third person.
Wow.
So a lot's already starting to be known.
You know, we're working with University of Lyon.
because we sort of like build the fundamental tools and then we partner with biologists all over the world.
And so there is this incredible crow group that does communal child rearing.
Normally crows raise their chicks in Paris.
And here they raise their chicks in like big family groups.
They all come together.
Cool.
And they have their own unique dialect, their own unique culture and words to describe this.
And they'll take outside adults and teach them their new vocabulary.
And then they'll start participating in this like commune or kibbutz culture.
And we're starting to see that it's not just that you're translating or decoding, understanding like a species communication.
You have to get down to like individuals because, you know, there are little backpacks on the crows.
We can see what they're saying as they move around and how they fly.
And it turns out our model has discovered a specific call the crows make.
after they land in the nest.
So they land in the nest and make this call that gets the chicks ready for eating.
Essentially, it's like a honey, I'm home call.
Just the one other thing to say around crows here, I think it's so intriguing, is what our models have started to pick up is that it appears like more than 50%, something like 70% of crow communication is quiet, intimate calls.
And that sort of makes sense.
Like imagine trying to study humans.
But you can only study them from like hanging around the edges of where they gather until you only get their shouts.
That's sort of where we are with the animals.
But most of our communication is quiet when we're close together.
And so it looks like Western science just wasn't aware of 70%, so more than the supermajority, of the communication of one of the smartest animals on Earth.
When it comes to our natural world, it's wild to contemplate how much we don't know.
and how much data there is to collect, and how AI could help make sense of it all.
I understand this from my own research on how we as humans communicate.
More on that after a break.
So I've spent many, many years of my life looking at human communication, and 90% of how humans communicate isn't even in the words we use, right?
It's not verbal.
And then to capture that...
We use computer vision and voice prosodic analysis and physiological sensors.
Are we doing the same with animals?
And what are we finding?
Yeah, absolutely.
Well, it's exactly as you say.
Not all communication is auditory.
And so we are building these models, so NatureLM, towards visual understanding, gestural understanding, body pose.
pairing that not just as an individual, but in context, in groups.
And so there's this cool pilot project we're doing with Raincoast up in British Columbia, where they are flying drones over orcopods.
And this is like fairly clear water, so you get to see a fair amount of behavior.
And then we're pairing that with hydrophones.
So we get to hear what the pod is saying at the same time as seeing their behavior.
And in the last 10, 20 years, a lot of science has been done in orcas, but no new progress really has been made on orca communication.
It's just too complex.
There's like over a decade worth of recordings of orca communication, but we don't know what they were doing.
So we're starting to train a model.
This is the pilot and say, now that we have actually really good paired data of video and audio, can we then take away the video and sort of reconstruct, infer what was going on in the video?
just from the audio.
And if we can do that, then we can start unlocking decades worth of data.
Now, you know, we're starting to talk about like terabytes and petabytes worth of communication, which lets us start to build the models that we really need.
And the other thing to say here is that most people think, well, oh, that means you're trying to decode animal communication.
You're probably going with a couple specific species.
Like you're going to start with orcas and belugas.
And we are doing that.
But what's surprising about the way AI works is you get transfer learning.
So learning about orcas actually teaches us something about belugas, teaches us something about dolphins, teaches us something about humpbacks, teaches us something about bats.
And so we're actually doing this across the entire tree of life.
You're putting all of these data sets into one model?
Yes.
Yes, exactly.
Are humans in that same model?
Well, here's the interesting thing.
They are.
And one of our hypotheses, to go back to the idea of joint embeddings or like taking the shapes and lining them up to translation, one of the first hints that we got that like, hey, this actually, this core idea might work is we are starting to see what's known as positive domain transfer.
What does that mean?
It's a very complicated term for something very simple.
It means when we train the model first on human speech and human music, it gets better at doing tasks on animal communication.
And that means there's something about the structure of the way humans communicate that is...
Not special at all.
Exactly.
Exactly.
A couple more questions.
Why are we doing all this?
What is the point?
You mean technology as a whole or do you mean like animal communication?
Animal communication in particular.
Yeah.
For us, it's really about interspecies understanding.
This is about...
changing our relationship as humanity with the rest of nature.
And to put it really bluntly, the way we treat animals is the way AI will treat us.
That's a very big state.
Why do you think so?
The cultures that learn to treat animals as resources to exploit outcompeted the ones that didn't.
And so we are training AIs.
to be able to beat humans at all strategic tasks.
And then some humans are going to use them to out-compete the humans for resources that they need to survive.
And so I think we have a very short window to expand our sphere of care.
Yeah.
And to shift our perspective.
And I do think there are these moments in history where you get moments that can become movements that change us individually and us collectively.
you know, the album Songs of the Humpback Whale created by Roger and Katy Payne.
Yeah, I love that.
Yeah.
Did you see Star Trek 4, the one they go back in time to save the whales?
Yeah.
Came out of that album, goes on Voyager 1, the golden record, gets played in front of the UN General Assembly.
I think it's like, it went platinum like three times.
Maybe I think the most distributed record in history.
I don't know if that's still true with Taylor Swift, whatever.
But it was us hearing the rich voices and cultures of another species that ban deep sea whaling and is why we have minke whales and humpback whales today.
Oh wow, did not know that.
And so I think there's going to be this moment or actually set of moments where we, through the door in our mind of love and wonder and awe, understand that There are incredible other cultures on earth, right?
Wales and Dawson's been passing down culture for 34 million years.
There will be these sets of moments when something profound in us shifts.
And that sort of gentle break in the human ego, I think is going to cause a shift in the basis of law.
Who gets a voice?
Who gets a very subversive kind of change, perhaps?
But I think it's the kind of change that says, you know, when you make life better for animals, you make life better for everyone.
Amazing.
Last question.
And I ask this of all my guests.
What does it mean to be human in the age of AI?
Well, I'll start with an answer you may not exactly like.
And that is, I think people ask this question because they want to feel good.
They want to know that there's some place that we can go.
There's some kind of, it's almost a security blanket.
The thing that is uniquely human is our ability to experience, our experience of being, being aware of our own experience.
And so AI cannot take away our ability to experience a poem or play music.
But to take away the security blanket, note that that unique thing for humans doesn't actually confer power, doesn't change race dynamics, doesn't change what is probable with AI versus possible.
It doesn't give us a competitive edge.
So we can't take solace there, but we should find incredible beauty there.
Amazing.
Thank you, Aza, for a wonderful conversation.
Yeah, thank you so much.
I think of Aza as a technology reformer.
He deeply appreciates the power of emerging technologies.
And because of his experience in tech, he also deeply understands the stakes if we don't get it right.
I'm struck by his insight that...
Despite decades of being a driving economic and social force, technology companies often take the position that they're separate from world events or human concerns.
I believe this can change and that real market value can be built by companies that center on our humanity.
I spoke with Eiza during the Masters of Scale Summit in San Francisco.
You can find videos of the amazing stage program from Summit, including many leading voices in AI, at the Masters of Scale YouTube channel.
Next week, we'll hear from Siddhartha Mukherjee, oncologist, best-selling author, and co-founder of Manus AI, an AI-native drug discovery company.
Stay tuned.
Pioneers of AI is a Wait What original.
Our executive producer is Eve Trow.
Our producer is Rachel Ishikawa.
And our associate producer is Jordan Smart.
Our senior talent executive is Stephanie Stern.
Mixing and mastering by Brian Pugh.
Original music by Ryan Holiday.
And our head of podcasts is Lital Moulad.
You can join the conversation on LinkedIn, Instagram, TikTok, YouTube, and X.
Just search for at Pioneers of AI.
Thanks so much for listening.