# Demis Hassabis on AI Safety, RSI, and Sovereignty

**Podcast:** Masters of Scale
**Published:** 2026-11-05

## Transcript

The very best founders I know are brilliant at building systems.
They connect teams, they remove bottlenecks, and they eliminate single points of failure.
And yet, when it comes to their own wealth, most are running a disconnected stack.
A tax accountant here and a state attorney there, a wealth manager who doesn't talk to either one of them.
Creative planning was built to fix exactly that.
One integrated team of tax professionals, state planners, investment specialists, all coordinated by a dedicated wealth manager who sees your full financial picture and keeps every piece working together.
Proactive tax efficiency, estate strategy, investments, all under one roof.
Creative planning, where wealth works together.
Learn more at creativeplanning.com slash mastersofscale.
AI is full of possibilities.
Enterprises run on realities.
That's why Pega combines the world's most powerful AI with the governance, transparency, and control enterprises demand.
So you can move faster, adapt faster, and scale AI with confidence.
All the power of AI, none of the uncertainty.
Learn more at pega.com slash AI.
A lot of the cyber issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering.
The human's very much still in the loop.
And the next phase, which I think carries the most risk, is closed RSI, closed loop, without the human reviewer process.
And that will obviously be faster, but I think is also a very dangerous way to get to full AGI.
I do think there are some things that generally all the Leaning Labs do agree on, like...
The idea of third-party auditors or evaluators coming in, obviously there's a question about which third parties, and I think we need a healthy kind of ecosystem of that.
You could have good startups and companies doing that.
There's enough revenue there for them, given the amount of investment going into AI as a whole.
This is Masters of Scale.
I'm Jeff Berman, your host.
In today's episode, our very own Reid Hoffman sits down with the incredible Demis Asabas.
Demis is the co-founder and chair of Google, DeepMind, and chief scientist of Alphabet.
Reid and Demis discuss the race to AGI, the future of AI-fueled scientific discovery, the need for more cooperation on safety efforts, and much, much more.
I hope you enjoy it.
Demis, thank you for joining us.
Great to be here.
Thanks for having me.
We've known each other for...
I think well over 10 years now, maybe approaching 12 or something like that.
It was an honor when you asked me to be on the AI ethics board for Google for our various funny meetings.
So let's start, because one of the things we've been trying to do at the summit is to be envisioning the world as we'd want to make it.
The theme has been agency.
And, you know, as you...
have been a leader in and knew well before most people were thinking about AI, that one of the key things for thinking about AI for human benefit is medicine and medical.
And so say a little bit about how you envision this, how this helps, you know, people say, oh, if I could be getting that, I understand why you AI guys are actually trying to build something that creates.
a longer human experience, a more quality of experience, call it anti-X risk.
Yeah, for sure.
I mean, if we take a step back for a moment as like big picture, and you know this knowing us from quite early in our deep mind journey, that the whole point of me working on AI for my whole career was to use it as a kind of like the ultimate tool to advance science and medicine and our understanding of the world around us, but also improving human health.
I've always said that the number one thing we should use these amazing tools that we've now built and we were dreaming about 20 years ago is to improve the human condition and improve human health.
And I've tried to do that certainly with my own.
more personal projects, obviously things like AlphaFold, but also the spin-out drug discovery company Isomorphic, I've tried to push it really hard in that direction, as I know you have too more recently.
And I think it's exactly the sorts of things we need to be showing the world that AI can do.
I think, you know, what better thing than to, you know, use AI to help us cure terrible diseases.
So some of the protein folding was this kind of...
intersection of the idea of like we have really good data we could use scale learning because the whole learning big cognitive revolution from learning starting with even Pong, and self-play and all the rest.
It's like, okay, this would be a really good way of doing it.
How does AlphaFold lead to medicines?
Because one of the things, the question I get frequently is like, yeah, you're talking about that, but is it true?
Is it going to happen anytime soon?
So I'm passing along the question I get to you.
Yes, sure.
Well, look, biology is the most complex systems.
in the universe, right?
So that's why it's so hard because of the emergent behavior, the interactions, and proteins are the building blocks of life.
That's how I've always seen them.
And the protein folding problem, which is what we tried to solve with AlphaFold, was considered to be like a 50-year grand challenge in biology.
It's sometimes described to me as like Fermat's last theorem, but for biology.
And it's critical to understand the structure of a protein from its genetic sequence, its amino acid sequence, to understand its function.
And that's really important to understand basic biological mechanisms, but also to understand what goes wrong in disease and what things to target.
You know, if you imagine you're designing a drug, then you need to know what the 3D surface of the protein looks like so you know which part of that protein to bind to.
And so it was always a critical, but not obviously only still a small part of the entire drug discovery process.
But it's an absolutely critical part of that.
And we were able to, you know, with AlphaFold, accelerate it by orders of magnitude by building AlphaFold.
And interestingly, at the time, and if you think about it, we finished AlphaFold 2 back in 2020, which is, you know, six plus years ago now.
And it's still probably the best example of...
concrete example of AI applied to a really hard scientific problem that was thought to be sort of uncrackable.
And so that just shows you how far ahead of its time it was.
And now, you know, we're really proud of the impact it's made.
Like four million researchers around the world use it.
Pretty much every biology and medical researcher in the world, almost every drug in future will have made use of AlphaFold, all the structures.
But it's not enough.
We want to expand it to biochemistry and...
also predicting things like toxicity or ADMI properties, all of these other things that you need to know about before you can have a drug that's safe to use and that's efficient in whatever it is you're trying to tackle.
So speaking of accelerations, let's go to one of the hot topics of the day, recursive self-improvement.
What is your kind of theory of the game?
And I say this because you...
or a master games person.
So I use that as kind of deliberate.
And if you don't know what I'm referring to, either look at the documentary or look it up on Wikipedia.
So like RSI, opportunities, concerns?
Yeah.
Well, look, RSI, it's obviously an extremely useful technique to make progress.
It mostly works best in what I call verifiable domains.
what's called variable domains.
So math is one, which is why we're seeing so much advance there.
Coding is another, but originally it was games.
So, actually, one of the first sort of famous recursive self-improving systems was one we invented quite a lot of years ago now called AlphaZero, built off the back of AlphaGo, which was the kind of, you know, famous program that beat the world champion at Go back in, you know, 10 years ago now, 2016, and kind of was the beginning, I think, of the modern AI era.
And then we built a second version that was in some ways more consequential, AlphaZero.
that didn't need to use any human data anymore and it just basically played against itself.
Started at random and then...
after around a dozen to two dozen different epochs, it would play like a million games against itself.
It would then train a new version of itself that was slightly better than the old version, and then that would play a million games against itself and so on.
And then it could play any game, not just Go.
It could play any two-player perfect information game.
And for me, it was especially visceral as a chess player because I remember watching it play chess.
And within, you know, 12 hours, basically, or less than 12 hours.
It would start from random play in the morning to, you know, you'd have your coffee break at 11 a.m.
and I could still just about beat it.
Then by lunchtime, it was grandmaster level.
And then by tea time, it was better than any chess-playing entity.
Tea time is dinner time for the U.S.
Yeah, computer time.
Or computer, you know, kind of handcrafted chess computer is better than all of them.
So we sort of saw the power of that recursive self-improvement.
And obviously the thing that all these domains have in common is you can verify the answer.
Is it a correct answer?
Is it an optimal answer?
Is the equation correct?
It does the code run.
Did you win the game?
So those domains are...
And the important thing about a verifiable domain is you can verify the answer so you know when you've hit the right point.
But also that means you can create almost infinite synthetic data.
because you can just use the current best versions you have to set new problems or make new moves and then just sort of verify whether the synthetic data was correct.
The problem with synthetic data a lot of the time is you can't verify that it's right from the right distribution.
But in these cases, you can.
So we've seen the enormous progress with maths and with coding, and coding's obviously become enormously valuable, and everyone is having to do that now.
But I do worry about it because in the limit, so there's different forms of RSI.
Right now we're in the early days of RSI, I would say, where what's happening is, and you're probably all experiencing this as well with your teams, is it's making a human engineer maybe three, four, five times more productive.
But the human's very much still in the loop and reviewing code and architecting things and writing code still, but it's all just enhanced.
The next phase, which I think carries the most risk, is closed RSI, closed loop, where you basically manage to somehow figure out how to do it without the human reviewer process in the whole procedure.
will obviously be faster, but I think is also a very dangerous way to get to full AGI because we're likely, for two reasons, it sometimes used to be referred to 20 years ago as like an intelligence explosion because you don't quite know, you know, how do you control, keep control of such a system, such a process, and then the end result will likely be quite alien.
It's going to be like an alien intelligence.
There's no reason you to think it would be sort of in the similar kind of mind space that humans are in.
So that will make it very hard for us to understand it and maybe for it to understand us.
So I think we should be very cautious as we get closer towards that closed-loop form of RSI.
And I think all the AI researchers sort of know that, but there is a sort of intense, obviously, competitive race to improve your code.
So, presume that the incentives to be in a race continue, and you and I, some people really know this, have tried to figure out if there's various ways you can get global collaboration to have less.
You've dedicated a huge amount of energy to that, so thank you.
It's a hard game, as you know.
But presume that the competitive pressure for continuing to do RSI and coding continues.
In that environment, what are the best ways that we can potentially navigate that?
Because we have to see where RSI is going, what it's pointed at.
What are the kinds of things that the leading labs, such as your own and the others, what should they be doing to try to navigate this environment?
Well, what we're doing and I think other leading labs are trying to do is you've got to...
So there's more prosaic things like make sure cyber defense is accelerating faster than cyber attack and making sure you give access to trusted cyber defenders first on maybe slightly fine-tuned versions of your general models that are fine-tuned for cyber defense.
And so you keep the defenders ahead of the attackers.
It's a pretty tricky problem to do in general.
Also building much more hardened sandboxes.
I think a lot of the cyber kind of issues that we've seen in the last six months or so could have been addressed with better engineering, more robust engineering.
I think better, you need really amazing monitoring systems.
You know, they shouldn't be possible for agents to be running for weeks without anyone realizing, you know, these kinds of things that we've seen some of our competitors and others.
Maybe it's just the beginning of this happening, so I think everyone will get rapidly better, but they have to as well, because otherwise something bad is going to happen inadvertently.
Some kind of incident, I sometimes call it like an oil spill, is going to happen on the current trajectory if we don't get collectively, if everyone doesn't pay a lot more attention and put a lot more resources on these types of systems.
Eventually you kind of want...
formal guarantees, maybe some bounds around what these systems can do and how robust, test how robust the things like the guardrails are.
This is actually something I encourage a lot of top mathematicians that I know well who are maybe pretty sad about the state of pure maths, but their skills would be incredible if they pivoted to these types of domains, more like complexity theory, game theory, how do you, and of course we're not going to have to start dealing with swarms of these agents, perhaps potentially collaborating in some way.
way.
How does one keep track of all of that?
Should there be ways that the labs are encouraged, required, permitted to collaborate on this?
And if so, what should some of those ways be?
Because it obviously gets stronger if there is sharing of intelligence, methodology, mutual accountability.
Yeah, I mean, there's been lots of proposals and I've been working on these things a lot and you saw my essay over the summer, maybe some of you read it about...
potential for like an SRO industry-run standards body with some kind of government oversight over it.
I think something like that is going to be needed, and then ideally it becomes international.
You know, it's US-led, and then obviously the state of geopolitics today makes it, and our international institutions are very, very weak, as we all know, just at the time when we need them to be strongest in a way.
So that's a bit unfortunate, but we've got to start somewhere.
By the way, British, a bit unfortunate is, holy f***.
Just to translate.
Just to translate.
Another translation.
Yes, like tea time and dinner time.
Yes, exactly.
A bit unfortunate.
And so we're trying to, you know, various of us are trying to do things about that.
And that essay did have an impact in the White House and in the administration and with our colleagues.
And eventually, you know, I think some of that energy became the accord that the famous letter everyone signed, which is...
which is better than nothing.
I don't think it goes far enough, but it's a start.
And at least it acknowledges from all of the main players the issues that are there and that they need to be addressed.
It doesn't say too much about what to do about them, but that has to come next.
And at least it gives air cover, I think, for the teams to work together on what sorts of benchmarks, what are safe best practices on safety?
What are you seeing in terms of analysis of what things could happen with these types of systems?
And I do think there are some things that generally all the leading labs do agree on, like...
the idea of third-party auditors or evaluators coming in.
Obviously, there's a question, well, which third parties?
And I think we need a healthy kind of ecosystem of that, that actually, you know...
you could have good startups and companies doing that, that there's enough revenue there for them.
And they should be given the amount of investment going into AI as a whole, a bit like there are auditors, you know, in the finance industry or in any of these industries.
So I could see that happening.
And some of the big players are moving into this, like Accenture, we're working in, other leaders are working with, I think they're a little bit ahead of the game because they've acquired, you know, a good startup, good friend of ours, Mark Warner startup.
So they have some.
expertise internally.
But I think we need more of that.
And then other things like, you know, the leading labs, maybe red teaming each other before general release of these systems.
When you've built substantial wealth through your business, it's often tied up in a single equity position.
The upside is real, but so is the risk.
And knowing when to act isn't always obvious.
Creative planning works with business owners to build a strategy around concentrated equity.
when to diversify, how to manage tax risk, and how to protect what you've spent years building.
Creative planning, where wealth works together.
Learn more at creativeplanning.com slash masters of scale.
Global hiring is such a headache when you're building fast, but don't let payroll compliance or onboarding slow you down.
Companies like Eleven Labs and Mistral are scaling globally and effortlessly with deal.
Eleven Labs quadrupled their headcount in 2025, and Mistral pays 364 employees across 15 countries, all from one system.
Build your global team with Deal.
Visit deal.com slash MOS and start expanding your business today.
That's D-E-E-L dot com slash MOS.
Every business hits the same wall the moment your business outgrows the systems that got you here.
Spreadsheets everywhere, five different logins, data scattered across entities that don't even talk to each other.
If your finance team spends more time finding data than using it, that's when you need the Intuit ERP.
Intuit Enterprise Suite is the AI-native ERP solution that's powerful, painless, and proven, built for businesses scaling fast without the complexity that usually comes with it.
From the makers of QuickBooks, learn more.
at intuit.com slash erp.
So let's move from that to a little bit more of the fantastical.
So, you know, it started actually a few years back.
I think it was a Google engineer who said, I asked Gemini if it was conscious and it said yes, so therefore it's conscious, which seemed like a little bit of a weak inference.
But, you know, that's now gaining some motion with some of the things especially Anthropic is doing and the way that it...
kind of strategizes thinking that, well, at some point, it's almost like a Pascal's wager, at some point, there will be consciousness in these systems, and we want those conscious systems to look back, saying that we were treating them the same way that we treat a two-year-old, which is, hey, we were good to you as a two-year-old, and so it kind of continues through.
Obviously, to many people, that looks kind of wacky, right?
And so...
What do you think, I mean, because part of what I think maybe many people don't realize is you also come to this from a depth of expertise in brain science.
And, you know, one of the things is not just a, oh, did the chatbot say it was conscious or not, is not the way to make a decision on this kind of thing.
What do you think some of the ways to think about this is and also how we should be deliberately trying to shape it?
Yeah, well, look, I think there's quite a few things to say about this.
First of all, we haven't...
despite thousands of years of thinking about this, there isn't an agreed-upon definition of consciousness.
So that's one issue, I think.
And that's one issue with labeling things conscious, especially the hard problem of consciousness.
Is it really feeling this internally like the way we seem to?
Well, I don't know about you, but...
Yes, exactly.
But I used to argue with Daniel Dennett about this, that there's an additional thing, is we're not just behaving in ways like we'd imagine conscious beings behave, but we also share the same substrate.
We have the same squishy brains, and so then it's most parsimonious to think you're feeling what I'm feeling.
But of course, with an AI, that's not the case.
You don't have the second, the substrate equivalence.
So I think it's going to be very hard to conclude that.
I think there are dangers.
Look, there's two reasons why I think we should be focusing on building intelligent tools today, not conscious entities.
I think for one thing is a safety argument and technical safety and risk, which is that, you know, there's a direct trade-off between creating something, let's say, with self-volition, right, or self-awareness, even prior to conscious disease or the prerequisites, right, to a conscious entity, that it will, and that and instruction following.
or sometimes called corrigibility.
So this idea of a system actually following instructions and if it's going wrong, you telling it, you as the user or the human user telling it, stop doing that, right?
Or switch yourself off or whatever it is that you're going to change course.
So if you are training your system to be able to reject commands based on its own internal workings, then you're setting up a situation where if you've got your constitution wrong or your rules wrong, and this is what Asimov was always warning about with his three laws, you know, that was a form of, a very simple form of a constitution.
And his books are all warnings that that's, you're never going to get that right in every corner case, in every edge case.
And I think his books show that very well.
And yet we're sort of, I think a bit hubristically thinking that that, some people thinking that, you know, sometimes called the misalignment problem is sort of under control.
Just because today's systems, the good news is they seem more malleable than we might have imagined 15, 20 years ago, which is great news in terms of their steerability.
But that doesn't mean we've solved the misalignment problem in total as we get closer to a GI and superintelligence and so on.
And I just think it's dangerous to think one has.
So that's the first thing.
There's a technical problem, especially if you combine it with something like an RSI.
Yes, exactly.
There's a reason why we're talking earlier.
So RSI with...
Self-volition is, you know, sort of doubly dangerous.
So that's one thing.
Second thing is a societal, more of a societal issue, which is that I don't think we are ready yet as a society.
We haven't had the debate with a wider society, philosophers, theologians, artists, humanities, not just technologists, as to, you know, and from many different cultures.
Like, do we want to cross that second Rubicon?
Are we ready to have the discussion about do these systems have moral standing?
Do we need to worry about what are we doing with them in this box?
You know, are they slaves of some sort?
All these questions come up that are complex questions, and I don't think we're equipped to even begin to start dealing with those questions.
So my view is we should cross the first Rubicon on its own, which is build these intelligent tools, and then actually use these intelligent tools to do more neuroscience, to go back and do more neuroscience to actually understand what the neural correlates of consciousness really are.
finally understand that.
And then, you know, that'll probably take five, ten years, and then cross the second Rubicon, well, to have a discussion about should we cross the second Rubicon, and then if so, how, right, of the conscious entity type.
And how would you, because, you know, you engaged in hours of these discussions, how would you answer the folks who say, look, it's the only way we get the intelligent correlate to be powerful enough is it has, as it were, the ephemeral accidental.
Yes.
It actually has this...
cognitive state and so on.
Right.
So we should talk about this a lot.
We discussed this, you know, even 10 plus years ago, and I used to discuss it in my PhD back 20 plus years ago of, look, I felt and I still feel intelligence and consciousness are double dissociable, which means that you can have an intelligence system that's very smart on IQ and so on, kind of like the systems we have today, which I don't, I personally don't feel are at all conscious yet, right from my own, you know, everyone probably has their own view, but I don't feel yet.
And then you can have conscious...
entities that are not that smart.
For example, my pet dog is lovely, and he's pretty conscious, I think.
He dreams and everything, but he's not that smart.
And that's provable.
So I think you can have double dissociation here.
Many of the higher animals, dolphins, monkeys, and so on, elephants, I think most of us would view that they have...
some, you know, they're conscious beings, but, and they exhibit all of that sort of behavior.
And of course, they're also made of carbons and the same neurons and types of things we're made of.
So I feel like it's pretty obvious they have some of those qualities, but they're not necessarily that smart.
So it feels like you can dissociate these things.
And I think to the extent that you can, I feel like we should.
You know, we're coming up close to the end of our session and the end of, you know, the summit, sad.
But what should, We think about, in terms of the whole notion of AI sovereignty, right?
Because obviously one of the things that is kind of going around in a lot of countries right now is, oh, is this really important?
And, you know, one of our earliest conversations was with you trying to solve this problem.
So I wanted to share some thinking, and obviously it's updated for whatever that was 13, 14 years ago, but...
Yeah.
Well, look, I think first of all, first thing to say is, it looks to me like there's going to be...
you know, half a dozen, six to eight, if you include Chinese companies, company, you know, offerings at the frontier or near the frontier.
And it feels like there's going to be leapfrogging and maybe something's like three to six months ahead.
But it's going to be relatively close.
So there's a lot of options.
It depends if you're talking about an enterprise or a country.
So I think as an enterprise, you have a lot of options to arbitrage between these different things.
Some of them are better at certain things, you know, multimodal understanding versus coding.
It's very hard to be better on all vectors of importance at once.
So I think that's going to happen.
And then open source, which also has its risks, because how do you, as we've discussed for many, many years, how do you restrict bad actors using that whilst enabling, of course, as many good actors, scientists and others to use these fantastic tools.
So that's an unresolved problem.
But open source is, you know, maybe only lagging six months to a year behind.
Obviously, we need kind of Western open source solutions.
But I think at this point, if we're talking about countries as well, all companies that need their data to be private or they want to...
fine-tune further on their own, you know, sensitive data, again, be that enterprise data or sovereign data, I think that's going to be one solution.
There's also looking at what are the downstream things that can be done with these tools, you know, big companies and big sectors that can benefit from these tools like healthcare, that you can just use this, you know, almost like commodities and then build the value on top of it, rather than...
I wouldn't recommend every country, for example, trying to build a frontier model.
That just doesn't make sense to me.
So they're going to have to find some other way of shortcutting that.
Yes, and also collaborating and working out arrangements with multinational companies like your own.
I think one of the reasons I wanted to kind of end on that question is that part of the theme of the summit is agency.
And agency is not just a question of individual agency, like how do you take agency, but it's also a collective.
How do we take agency?
And I think part of the thing that's actually important in the AI age is that we take agency not just as any one country, but as countries together.
And it's part of how we have a, we are in this together and we are building for the benefit of humanity.
And I think that's the kind of arc of, we started with Sarah Fryer and thank you for joining us because Ending and Demis has been...
with such intensity that I was so grateful that you set up the agenda for me thinking about how to help in this from our very first, you know, discussion at King's Cross.
So let's give Demis a hand.
Thank you.
Thank you.
Thanks again to Demis Asabas for joining us at the 2026 Masters of Scale Summit.
This conversation was recorded live on stage in San Francisco at the amazing Presidio Theatre.
I'm Jeff Berman.
Thanks for listening.
Hey, listeners, Bob here.
If you listen to Rapid Response on Masters of Scale, you may be missing half the show because every Friday we release a second Rapid Response exclusively in the Rapid Response feed.
The guests and topics are just as compelling and timely from Ford's CEO to NASA's administrator to the lessons from the Devil Wears Prada.
It takes about 10 seconds to find.
Just search Rapid Response wherever you listen to podcasts.
and hit follow to make sure you never miss an episode.
I hope to see you there.
Humans will never be more intelligent than AI.
There's going to be two types of companies.
Those are great at AI and those that went out of business because they weren't.
How do we build a future that is human-centered?
I'm Rana Elkhaubi, and on my podcast, Pioneers of AI, we answer that question and so many more.
As an AI scientist, entrepreneur, and investor, I know what it takes to build AI that works for everyone.
Every week, I sit down with the pioneers shaping our future, and we take you behind the scenes of the AI that's transforming our lives.
Find Pioneers of AI wherever you tune in.
Masters of Scale is a Wait What original.
Our executive producer is Eve Trow.
Senior supervising producer is Tricia Bobita.
Associate producer is Masha Makatanina.
Video editor is M.G.
Frederick.
Senior talent executive is Stephanie Stern.
Mixing and mastering by the audio boys themselves, Aaron Bestinelli and Brian Pugh with original music by The Remarkable.
Ryan Holiday.
Visit mastersofscale.com to find the transcript for this episode and to subscribe to our newsletter and be sure to subscribe to our YouTube channel.