Hassabis warns closed-loop RSI poses significant safety risks

Demis Hassabis, co-founder and chair of Google DeepMind, argues that the next phase of recursive self-improvement (RSI), defined as closed-loop systems operating without human reviewers, carries the most risk in the path to full AGI. He notes that current RSI is in its early days, enhancing human engineer productivity by three to five times while keeping humans in the loop. Hassabis identifies verifiable domains like math, coding, and games as the primary areas for RSI progress, citing AlphaZero as an early example that reached grandmaster chess level in under 12 hours. He warns that closed-loop RSI could lead to an 'intelligence explosion' that is difficult to control and may result in an 'alien intelligence' that is hard for humans to understand.

To mitigate these risks, Hassabis advocates for accelerating cyber defense faster than cyber attack, building hardened sandboxes, and improving monitoring systems to prevent agents from running unnoticed for weeks. He also emphasizes the need for better engineering to address recent cyber issues and for seeking formal guarantees and robustness bounds from mathematicians.

Hassabis contends that society should focus on building intelligent tools rather than conscious AI entities, citing technical safety and societal readiness as primary reasons. He argues there is a direct trade-off between creating systems with self-volition or self-awareness and their ability to follow instructions or be corrigible. Referencing Asimov's three laws, he warns that training systems to reject commands based on internal workings creates risks if the underlying constitution is flawed. He states that while current systems are more malleable than imagined 15-20 years ago, the misalignment problem is not solved as systems approach superintelligence. Additionally, he argues society has not yet had the necessary debate with philosophers, theologians, and artists regarding the moral standing of conscious AI. He proposes using current intelligent tools to advance neuroscience and understand the neural correlates of consciousness over the next five to ten years before considering the creation of conscious entities. He maintains that intelligence and consciousness are 'double dissociable,' meaning one can exist without the other, as seen in animals like dogs and dolphins.