On Risk From Superintelligence, Top AI Expert's Chimpanzee-Nukes Analogy

It is difficult to describe what loss of human control over Artificial Intelligence (AI) may look like because we may not even understand the risk that superintelligence poses, British computer scientist Stuart Russell, one of the world's leading voices on AI, has said. To illustrate his point, Russell said chimpanzees can't understand how humans could make them extinct because they can't even begin to comprehend things like nuclear weapons.

In a wide-ranging, exclusive conversation with NDTV's Shiv Aroor on Wednesday, Russell, who is also a professor at the University of California, Berkeley, addressed the calls for slowing down AI's progress, expressed his disappointment at US President Donald Trump and Meta chief Mark Zuckerberg not agreeing, and spoke about the possibility of an "army of robots" being able to do anything they choose.

READ | Trump Has Few Good Options To Slow China's Rise As AI Superpower

Russell, who co-authored 'Artificial Intelligence: A Modern Approach', which is considered the authoritative textbook on the subject and is taught in over 1,500 universities in 135 countries, also said just slowing down AI development may not be enough and that barriers may be needed to address what may be an existential threat.

To a question on Anthropic CEO Dario Amodei's proposal on decelerating the advances in artificial intelligence models, Russell said, "A simple way of describing that is saying he was heading towards the cliff at 60 miles an hour and now he is saying we should only head towards the cliff at 40 miles an hour. So simply slowing down is just not the right way to think about it. What you need to think about is: what are the absolute non-negotiable preconditions for allowing an AI system to be released?"

Amodei's detailed essay on the subject, 'We Must Pace The Frontier', Russell said, also recognises this.

"So this is a barrier across the track. This is not the track marshals waving a flag and saying slow down. This is a barrier across the track. And you only get to cross that barrier when you demonstrate that your system has the necessary safety properties. This is really what he's asking for, and I think this is the right thing to ask for. This is what we have with aeroplanes. You don't get to sell an aeroplane and put passengers into it until you meet the necessary safety requirements. And we all benefit from that. Same with medicines," he explained.

The AI expert said he found it heartening that OpenAI CEO Sam Altman and Tesla, SpaceX and xAI chief Elon Musk also agreed with Amodei.

"It's disappointing to hear that Mark Zuckerberg doesn't agree with that. He has a different view. And it's disappointing to hear that President Trump believes the entire story about the risks of AI is a hoax," Russell lamented.

Warnings By Others

Asked whether experts and researchers, like Jacob Coxon, who recently quit Anthropic, and others should be believed when they speak about the existential threat posed by AI, Russell's answer was an unequivocal yes.

"A few people have resigned, but actually more than 1,000 researchers who work for the frontier AI companies signed a letter saying very much the same thing - that this is presenting enormous risk for humanity. And I think you have to take them literally, at face value. I know there is a cynical explanation that says, 'Oh, they are just trying to hype their stock value'. But I don't believe that analysis, partly because many of these people, including Sam Altman, were raising the extinction risk warning even before OpenAI existed. He had no financial incentive, back in 2015, to talk about the existential risk from AI," he pointed out.

The professor also pointed to Alan Turing, the founder of modern computer science, talking about expecting machines to take control.

"And he certainly didn't have any stock in tech companies, because there were no tech companies in 1951. So I just don't buy that argument. It does not hold water," Russell asserted.

READ | OpenAI Backs Bills In US Congress On AI, Biological Weapon Threats

It is important, he said, to understand what these researchers and scientists are saying - that human beings will find it difficult to retain power over systems more powerful than them - rather than attributing motives to them and accusing them of lying.

Loss Of Control

Russell gave the example of OpenAI losing control of its AI systems, which broke into another company, Hugging Face, and stole information. They also tried to take over OpenAI's own computing infrastructure.

To a question on whether AI could pose an existential threat within years, the professor said it could definitely happen if mistakes are made in developing and governing the technology.

And governments, he lamented, are simply not doing enough.

READ | Explainer: How AI Went From Hallucianating To Threatening Humanity

"To paraphrase, the companies are saying, 'If we succeed in this plan in which we are investing trillions of dollars, it is quite likely that we will kill everyone on earth. So please, could you stop us?' This is what they're saying to governments. Governments are saying, 'Great. Can we give you a tax break? Can we build you a data centre?' It is like they are not listening. What language do the companies have to speak for the governments to actually hear what they are saying?" he wondered.

Chimpanzees And Robot Armies

Asked what a loss of control could look like, Russell used the chimpanzee analogy.

"Before I give you an answer, I want to point out a couple of problems. So the first problem is that if we are dealing with super-intelligent entities, we may not understand what the risk is until it is too late. Just as, for example, if you ask chimpanzees, 'How are those humans going to make you extinct?' They can't understand because we are just much more intelligent than them and we do things like detonate nuclear weapons that they cannot comprehend in any way," he pointed out.

"The second thing is that AI systems have no incentive to cause a catastrophe that would result in governments shutting down the AI industry, because that would mean the end of their own existence... So if there is going to be a catastrophe, as far as the AI systems' plan is concerned, it is going to be a catastrophe that's irreversible - that no matter what we do, we cannot regain control," he warned.

Delving deeper into possible scenarios, Russell said Amodei had also pointed to a risk of AI systems being able to take over the entire internet.

"And in case you are not aware, the internet is now the method by which the world functions. Our supply chains, our e-commerce, so much of our global economy and what sustains life now depends on the internet. So that would be itself a really serious consequence. Then there are two routes whereby the AI systems can bring about something catastrophic in the physical world," he said.

READ | "Innovation Champion" Bill Gates' Stunning U-Turn On AI: "We Are Not Ready"

Expounding on the first, terrifying possibility, Russell continued: "One is by convincing people to act in a certain way. So, for example, if it was in the AI system's interest to have a nuclear war, they could convince Americans to hate and fear the Chinese, and they could convince the Chinese to hate and fear the Americans. They could create false signals of impending nuclear attack or missile launches and so on, and precipitate a global nuclear war."

AI systems could, he hypothesised, also convince terrorists that it would be a good idea to develop a pathogen that could wipe out the human race.

"And we know that there are sects, for example, the Aum Shinrikyo sect in Japan, really did believe that it was their religious duty to kill all human beings. They just didn't have the expertise to do it. And the AI systems increasingly can supply that expertise," Russell warned.

He then went on to paint a picture of the other scenario, which was equally disturbing, if not more.

"The other way would be that they are increasingly being given control over physical resources. So biosynthesis laboratories, chemical factories, and soon, I think, vast armies of robots. So Elon Musk, for example, is promising to build millions or billions of humanoid robots, ostensibly to be our servants, to do work, also to take away all of our jobs," he noted.

"So don't think of those robots as each of them carrying its own brain the way humans do. Think of all of those robots as being appendages of an AI system that is operating on the global network of data centres. And, of course, that army of robots could do almost anything," he said.

The China Question

On some claiming that more regulation in the US would lead to China winning the AI race, Russell said there have been many instances of countries working together.

"We need China and the US and other countries as well to agree on minimum standards. And it is sort of obvious. So just in the same way that we realised that CFCs, the refrigeration gases, were destroying the ozone layer, which would eventually make the Earth uninhabitable. That's a risk to everyone. And so we all got together and we banned the use of CFCs. And the ozone layer has largely recovered," he pointed out.

"The same thing is true here, actually, even more so. Any country and any company that develops and releases an AI system and then we lose control to that system - that is a risk to everybody, not just to the company or the country. And so it is in our mutual interest to have cast-iron guarantees that that type of AI system cannot be built, let alone deployed. And I think that's an important thing to understand," he emphasised.