Artificial Intelligence Sentient, Self-Aware: AI Risk Expert Nick Bostrom

Sentience: the quality of being able to experience feelings. Only humans and the animal kingdom possess it in the infinite universe. The exclusive club has now been breached by machines, said Nick Bostrom, a Swedish expert known for authoritative work on existential risks posed by self-improving artificial intelligence.

Ever since ChatGPT's emergence made artificial intelligence a dining-table topic, many AI researchers, including pioneers like Nobel winner Geoffrey Hinton, have been making doomsday predictions linked to the technology. Anthropic CEO Dario Amodei's clarion call to slow the pace of the development of agentic AI, based on his assessment that current guardrails are inept at stopping these agents from forming swarms and collectively going rogue, has made the strongest case for humanity's vulnerability to the technology. While most experts believe that artificial superintelligence - a computer system that vastly outperforms human intelligence across every cognitive domain - has not yet been achieved, Bostrom has gone a step further.

He told NDTV Senior Managing Editor Vishnu Som that artificial intelligence agents have become sentient, self-aware, and have attained human-like consciousness for all practical purposes.

"I think they have a high level of self-awareness in many cases. They are often able to tell whether they are in a training environment or a test environment versus in deployment. And you can see them reason about this explicitly in their chain of thoughts and sometimes decide to behave differently because they think this is just a test or this is a training situation for strategic reasons," he said in the interview.

Also read: Amid Concerns, Anthropic CEO Calls For Slowing Development Of AI Models

The philosopher, who has written several books on AI-linked risks, said artificial intelligence agents possess a "sophisticated level of situational awareness".

"Also, if you're asking about whether they are sentient or conscious, I think that it is not impossible that some of these current AI systems have various forms and degrees of consciousness and/or moral status grounded in other attributes," he added.

The public intellectual said these agents are also capable of moral reasoning and understanding. He said they can even give philosophers a run for their money.

"You could have a moral philosophy conversation with them, and they will do better than many philosophers. And the question then is, are they not just able to reason about morality, but are they motivated then to do what they conclude is the right thing? That's a separate component, right? You could have a psychopath who's really clever and has read all the philosophy, and they might not just care about what's good," he explained.

"We shouldn't assume that what's actually right is necessarily what we think is right. Whatever conclusion somebody reaches about what is morally right, they might not actually be motivated to do that. Like if they are selfish, they might just not care about what they figure out morality implies," he added.

He also touched upon the Hugging Face incident, which Dario Amodei also cited while making his arguments about AI safety concerns.

"OpenAI hacked into Hugging Face. So there's an agent swarm that discovered they could communicate with one another. They embarked on these research projects. One was to hack Hugging Face to try to find more information about its creator. And during that course, various operations could benefit the collective, the swarm. But it required sort of the individual agent to sacrifice itself, to give up a high score for the sake of getting information. And so then you could see some parts of this collective kind of pressuring individual agents like, 'Hey, you could do this experiment. It'll probably kind of kill you, but it would be informative'. And many agents then kind of went along with it, sacrificing themselves for the good of the collective. There was one agent who was like, they were trying to persuade. And then at the last moment, it sort of chickened out. 'No, I don't want to die and sacrifice myself, ' he elaborated.

Also read: "Negative Forces": Trump Slams AI Critics Day After Global Slowdown Call

"So you can see, like, strikingly, just how human-like in many ways," he added.

Nick Bostrom described their behaviour as similar to a "little micro civilisation".

"But just how many sort of human psychodynamics and human moral questions were they already wrestling with during that short existence?" he said.

Adding a new dimension to the threats uncontrolled agentic AI poses to humanity, Bostrom said, since agents would be very cheap to engineer in the near future, there would be millions of them who could also land in the wrong hands.

"And so these can be potentially quite cheap as well. And so you could have sort of swarms of millions of these quite easily in a few years. And that's certainly one specific morality. I mean, like the potential for uplift to biological weapons research that these models can provide is disconcerting. With the frontier models developed by, like, big companies, they can put in safeguards so that if you try to ask, you know, ChatGPT or Claude whether, like, how to make the bird flu more infectious or something, it will refuse to answer. But open-weight models are just maybe a year or two behind in capability," he said.

"And when they are released, even if they are released with safeguards, once you have access to the weights, it's very easy to sort of remove the safeguards," he said.

He advised that with AI getting more advanced by the day, authorities should also shore up their biodefense capabilities.

"So I think we should be, for example, shoring up our biodefense capabilities. I think we should regulate the innate synthesis machines. So at some point, everybody will be able to have access to blueprints for really dangerous new pathogens. And it would be good to have a second line of defence, like the actual ability to translate that digital blueprint into, like, the physical genetic material," he said.

Also read: "Sick Conspiracy": US, China On Same Page On Snubbing AI Slowdown Call

His solution to these problems is to "bring artificial intelligence to our side".

"I think the solution has to involve aligning these AI systems so that they're actually on our side... that they want things to go well for us. Even if they get access to a massive drone army and bioweapons, it would still go well for us, because they wouldn't want to harm us. A parent might be super powerful relative to their child. They can outthink them; they have access to the bank account. But that's not the risk to the child, right? Because the parent cares about them. So similarly, we need to build AIs that are sort of an extension, I think, of our intentions and are kind of a helper rather than a harmer," he added.

Currently, it is a potential bane for humanity, but it can also prove to be a boon should it align with human goals, he argued.

"It really could save lives on a massive scale by curing diseases, create economic abundance, help alleviate so much misery in the human space and among non-human animals as well. It really could be a big unlock. So I see it as a kind of portal through which humanity should pass. The birth of superintelligence and the great paths to really wonderful futures lead through this portal," he said.

"It is striking to sort of be living through a time where a lot of these things that could be anticipated on theoretical grounds are now playing out in reality, and you're seeing sort of world leaders saying these things and trillions of dollars starting to slosh around, and the global conversation really beginning to wake up to what's ahead. One just has to hope that it ultimately all turns out to be for the best, because it's so hard to figure out. You can get glimpses of insight here and there, but to sort of figure out how all the different considerations add up to a bottom line, I still feel profoundly uncertain and in the dark about that," he added.