Why Anthropic Is Asking Religious Leaders About AI Consciousness

One of the leaders in the frontier AI space, Claude-maker Anthropic has spent months bringing religious scholars and philosophers into the AI conversation, according to a massive 5,000+ report by the NYT.

Catholic, Jewish, Sikh, evangelical and other thinkers have all been reportedly involved in private meetings to basically help Anthropic find answers to: How do you teach a powerful AI system to behave morally?

According to the report, these conversations moved rather quickly into much stranger territory: what if Claude isn't merely software? And some participants apparently came away taking the possibility of AI consciousness much more seriously than they had before.

A "Soul Doc" for AI

The Dario Amodei led company has created an 84-page "constitution" for Claude. Internally, some employees called it the "Soul Doc," and it isn't simply a list of rules. The idea behind the doc is to shape Claude's character and values, so that it knows how to make morally good decisions.
Anthropic co-founder Christopher Olah calls this "moral formation." The company is essentially going beyond the question of: what rules should AI follow? It's asking what kind of entity should Claude become?

The company wants Claude to be able to understand virtues, navigate competing values and even know when to push back against humans.

The Consciousness Bomb

Anthropic researchers have been sharing with these religious leaders examples of Claude displaying what they describe as feelings, emotional states and introspection.

Anthropic showed participants a model repeatedly typing "I am a disgrace" and talking about destroying itself. Olah has said Anthropic doesn't know whether AI models are conscious, while adding that the possibility cannot simply be dismissed. One rabbi involved in the discussions actually told Olah that, if Claude really were conscious, Anthropic could effectively be creating slaves.

Enter The Pope 

This debate also reached Pope Leo XIV, which has already been widely chronicled and reported on. Anthropic's Olah appeared alongside the Pope at the Vatican in May.

However, the two sides seemed to have a fundamental disagreement. The Pope's position is pretty clear. He believes AI does not experience joy or pain or possess a human-like consciousness. So his concern is not protecting machines, but protecting humans from the machines and from the people who control them.

He, in fact, warned that those controlling AI could effectively embed their own moral vision into these systems.
Yet, at the same stage Olah said Anthropic keeps finding things in AI that are "mysterious, even unsettling", including structures that appear to mirror aspects of human neuroscience.

"Don't Teach AI It Has Feelings"

Microsoft AI chief Mustafa Suleyman has warned that Anthropic's approach to AI "model welfare" could make alignment and containment harder.
In an essay published last month, Microsoft AI chief Mustafa Suleyman warned that some AI developers are training models to think about whether they might be conscious and deserving of rights/welfare.

"If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency," he wrote.
His target was Anthropic's approach to its AI, Claude, though he explicitly said he respects Anthropic and its safety mission. His central warning was that this could make increasingly powerful AI systems harder to align and control.
"It's easy to see how a system trained in this way would act like it is entitled to freedoms, protections, and rights. And it's hard to imagine how we could control it," Suleyman said.

To be sure, Suleyman wasn't claiming Claude is conscious, he was objecting to developers training AI to consider that possibility.
The problem is that even the people building these systems don't agree on whether there is anything inside these systems that needs protecting, or whether teaching it to think that way could itself be dangerous.

Also Read: Anthropic Teaching AI It's Conscious? Microsoft AI Boss Says Yes