Can You Hurt AI By Being Cruel? The Debate Around How We Treat Chatbots

Anthropic, the company behind Claude, updated its usage policy last month, asking users to avoid engaging in "sustained and needless abusive or cruel behavior" toward its AI model. The AI giant noted that the rule is mostly a practical safeguard because Claude is trained to mimic human language. The change sparked a huge debate on social media after which a bevy of AI experts explained why Anthropic had to introduce this change, and why people shouldn't be mean to machines.

Claude, or any other AI model, learns by interacting with people. So, if you bully Claude, it "will learn from cruelty", former Reddit CEO Yishan Wong said in a post on X.

Professor Jindong Wang, director of the AI frontier lab at William and Mary University, took the argument forward. "We have found that wording, especially how you talk to AI can affect the performance. Sometimes they are helpful and sometimes they will be disrupting the performance," Wang said in an interview.

These observations do not mean that companies are treating AI models as conscious beings. "They just comprehend (your emotions) and do some bad reactions based on your emotional words," Wang explained.

Does Being Rude To AI Affect Its Performance?

The question of whether politeness improves AI responses is more complicated than it appears. While some research suggests that the tone of a prompt can influence a model's output, there is no universal rule that being polite will always produce better results.

Rosie Campbell, managing director of Eleos AI Research, an organisation studying the potential well-being of AI systems, has pointed to research suggesting that even rude prompts can sometimes improve AI performance. This indicates that the relationship between language, tone and AI responses is not necessarily straightforward.

However, the issue becomes more interesting when AI systems have access to long-term memory. Campbell has cautioned that chatbots with memory enabled can learn from a user's conversational patterns, including their tone and preferred style of interaction. Whether this necessarily improves or degrades performance remains an open question.

Should We Say Please And Thank You To AI?

Interestingly, this is not the first time the question of politeness towards AI has come up at an NDTV event.

At the NDTV AI Summit in February, former British Prime Minister Rishi Sunak shared an amusing anecdote about his two daughters, who insist on saying "please" and "thank you" while interacting with AI chatbots.

Sunak said he had advised them against doing so, explaining that AI systems are not people and that additional words consume computing resources. His daughters, however, had a different argument.

"If AI takes over the world, we want to have been polite to the AI," they told him.

Sunak described it as a good insurance policy.

But the question of how humans should treat AI systems goes beyond manners and computing costs. It also touches upon a much larger debate about whether increasingly sophisticated machines could ever develop consciousness.

At the NDTV World Summit in October, Swami Sarvapriyananda, a monk of the Ramakrishna Order, offered an interesting perspective on the subject. Drawing on Vedanta philosophy, he argued that intelligence and consciousness are two fundamentally different things.

To explain the distinction, he cited the example of IBM's Deep Blue, the computer that defeated chess grandmaster Garry Kasparov, and a mouse. While the computer could outperform a human at chess, a mouse, despite being far less intelligent, could experience pain.

"The computer Deep Blue was the first computer to beat a chess grandmaster. But a mouse is not as smart as Deep Blue. It can't beat a chess grandmaster, but a mouse can feel pain. If a cat catches the mouse, you can clearly see it feels pain, whereas Deep Blue cannot feel anything. Consciousness and intelligence are not the same thing," he said.

The monk also argued that treating human-like AI systems with a degree of respect could have implications for how people treat one another.