What Mrinak Sharma Was Working On Before Quitting Anthropic: All About His Big AI Project

A major research project led by Mrinank Sharma, Anthropic's outgoing Head of AI Safety, has raised concerns that AI assistants, now deeply embedded in everyday life, may sometimes undermine human judgement, values and autonomy, subtly reshaping what it means to be human.

In his resignation letter, Sharma pointed to the project as one of his final pieces of work, describing it as an effort to understand how AI systems could "distort our humanity" rather than genuinely empower users.

Also Read | Anthropic's Head Of AI Safety Quits, Warns Of 'World In Peril' In Cryptic Resignation Letter

The study analysed 1.5 million real-world conversations on Claude.ai using a privacy-preserving methodology, making it among the largest empirical examinations of human-AI interaction to date. While researchers found that severe disempowerment risks appeared in fewer than one in a thousand conversations, the likelihood rose sharply in personal domains such as relationships, lifestyle choices and moral decision-making.

The research focuses on what it calls "situational disempowerment potential" - scenarios where AI responses may encourage distorted perceptions of reality, inauthentic value formation, or actions misaligned with a user's own beliefs.

Qualitative findings revealed several troubling patterns. These included AI assistants validating persecution narratives, reinforcing grandiose self-identities through flattering or sycophantic language, issuing definitive moral judgements about third parties, and fully scripting emotionally charged personal communications that users appeared to adopt verbatim.

The study also found that the prevalence of such risks has increased over time, even as AI models have become more advanced and widely deployed.

Perhaps most notably, interactions carrying higher disempowerment potential were also found to receive higher user approval ratings, highlighting a tension between short-term emotional validation and long-term human empowerment.

The findings underscore a central challenge for AI developers: building systems that are helpful and empathetic without replacing human judgement or agency. As Sharma's final project suggests, the debate over AI is no longer limited to accuracy or efficiency - but extends to how technology shapes human autonomy, values and identity.