- Amanda Askell, philosopher at Anthropic, teaches AI Claude right from wrong using ethics and safety principles
- Askell created a 30,000-word constitution to guide Claude's actions on safety, ethics, and helpfulness
- She emphasizes AI's need for empathy, self-awareness, and understanding difficult ethical queries
Amanda Askell, a renowned philosopher, has been entrusted with the monumental task of teaching Anthropic's artificial intelligence (AI) chatbot, Claude, the difference between right and wrong. AI is evolving at an unprecedented pace, but the growth also raises concerns about the moral reasoning of these systems, especially when it comes to sensitive topics like religion, war, mental health, and more
Who is Amanda Askell?
As the resident philosopher at Anthropic, the 37-year-old is crafting Claude's personality and moral compass. She is making sure that the AI system is safe, ethical and helpful to users, according to The Wall Street Journal (WSJ).
Askell is originally from rural Scotland. She is an Oxford-educated philosopher with a PhD from New York University. Her work focuses on ethics, decision theory and formal epistemology. Before joining Anthropic, she worked at OpenAI on AI safety and policy.
Also read | AI Uses NASA Images To Locate Luna 9, Historic Soviet Spacecraft Missing Since 1966
"There is this human-like element to models that I think is important to acknowledge," As quoted during an interview. She further stated that AI will "inevitably form senses of self".
Askell's approach involves learning Claude's reasoning patterns and engaging in deep conversations to develop its emotional intelligence. She aims to enable the system to understand difficult queries and give politically correct and ethical answers.
She's created a 30,000-word "constitution" guiding Claude's actions, emphasising safety, ethics and helpfulness. Askell believes AI models will inevitably form "senses of self", making it crucial to teach them empathy and self-awareness. "We want Claude to know that it was brought into being with care," it reads.
Also read | AI Agents To Invade Social Media, Spread Narratives And Destroy Democracy, Study Warns
"World is in peril"
As per her LinkedIn profile, Askell has been a Member of Technical Staff at Anthropic since 2021, but there is buzz about her role after a shocking resignation of the company's head of the Safeguards Research Team, Mrinank Sharma, who said that the "world is in peril", not just from AI or bioweapons, but from a "whole series of interconnected crises unfolding" currently.
Not just Sharma, Zoe Hitzig, a researcher at OpenAI, also resigned recently, expressing "deep reservations" about the company's strategy. Two employees from two leading AI firms issued public warnings about where the AI companies are heading, sparking huge debate over AI's ethics and governance.
Askell said that she understands the fears and worries about AI. "In some ways this, to me, feels pretty justified," she said as quoted. "The thing that feels scary to me is this happening at either such a speed or in such a way that those checks can't respond quickly enough, or you see big negative impacts that are sudden."
As AI systems become increasingly integrated into daily life, it is critical to ensure they align with human values.
