"I Worry Something Will Go Wrong": Anthropic CEO On AI

"I worry that something will go wrong," Anthropic co-founder and CEO Dario Amodei said while discussing rising concerns around AI and its impact on society.

"Are we doing literally everything we can? We're certainly trying our best," Amodei - who's extremely vocal about fears that AI could get out of hand - said in a Bloomberg documentary.

According to a Reuters poll, more than 50% of Americans fear that the rise of AI could put them or someone in their family out of work. The survey showed that this fear is spread fairly evenly across respondents by age, gender and education level.

Amodei, an outlier among AI leaders, constantly talks about approaching rapid artificial intelligence with caution. In fact, he's even called for all AI companies across the world to come together and pause or slow down development until societal implications are fully understood. However, he remains skeptical that such an understanding can be mutually arrived at.

Amodei has also been increasingly highlighting the need for stricter governance of frontier AI systems through mandatory safety checks before deployment.

Also read: What Happens Inside AI's Mind Is "Mysterious, Unsettling": Anthropic Co-Founder

In the Bloomberg documentary, responding to whether AI could be banned at some point, Amodei said it's "possible."
"The main way I could see AI being banned or blocked is if something really went wrong. And if something really went wrong, then maybe it deserves to be," he said. He highlighted this again in an essay on Wednesday.

The former OpenAI employee who left the company over disagreements around "safety" to start Anthropic has said that, "when you feel that you can't trust someone, when you feel that their values are not what they say they are, when you feel that they are not honest, that makes it very hard to continue to work with a company, to continue to trust the company."

OpenAI has seen several high-profile exits, including co-founders and senior researchers, over concerns that the company was putting commercial interests and rapid product ahead of rigorous AI safety. In 2024, Altman had publicly apologised and said the company strives to do better on the safety front. Earlier this week, Altman announced that OpenAI's third phase will focus on safe, accessible AI, moving beyond breakthroughs to real-world impact.

In an official blog post co-authored by Altman and OpenAI's chief scientist, Jakub Pachocki, released on Monday, the duo said they were "clear-eyed about the risks" of AI, despite the productivity benefits it brings. "Entirely automating everything is not the future we want," they wrote. "It would be unfulfilling, and it would be dangerous."

Altman's house was recently attacked once with a Molotov cocktail and once with gunfire. The Molotov attacker was an anti-AI protestor. However, it has not been established what the motive of the gun attackers was.

Amodei Wants Governments to Be Able to Block AI

In the policy essay published on Wednesday, Amodei argued that governments should have the authority to block the deployment of the most advanced AI systems if independent evaluations find them to pose unacceptable risks. He said that voluntary transparency from AI companies is no longer enough because "the risks are clearly here."

Amodei said that frontier AI systems should undergo mandatory independent testing before deployment, especially for high-impact risks such as cybersecurity vulnerabilities, biological misuse, and loss of human control. If systems fail these evaluations, governments should be able to pause or even block their release, he said.

Also read: Humans May Lose Control As AI Starts Building Itself, Warns Anthropic

He framed this as a structural mismatch between technology and governance - where AI capabilities are advancing on an exponential curve, while regulation and institutional response move far more slowly, often over years.
"In addition to transparency, I now believe frontier models should face mandatory third-party testing for cyber, bio, and autonomy risks - with the power to block or revoke deployment of models that pose catastrophic risk," Amodei separately said on X (formerly Twitter).

The proposal also shifts the focus away from post-deployment regulation toward pre-deployment safety gating, similar to how drugs or aircraft systems are tested before they are released to the public.

The essay reflects a broader concern that AI capabilities are advancing far faster than governments can formulate policy, making reactive regulation insufficient for a technology that could reshape economies and national security within a matter of months.

Back in India, Union IT Minister Ashwini Vaishnaw on Wednesday said that a dedicated new legal framework is required to regulate Artificial Intelligence in India. Emphasising a balance between innovation and citizen safety, he stated that the original IT Act from 2000 is unsuited for the modern AI landscape.