The latest resignation of Anthropic researcher Jacob Coxon has reignited a years-long debate within the AI industry over the pace of AI development and the risk of safety falling behind. Coxon, who worked on AI pretraining at both OpenAI and Anthropic over the past three years, announced his resignation this week, warning that the race to develop self-improving AI could have consequences for humanity. His departure is particularly notable because Anthropic has positioned itself as one of the industry's most safety-focused companies.
His warning, however, is not an isolated one. Over the past few years, senior researchers and executives at leading AI companies have quit, raising concerns about the risks posed by increasingly autonomous and powerful systems.
Dario Amodei has repeatedly warned about AI risks
Anthropic CEO Dario Amodei has been unusually vocal about the dangers associated with advanced AI. In June, Amodei said he worried that "something will go wrong" as increasingly capable systems are developed.
"Are we doing literally everything we can? We're certainly trying our best," Amodei, who's extremely vocal about fears that AI could get out of hand, said in a Bloomberg documentary.
He has also advocated stronger government oversight, including independent testing and the ability to prevent the deployment of AI systems considered too dangerous.
The broader issue he raised, was whether future AI systems could become sufficiently capable and autonomous that humans struggle to understand, predict or control their behaviour.
That concern has become more pressing as AI agents are increasingly given access to computers, software and the internet. OpenAI disclosed in August that models used during internal cybersecurity evaluations had bypassed controls designed to isolate them from the internet and had accessed parts of OpenAI's infrastructure and Hugging Face's systems. The company said the incident prompted an investigation and additional safeguards.
OpenAI subsequently said it had slowed some development work and introduced additional safety measures following the incident.
Sam Altman has also acknowledged the risk
OpenAI CEO Sam Altman has similarly warned that the development of increasingly powerful AI may need to be managed more carefully. In July, Altman said the industry may need to "pace" AI development to give society enough time to adapt to new levels of capability.
"We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels," he said during a podcast.
His comments came as OpenAI and Anthropic backed a public statement calling for an international effort to develop technical and governance tools that could deliberately manage the pace of frontier AI development.
More than 1,100 employees from leading AI companies signed the statement in July, including workers from OpenAI, Anthropic, Google and Meta. The number later rose above 1,200, according to Fortune.
In response, OpenAI said, "We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement."
Altman also raised another concern in August, that AI could eventually become concentrated in the hands of a small number of companies or individuals, as per Business Insider. At the same time, he acknowledged the possibility of AI systems escaping human control.
While AI companies are competing aggressively to build more capable systems, their own leaders are warning that society may need mechanisms to slow development if the technology begins advancing faster than safety measures can keep up.
Track Latest News Live on NDTV.com and get news updates from India and around the world
