An artificial intelligence researcher, Jacob Coxon, has resigned from Anthropic, warning that the rapid development of increasingly powerful AI systems by major technology companies could pose an existential threat to humanity.
Coxon, who has conducted AI safety and retraining research at both OpenAI and Anthropic over the past three years, said he no longer believes the companies are developing the technology responsibly.
In a lengthy post on X, Coxon said AI companies were effectively racing towards self-improving superintelligence without adequately addressing the risks associated with systems that could eventually surpass human capabilities.
“They are racing straight to self-improving super-intelligence and gambling with our lives,” he wrote.
Coxon warned that the capabilities of advanced AI systems should not be underestimated, arguing that they could soon become capable of hacking computer systems, transforming entire fields and acquiring significant real-world power and resources.
He further claimed that some of the people building advanced AI systems genuinely believe the technology “could kill us all by the end of the decade,” while expressing concern that progress across key areas of AI development is showing no sign of slowing.
The researcher questioned why AI companies continue to accelerate development despite the capabilities already demonstrated by computer intelligence, suggesting that the industry may be increasingly “locked in a race to get there first.”
Coxon also urged AI researchers to think carefully about what the next few years could look like, particularly before deploying systems capable of rapidly improving themselves.
He questioned whether researchers would be comfortable launching a highly advanced reinforcement-learning system without first having a sufficiently thorough understanding of how the system’s underlying processes and behaviour work.
His resignation and warning come amid growing concerns from prominent technology and political figures about the consequences of accelerating AI development.
The caution marks the third major warning about AI this week, following comments from OpenAI chief scientist Jakub Pachocki and French President Emmanuel Macron.
Pachocki warned that increasingly powerful AI could potentially break into computer systems and influence human minds, stressing that society may not be prepared for the consequences of a continued rapid increase in machine intelligence.
Macron, speaking at the Lumière Summit in Saint-Paul-de-Vence, also cautioned against developing AI specifically to replace professionals in certain occupations, describing such an approach as a potentially “tremendous mistake.”
The warnings have added to the wider debate over whether the race to develop increasingly capable AI systems is moving faster than efforts to understand, regulate and control their potential consequences.