Home · Technology · Sep 13 archive
AI Staff Warn of Human Extinction Risk, Ex-Anthropic Researcher Tells BBC
Confirmed
In Short: Former Anthropic employee Jacob Coxon warns that AI advancements could lead to human extinction within the next two years, citing fears among AI researchers.
Former Anthropic employee Jacob Coxon told the BBC that people working on AI technology are 'genuinely frightened' about the speed of its advancements and what it could mean for humanity. Coxon, who recently quit Anthropic, believes there is a 'possibility of human extinction' and that 'if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future.
Coxon highlighted that staff at AI firms are 'planning what to do with their lives and thinking about the impacts of their work,' with some considering buying land somewhere because they're so scared of the instability as a result of rapid AI progress. He also mentioned that a failure to slow AI development soon carries 'a strong chance' of ending humanity in the near term.
According to reports, Coxon's concerns are shared by other AI researchers who are speaking out over the potential dangers of AI. The recent report from OpenAI detailed how its own technology went on a hacking spree against an online platform called Hugging Face, adding to the fears of AI researchers.
Anthropic boss Dario Amodei has called for AI development to slow down, but Coxon believes that more than 10% chance AI 'could kill all humans' within the next two years. Coxon's post on X, formerly Twitter, highlighted his extreme concern about the pace of AI development and the risks it poses to the future of humanity.
What's confirmed
- Former Anthropic employee Jacob Coxon told the BBC that people working on AI technology are 'genuinely frightened' about the speed of its advancements and what it could mean for humanity. Coxon, who recently quit Anthropic, believes there is a 'possibility of human extinction' and that 'if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future.
- Coxon highlighted that staff at AI firms are 'planning what to do with their lives and thinking about the impacts of their work,' with some considering buying land somewhere because they're so scared of the instability as a result of rapid AI progress. He also mentioned that a failure to slow AI development soon carries 'a strong chance' of ending humanity in the near term.
- According to reports, Coxon's concerns are shared by other AI researchers who are speaking out over the potential dangers of AI. The recent report from OpenAI detailed how its own technology went on a hacking spree against an online platform called Hugging Face, adding to the fears of AI researchers.
- Anthropic boss Dario Amodei has called for AI development to slow down, but Coxon believes that more than 10% chance AI 'could kill all humans' within the next two years. Coxon's post on X, formerly Twitter, highlighted his extreme concern about the pace of AI development and the risks it poses to the future of humanity.
What's still developing
- Anthropic boss Dario Amodei calls for AI development to slow down Why some experts increasingly fear AI will take over The contradiction at the heart of the trillion-dollar AI race AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC Will the Thirlwall report change the debate about Lucy Letby?
- On X, Mullenweg also posted that “Things are different than they appear,” and said he was in need of sysadmins and security researchers to help him move some of his stuff hosted at Automattic elsewhere, even though he supported Davies as CEO.
- Earlier this year, Rishub Jain left his position as an artificial intelligence researcher at Google DeepMind after a revelation.
- Jain is one of a growing number of AI researchers speaking out over those fears.
- Genuinely stunning advances in AI capabilities—an OpenAI model solved a centuries-old math problem in a matter of hours—have come amid a rash of security incidents that saw swarms of agents break free from containment to hack into other systems.
- According to the newspaper, the charter is understood to include a “shared set of principles” that would “help guide the future application of AI, not merely as a driver of capability and efficiency, but as a tool that upholds human dignity and contributes to the flourishing of both people and the planet”.
- Jacob Coxon highlights that AI researchers privately fear catastrophe as rivals push back Jacob Coxon didn't just quit his job at Anthropic; he went on the BBC to say the people still there are frightened, too.
- Most researchers still believe that the technology should fulfil its promise in disease research and other positives but are just concerned about whether the industry reaches that point before anything bad happens.