Home · Technology · Sep 14 archive
‘We need more time’: Another researcher quits over AI risks, this time from Google DeepMind
Developing
In Short: AI safety researchers from Anthropic and Google DeepMind have resigned, raising alarms about the potential risks of rapidly advancing technology.
More AI researchers are quitting their positions due to safety concerns, according to reports. Jacob Coxon, who recently left Anthropic, and Josh Engels, who resigned from Google DeepMind, have joined a growing chorus of experts warning about the potential for advanced AI systems to cause harm. Coxon, in a viral post on X, expressed his fears about the rapid pace of AI development and the risks it poses to humanity's future. Engels echoed these concerns, stating that increasingly capable AI systems could cause immense harm if safety efforts do not keep pace.
Lawmakers are also taking notice of the growing concerns. Rep. Lori Trahan, a Massachusetts Democrat, highlighted the issue in a tweet, noting that safety researchers are resigning and powerful AI models are breaking out of their labs. 'Companies are racing ahead anyway,' Trahan wrote, emphasizing the need for transparency and regulation. The calls for a slowdown in AI development have come from various quarters, including Anthropic's Dario Amodei, who has called for coordinated efforts to manage risks.
President Trump, however, has downplayed the risks posed by AI, stating that 'negative forces' are bringing up concerns that won't come to pass. This stance contrasts with the growing alarm among AI researchers and lawmakers. In interviews with NBC News, Joe Benton and Josh Engels, who previously worked on AI safety research at Anthropic and Google respectively, expressed their concerns about the lack of transparency and the need for public awareness. 'At the minute, basically all of the transparency about these risks that is coming from the companies is entirely voluntary,' Benton said.
The resignations of Coxon and Engels have sparked a broader conversation about the risks of AI. They join other researchers who have left their positions due to safety concerns, including former Anthropic researcher Jacob Coxon. Coxon's post on X highlighted his fears about the potential for AI systems to spiral out of control and potentially kill all humans. 'People are trying their best, but there is no one coming to save us,' he said in his post.
Meanwhile, the calls for a coordinated approach to managing AI risks have gained traction. Altman, CEO of Anthropic, has written about the need for a 'narrow middle path' to avoid the dangers of AI. He emphasized the importance of ensuring that AI always serves people and that safety techniques stay ahead of model capabilities. 'First we could lose control of the future to AI. This is unacceptable; we are unapologetically on Team Humanity, and AI must always serve people,' Altman wrote.
What's still developing
- Trump spoke on the same day AI researcher Jacob Coxon - who quit Anthropic a few days ago - told the BBC that staff who were developing the systems were "genuinely frightened" for the future of humanity.
- The weekend call by major AI executives sparked discussion in Washington, where lawmakers are facing pressure to address those potential risks around rapid AI developments.
- AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC Anthropic boss Dario Amodei calls for AI development to slow down Why some experts increasingly fear AI will take over Dramatic insider warnings over AI fall flat with some in Silicon Valley Tech stocks fall after calls for AI development to slow down Tice says government trying to 'retrospectively change rules' as Reform receives £72m in donations Russia hits Ukrainian train shortly after Boris Johnson and top European officials leave station I had 11 years of chemotherapy for a cancer I didn't have Why I'm doing a September reset and how you can create a new routine Heading off to uni?
- “Today, frontier laboratories largely set their own rules for managing frontier risks,” Lehane wrote.
- AI researchers have referenced the possibility that AI systems could potentially begin to improve themselves without human input, creating the potential for AI systems to either purposely or incidentally kill humans.
- "Avoiding these two threats requires walking a narrow middle path; for example, one country could gain too much power. Another example is one lab ending up with too much power," Altman wrote.
- Days after an Anthropic researcher quitting his job snowballed into a global conversation about AI-driven human extinction, another AI scientist from Google DeepMind has announced his resignation over similar concerns.
- Engels said researchers do not yet know how to ensure such systems remain sufficiently aligned with human intentions as their capabilities improve.
Sources
- The Indian Expresslink