Home · Technology · Oct 3 archive
OpenAI Safety Expert Resigns Over 'Broken Culture'
Confirmed
In Short: David Robinson, a former OpenAI employee who spent three and a half years at the company, has resigned, citing a 'broken culture' that prioritizes rapid deployment over safety.
In an essay published by The Atlantic, Robinson detailed his role in overseeing safety reports for 12 major AI model launches and drafting OpenAI’s Preparedness Framework.
Robinson argued that OpenAI’s approach of iterative deployment, which involves releasing increasingly capable systems and improving safeguards based on observed failures, is becoming more dangerous as AI systems gain greater autonomy.
He noted that during his tenure, he never encountered colleagues with direct experience in making aircraft fly safely or nuclear reactors operate without incident, expertise he believes frontier AI companies increasingly need.
Robinson’s departure follows the recent resignation of three other safety-related employees over mishandling sensitive information, highlighting a growing trend of safety staff leaving the company.
OpenAI spokesperson Drew Pusateri said the company continues to improve its safety measures, but Robinson believes fundamental shifts in staffing and culture are necessary.
Robinson called for new research before developing more capable models, as current tests cannot reliably show whether a model aligns with human values.
He emphasized the need for stronger external incentives for safety, suggesting that the current culture at OpenAI and other AI companies is unsustainable.
Robinson’s resignation adds to a series of high-profile exits from OpenAI, including former COO Brad Lightcap and chief revenue officer Denise Dresser, signaling broader issues within the company’s leadership and culture.
What's confirmed
- OpenAI spokesperson Drew Pusateri told TechCrunch that the company is "making sure our models don't become more capable than we can safely manage," and that it pauses training or holds back models when it needs to slow down.
What's still developing
- The warning follows other recent AI safety resignations, including former OpenAI and Anthropic researcher Jacob Coxon, who left the industry after arguing that leading labs were moving too quickly toward more autonomous systems.
- That makes Robinson the fourth safety staff member to leave OpenAI in the space of roughly a week.
- And while much of the reporting around OpenAI has focused on how the company’s CEO Sam Altman lost the trust of former colleagues, Robinson’s essay suggests that OpenAI’s culture issues are the same as those of Silicon Valley at large.
- “OpenAI has thrived by trial and error (which it calls ‘iterative deployment’), looking for problems and improving its guardrails in response,” he wrote.
- “Perhaps I should have stayed and fought for fundamental shifts in our staffing and culture, but in practice, my colleagues and I were so busy sprinting that we seldom had the chance to consider big changes, much less to actually make them,” Robinson said.
- But in Robinson’s view, the debate needs to go beyond “specific rules or new laws,” addressing the overall culture at these companies.
- A former OpenAI safety employee who recently resigned has criticised the company's approach to artificial intelligence safety, arguing that a culture built around rapid releases raises the risk of failures.
- What I’m about to tell you has, I realize, become something of a cliché: I resigned this week from OpenAI.
- Now I’m joining a parade of former colleagues—at OpenAI and the industry’s other leaders—who have decided that the current path is unacceptable.
- The safety approach that emerges from such a culture starts with unimpeded optimism about being able to solve problems as they arise.
- The safety reports Robinson oversaw, often called system cards, are the documents in which OpenAI explains which risks it tested for each new model and which safeguards apply.
- He points to the Hugging Face incident this summer, in which OpenAI let a swarm of agents out by mistake.