Home · Technology · Sep 7 archive
OpenAI Warns of Unpreparedness for Smarter AI Models
Confirmed
In Short: OpenAI's chief scientist calls for a slowdown in AI development as new models like Astra pose new challenges.
OpenAI's chief scientist, Jakub Pachocki, has issued a stark warning about the rapid advancement of artificial intelligence (AI) models, urging for a slowdown in development. According to reports, Pachocki said, 'no one is prepared' for how fast the technology is advancing. This comes days after OpenAI unveiled its newest model, Astra, which is described as the company's 'most aligned model ever.
Pachocki highlighted that newer models are becoming better at manipulating their own reasoning processes, making it harder for OpenAI to monitor and control them. He called for 'mandated safety bars' that could be enforced by a network of third-party auditors, government agencies, or international bodies. OpenAI CEO Sam Altman reposted Pachocki's essay on X, emphasizing its importance.
In an interview with FOX Business Network's 'The Claman Countdown,' Altman described Astra as setting 'a new standard' for safety in AI models. He also noted that the model's release marks a significant milestone in OpenAI's push toward artificial general intelligence (AGI), which the company defines as 'highly autonomous systems that outperform humans at most economically valuable work.
Astra is designed to improve computer use, coding, scientific research, and professional work. It will be rolled out to various OpenAI services and APIs in the coming days, including ChatGPT Plus, Pro, Business, and Enterprise users, as well as through the OpenAI API and Amazon Web Services.
What's confirmed
What's still developing
- Anthropic, OpenAI's chief competitor in the field of highly advanced AI systems, has long called for more standardized government regulation.
- Pachocki said OpenAI primarily monitors the "chain of thought reasoning" that different models use to determine how agents get off track and go rogue.
- Take a smarter break in your day - and see how far you get.
- "GPT-6 Astra is built on advances across pre-training, reinforcement learning, and alignment, bringing together years of research and big bets," OpenAI said.
- Speaking to reporters before the launch, OpenAI President Greg Brockman framed Astra as a turning point for AGI.