Home · Technology · Oct 6 archive
NYC Council Holds Hearing on AI Risks, With Divergent Views
Developing
In Short: At a New York City Council hearing on Monday, October 5, policymakers examined legislative proposals to regulate AI tools, including requiring companies to disclose information about AI systems and allowing whistleblowers to receive a portion of fines from AI companies that violate laws.
Former Anthropic researcher Jacob Coxon, who recently quit his job over concerns about AI leading to human extinction by the end of the decade, reiterated these warnings during the hearing.
Coxon expressed his fears, stating, “On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction.”
Representatives from tech companies such as OpenAI, Google, Anthropic, and Meta also testified at the hearing.
OpenAI’s Morgan Dwyer shared, “I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared.”
Logan Graham from Anthropic emphasized the company’s support for “smart regulation” and the role of state and local governments in developing AI frameworks.
Coxon and others also highlighted the potential catastrophic consequences of AI if not properly managed. Coxon added, “I don’t know — I also don’t think it matters whether it’s 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable. We should not train models that we cannot make an extremely strong case that we can keep under human control.”
The hearing comes amid growing global efforts to strengthen AI safety, with policymakers in the US and other countries exploring similar measures.
The companies present assured the Council that their systems are already improving people’s daily lives and that safety is their top priority.
What this adds
The Council for the Indian School Certificate Examinations (CISCE) is launching a separate initiative on student mental health, unrelated to the AI hearing.
What's still developing
- The hearing comes at a time when the Council is examining legislative proposals to require companies to disclose information about AI tools, require third-party approval for AI systems in the city, and allow whistleblowers to receive a portion of fines from AI companies that violate laws.
- “AI is powerful enough that getting this wrong has real consequences, for individuals, institutions, public trust. We have to work relentlessly to do the science of mitigating the risks,” he said.
- The hearing was held to discuss the risks that AI poses to humanity.
- The Council for the Indian School Certificate Examinations ( CISCE ) will launch a three-tier national initiative on student mental health.