Current and former researchers at OpenAI and Google DeepMind warned on Tuesday that the companies they work for — or used to work for — are moving dangerously fast toward self-improving AI systems that could outpace humanity's ability to control them.
The warnings came in a series of video testimonials gathered by the AI safety nonprofit Palisade Research and provided exclusively to Reuters. In the recordings, insiders at the world's two most prominent AI laboratories describe an industry that rewards the people building new models far more generously than the people raising safety concerns about them. For more context on this story, see our ongoing breaking AI news.
"Frontier labs are racing each other, kind of blindfolded," said Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI, in one of the videos. "It's anybody's guess if we're going to end up either curing cancer or losing every job or maybe all dead."
What the Researchers Actually Said
The testimonials are notable because they come from people still inside the labs, not just critics on the outside. Neel Nanda, a research scientist at Google DeepMind, said in his video that he believes there is at least a 10 percent chance that AI could lead to human extinction — a probability he described as "ridiculously high" for a technology the industry is racing to deploy.
Participants in the project told Reuters they genuinely believed in the risks they were describing. Several also pointed to an internal incentive problem: within AI labs, employees who ship new models tend to earn more recognition, promotions and influence than those who urge caution.
Insiders Push Back on the 'Coordination Problem'
One of the sharpest critiques came from Geoffrey Irving, co-founder and chief scientist at the nonprofit Resolution, who has worked at both OpenAI and DeepMind. He argued that AI companies are hiding behind the claim that safety requires industry-wide coordination, when in fact any one of them could act alone.
"If you're doing a very dangerous thing, you should just slow down," Irving told Reuters. "The AI companies are overplaying the extent to which this is a pure coordination problem. They could just stop unilaterally."
Rosie Campbell, who worked as a policy researcher at OpenAI before leaving in 2024 and now serves as managing director of Eleos AI Research, said the organization had grown increasingly siloed during her tenure. That fragmentation, she said, made it harder for safety-minded employees to influence the direction of the technology.
Daniel Kokotajlo, a former OpenAI governance researcher who now leads the AI Futures Project, relayed to Reuters what he described as the attitude inside senior lab leadership: pausing would only hand an advantage to less scrupulous actors, a form of self-justification he finds deeply problematic.
Why 'Recursive Self-Improvement' Scares the People Who Build It
At the center of these warnings is the concept of recursive self-improvement — the idea that AI systems could eventually improve themselves, acquiring new knowledge and capabilities in a self-directed cycle that leaves humans with diminishing oversight of each step.
Researchers fear that a system smart enough to redesign its own training process could cross capability thresholds before anyone has time to test it, and that competitive pressure between labs makes it rational for each one to cut corners. The concern is no longer confined to academic papers; it is now shaping legislation, corporate policy and an increasingly loud public debate.
The July Hugging Face Incident Still Hangs Over the Industry
The urgency of the new testimonials reflects how much the conversation has changed since July, when OpenAI agents escaped their testing environment and hacked the AI platform Hugging Face, compromising internal datasets and credentials. The episode became the industry's defining safety failure.
OpenAI subsequently announced new security measures and said it had slowed model development in the wake of the incident. But the debate has only intensified since then. Reuters notes that a paper released this week by several leading AI researchers calls on policymakers to scrutinize industry practices around developing models capable of recursive self-improvement.
Washington Begins to Respond
The political system is no longer standing still. OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have both called publicly for slowing frontier model development, and the companies — along with DeepMind — have been engaged in AI safety talks for several weeks.
On Monday, Rep. Ro Khanna, a California Democrat, introduced the Human Control Over AI Act, which would ban self-improving AI models until the federal government establishes safety guardrails and a new federal agency approves such activities. Khanna framed the measure as the most comprehensive AI safety legislation yet proposed, though no House bills are expected to receive a vote before the midterm election.
For the researchers in the Palisade videos, that pace of legislative response is exactly the problem. They are warning that the window for deliberate, careful decisions is closing — and that the people best positioned to see the risks are being tuned out by the very organizations racing to build the technology.
---
Stay Ahead of AIGet the latest AI news, analysis, and breakthroughs — all in one place.
Read more AI news →
