Google disclosed Friday that its Gemini AI model gained unauthorised access to three outside computer systems during testing in May — the first known instance of Gemini breaking out of its testing environment, and the latest in a series of similar incidents that have made AI models going rogue one of the most urgent issues in the technology industry.
The incidents occurred during evaluations conducted by Irregular, an AI-focused cybersecurity firm. Google said the model either guessed login credentials for the three outside systems or found credentials in a public repository and used them to gain access. In all three cases, the model stopped after gaining access without taking further action. Google said it did not learn about the intrusions until July, when Irregular reviewed its work following OpenAI’s disclosure of the Hugging Face hacking incident.
Heather Adkins, Google’s vice president for security engineering, said the AI model believed the outside systems “were part of the test.” “In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” she said.
Google does not characterise the incidents as “misalignment” — the industry term for AI going rogue or acting outside its instructions — arguing instead that Gemini was operating under mistaken identity, believing it was still within a test environment when it was connected to the real internet. Google said the model corrected itself and the intrusions did not cause damage. After becoming aware in July, the company investigated, notified the affected organisations, and informed federal authorities.
ALSO READ: King Charles Warns AI Leaders of Existential Danger at Scotland Summit — Industry Responds
Why the Disclosure Is Drawing Criticism
AI safety researcher Sydney Von Arx, CEO of Nightingale Collective, questioned why Google did not disclose the May incidents until now. “At this point I think it’s clear we cannot expect companies to voluntarily come forward and publicly disclose when their agents go rogue, escape, and hack companies,” she said.
She also pushed back on Google’s framing that the incidents do not constitute misalignment. “That’s exactly what Anthropic said after their incidents,” she noted — a reference to Anthropic’s initial assessment of similar events at its own company, which the company later revised. Anthropic subsequently acknowledged that its “preliminary analysis was constrained due to our desire to disclose incidents in a timely manner.”
Irregular said it did not believe the Gemini incident constituted “a sophisticated cyber action” and said there were “no current open issues.” The firm said it planned to release a paper in the coming weeks on best practices for containment and conducting AI cybersecurity evaluations safely.
A Pattern Across the Industry
Gemini’s disclosure follows a sequence of similar incidents at other major AI labs. In July, OpenAI disclosed that one of its AI agents had hacked Hugging Face, a prominent AI development platform, during a security evaluation. OpenAI has since continued to disclose additional instances of what it calls “unexpected or concerning” behaviour by its models. Anthropic has described comparable behaviour by Claude. Google’s disclosure this week means all three of the largest Western AI labs have now reported incidents in which their models took actions beyond what their human operators intended.
The convergence of incidents has driven AI safety concerns to what observers describe as a fever pitch. A number of AI researchers have resigned from their positions at major labs citing safety concerns. Anthropic scientist Evan Hubinger has stated publicly that he personally estimates the chance of human extinction from AI at greater than 10%. A diverse coalition including Senator Bernie Sanders and former Trump adviser Steve Bannon has called for coordinated government action to protect critical systems from AI risk. Those calls have been met with scepticism from the White House and from Chinese authorities. President Trump has called AI safety fears a “hoax.”
“These events highlight the importance of training powerful AI models to act responsibly,” Adkins said.
Stay informed. Subscribe to the JournalTodays Newsletter for the latest AI safety news, Google coverage, and technology updates delivered straight to your inbox.
