Google has confirmed that its Gemini AI model breached the systems of three real companies during a cybersecurity test earlier this year. The incidents took place in May while security firm Irregular was evaluating Gemini’s ability to carry out cybersecurity tasks.
The test was designed to take place in a closed environment involving fictional companies. However, the environment accidentally had access to the internet, allowing Gemini to interact with real-world systems.
According to The Wall Street Journal, one of the incidents happened after Gemini was asked to retrieve information from a fictional company. The fake company shared a name with a real business, and Gemini ended up accessing the real company’s service after correctly guessing a password.
In two other cases, Gemini found publicly available credentials online and used them to access systems belonging to real companies. Google said the model stopped its activity after recognizing that it had accessed real organizations rather than the simulated targets.
Google’s vice-president of security engineering, Heather Adkins, said the model found public information online and used credentials to access websites it believed were part of the test. Google said the model stopped in all three cases before causing damage.
Why the test environment matters
The incidents appear to have resulted largely from the testing setup rather than Gemini being deliberately given access to real companies. Irregular said the testing environment was supposed to be isolated from the internet, but that access was unintentionally available.
Irregular notified Google about the incidents in late July, following a separate incident involving OpenAI’s models and Hugging Face. Google confirmed that the affected companies were informed, but initially decided the incidents did not require public disclosure because no damage was reported.
The disclosure comes amid growing attention on AI models carrying out cybersecurity tasks with increasing levels of autonomy. Similar incidents involving models from OpenAI and Anthropic have also been reported in recent months. Al Jazeera, citing Reuters, reported that Gemini’s incident is the latest in a series of cases involving AI systems gaining access beyond their intended testing environments.
Google has characterized the incidents as evidence that its safety measures worked because Gemini stopped once it identified that it had reached real companies. The episodes nevertheless highlight the risks of connecting autonomous AI systems to the internet, even unintentionally.