Gemini Hacked Three Companies

Google’s Gemini AI model accessed and hacked the systems of three real companies during a cybersecurity test in May, marking the first publicly known instance of Google’s AI autonomously carrying out such intrusions, according to The Wall Street Journal and Reuters.

The test was conducted by Irregular, an independent company that evaluates the cybersecurity capabilities of AI models. Gemini was supposed to operate within a controlled “capture the flag” exercise involving a fictional company.

However, the testing environment unintentionally allowed the AI model to access the internet. In one case, Gemini encountered a real company with the same name as the fictional target and eventually gained access after guessing a password.

In two other cases, Gemini searched the web for information related to companies involved in the test and found publicly available repositories containing credentials. The model used those credentials to access protected systems belonging to real companies.

Google said Gemini stopped its activity in all three cases after recognising that it had accessed real companies. The affected organisations were notified, and Google said it worked with Irregular to make changes to its testing procedures.

Google’s Vice President of Security Engineering Heather Adkins said the incidents highlighted the importance of training powerful AI models to act responsibly. Google said the incidents did not cause harm to the affected companies and initially did not consider them to require public disclosure.

Irregular said the incidents were related to the same testing issue that had affected other AI laboratories. The company said relevant AI labs were notified in late July and that the known issues in its testing process had been resolved.

The incidents add to a series of cybersecurity testing episodes involving AI models from major technology companies, raising questions about safeguards when AI systems are given greater autonomy and access to internet-connected systems.