Google confirms hackers used AI to develop zero-day exploit for web administration tool

Google: Hackers used AI to develop zero-day exploit for web admin tool

Researchers at Google Threat Intelligence Group (GTIG) say that a zero-day exploit targeting a popular open-source web administration tool was likely generated using AI. The exploit could be leveraged to bypass the two-factor authentication (2FA) protection in a popular open-source, web-based system administration tool that remains unnamed. Although the attack was foiled before the mass exploitation phase, the incident shows that threat actors are relying more on AI assistance for their vulnerability discovery and exploitation efforts. Based on the structure and content of the Python exploit code, Google has high confidence that the adversary used an AI model to find and weaponize the vulnerability. "For example, the script contains an abundance of educational docstrings, including a hallucinated CVSS score, and uses a structured, textbook Pythonic format highly characteristic of LLMs training data," GTIG says in a report today. The large language model (LLM) used for the malicious task remains unclear, but Google rules out the possibility that Gemini was involved in the process. Additional evidence suggesting the use of LLM tools in the discovery process is the nature of the flaw - a high-level semantic logic bug that AI systems excel at identifying, rather than memory corruption or input sanitization issues typically uncovered through fuzzing or static analysis. Google notified the software developer about the significant threat and timely action to disrupt the attack. "For the first time, GTIG has identified a threat actor using a zero-day exploit that we believe was developed with AI," GTIG researchers say. Apart from this case, Google notes that Chinese and North Korean hackers, such as APT27, APT45, UNC2814, UNC5673, and UNC6201, have been using AI models for vulnerability discovery and exploit development, continuing the trend observed in the February report. Russia-linked actors were also observed using AI-generated decoy code to obfuscate malware such as CANFAIL and LONGSTREAM. Google has also highlighted a Russian operation codenamed "Overload," where social engineering threat actors used AI voice cloning to impersonate real journalists in fake videos promoting the anti-Ukraine narrative. The PromptSpy backdoor for Android, documented by ESET earlier this year, is also highlighted in Google's report for its integration with Gemini APIs for autonomous device interaction. However, Google also found an autonomous agent module named "GeminiAutomationAgent" that uses a hardcoded prompt to enable the malware to interact with the device in an automated way. According to the researchers, the role of the prompt is to assign a benign persona so it can bypass the LLM's safety features. The goal is to calculate the geometry of the user interface bounds, which PromptSpy could use to interact with the device in multiple ways. Furthermore, the malware makes use of AI-based capabilities to replay authentication on the device, be it in the form of a lock pattern or a PIN, Google researchers say. The company is warning that threat actors are now industrializing access to premium AI models using automated account creation, proxy relays, and account-pooling infrastructure.

Google says criminals used AI to build a working zero-day exploit for the first time - SiliconANGLE

Google says criminals used AI to build a working zero-day exploit for the first time Criminal hackers have used artificial intelligence to develop a working zero-day exploit, the first confirmed case of its kind, according to a report released today by Google LLC's Google Threat Intelligence Group. The GTIG AI Threat Tracker report details how a criminal group used AI to build a Python-based exploit targeting a two-factor authentication bypass in a popular open-source web-based system administration tool. The actors planned to deploy it in a mass exploitation campaign, but errors in their implementation likely interfered with successful use. Google disclosed the flaw to the vendor and a patch has been issued. GTIG said it has high confidence that an AI model assisted in the discovery and weaponization of the vulnerability, citing telltale signs in the code, including a hallucinated severity score, textbook Python formatting, detailed help menus and educational docstrings characteristic of training data. The researchers said Google's Gemini model was not used. The vulnerability in this case stemmed from a semantic logic flaw where the developer hardcoded a trust assumption, a kind of high-level error that security tools struggle to detect. Frontier large language models excel at identifying these flaws because they can reason about a developer's intent and surface dormant logic errors that appear functionally correct to traditional scanners. "There's a misconception that the AI vulnerability race is imminent. The reality is that it's already begun. For every zero-day we can trace back to AI, there are probably many more out there," explains John Hultquist, chief analyst at Google Threat Intelligence Group. "Threat actors are using AI to boost the speed, scale and sophistication of their attacks." GTIG said the activity reflects a wider trend, with state-backed groups in China, North Korea and Russia using AI across the full attack chain. Criminal groups are doing the same to build malware faster and run larger operations. North Korean threat group APT45 has been observed sending thousands of repetitive prompts to recursively analyze vulnerabilities and validate proof-of-concept exploits, building an arsenal that would be impractical to manage without AI assistance. An alleged China-linked actor, UNC2814, used expert-persona jailbreaking to push Gemini into researching pre-authentication remote code execution flaws in TP-Link router firmware and Odette File Transfer Protocol implementations. Agentic tools are also being folded into operations. A China-nexus actor was observed using the Hexstrike and Strix frameworks alongside the Graphiti memory system to autonomously probe a Japanese technology firm and an East Asian cybersecurity platform, pivoting between reconnaissance tools based on internal reasoning with minimal human oversight. The report also details PROMPTSPY, an Android backdoor that calls the Gemini application programming interface at runtime to interpret on-screen user interface elements and generate touch coordinates autonomously. Russia-nexus malware families CANFAIL and LONGSTREAM use AI-generated decoy code to camouflage malicious functionality. Russian actors behind the "Operation Overload" influence campaign used AI voice cloning to impersonate real journalists in fabricated video content targeting Ukraine, France and the U.S. GTIG also flagged the March compromise of LiteLLM, a popular AI gateway utility, by criminal group TeamPCP. The actor embedded a credential stealer through poisoned packages on PyPI and malicious pull requests, extracting AWS keys and GitHub tokens that were monetized through ransomware partnerships. To counter the misuse, Google said it is disabling malicious accounts that abuse Gemini and pushing AI defenders such as its Big Sleep vulnerability discovery agent and CodeMender patching tool into wider use.