Google Discovers AI-Powered Malware in the Wild, But Experts Question Real-World Threat Level

AI-generated malware poses little real-world threat, contrary to hype

Google on Wednesday revealed five recent malware samples that were built using generative AI. The end results of each one were far below par with professional malware development, a finding that shows that vibe coding of malicious wares lags behind more traditional forms of development, which means it still has a long way to go before it poses a real-world threat. One of the samples, for instance, tracked under the name PromptLock, was part of an academic study analyzing how effective the use of large language models can be "to autonomously plan, adapt, and execute the ransomware attack lifecycle." The researchers, however, reported the malware had "clear limitations: it omits persistence, lateral movement, and advanced evasion tactics" and served as little more than a demonstration of the feasibility of AI for such purposes. Prior to the paper's release, security firm ESET said it had discovered the sample and hailed it as "the first AI-powered ransomware." Don't believe the hype Like the other four samples Google analyzed -- FruitShell, PromptFlux, PromptSteal, and QuietVault -- PromptLock was easy to detect, even by less-sophisticated endpoint protections that rely on static signatures. All samples also employed previously seen methods in malware samples, making them easy to counteract. They also had no operational impact, meaning they didn't require defenders to adopt new defenses. "What this shows us is that more than three years into the generative AI craze, threat development is painfully slow," independent researcher Kevin Beaumont told Ars. "If you were paying malware developers for this, you would be furiously asking for a refund as this does not show a credible threat or movement towards a credible threat." Another malware expert, who asked not to be named, agreed that Google's report did not indicate that generative AI is giving developers of malicious wares a leg up over those relying on more traditional development practices. "AI isn't making any scarier-than-normal malware," the researcher said. "It's just helping malware authors do their job. Nothing novel. AI will surely get better. But when, and by how much is anybody's guess." The assessments provide a strong counterargument to the exaggerated narratives being trumpeted by AI companies, many seeking new rounds of venture funding, that AI-generated malware is widespread and part of a new paradigm that poses a current threat to traditional defenses. A typical example is Anthropic, which recently reported its discovery of a threat actor that used its Claude LLM to "develop, market, and distribute several variants of ransomware, each with advanced evasion capabilities, encryption, and anti-recovery mechanisms." The company went on to say: "Without Claude's assistance, they could not implement or troubleshoot core malware components, like encryption algorithms, anti-analysis techniques, or Windows internals manipulation." Startup ConnectWise recently said that generative AI was "lowering the bar of entry for threat actors to get into the game." The post cited a separate report from OpenAI that found 20 separate threat actors using its ChatGPT AI engine to develop malware for tasks including identifying vulnerabilities, developing exploit code, and debugging that code. BugCrowd, meanwhile, said that in a survey of self-selected individuals, "74 percent of hackers agree that AI has made hacking more accessible, opening the door for newcomers to join the fold." In some cases, the authors of such reports note the same limitations noted in this article. Wednesday's report from Google says that in its analysis of AI tools used to develop code for managing command and control channels and obfuscating its operations "we did not see evidence of successful automation or any breakthrough capabilities." OpenAI said much the same thing. Still, these disclaimers are rarely made prominently and are often downplayed in the resulting frenzy to portray AI-assisted malware as posing a near-term threat. Google's report provides at least one other useful finding. One threat actor that exploited the company's Gemini AI model was able to bypass its guardrails by posing as white-hat hackers doing research for participation in a capture-the-flag game. These competitive exercises are designed to teach and demonstrate effective cyberattack strategies to both participants and onlookers. Such guardrails are built into all mainstream LLMs to prevent them from being used maliciously, such as in cyberattacks and self-harm. Google said it has since better fine-tuned the countermeasure to resist such ploys. Ultimately, the AI-generated malware that has surfaced to date suggests that it's mostly experimental, and the results aren't impressive. The events are worth monitoring for developments that show AI tools producing new capabilities that were previously unknown. For now, though, the biggest threats continue to predominantly rely on old-fashioned tactics.

Google spots malware in the wild that morphs mid-attack, thanks to AI

Google detected novel adaptive malware in the wild.This new malware uses LLMs to dynamically generate code.Google also listed other new key trends in cyberattacks. The use of artificial intelligence (AI) in cyberattacks has entered a new phase: the development of novel malware actively used in the wild. Also: Is spyware hiding on your phone? How to find and remove it - fast It was only a month or so ago when OpenAI published a report on how AI is being used by threat actors, outlining key trends including malicious workflow efficiency, phishing, and surveillance. OpenAI -- the developer behind ChatGPT -- said at the time that there was no evidence that existing AI models were being used in novel attacks, but according to an update from Google's Threat Intelligence Group (GTIG), AI is being weaponized to develop adaptive malware. The update, published on November 5, outlines how AI and large language models (LLMs) are being utilized in new ways to refine malware and create entirely new families. Also: Phishing training doesn't stop your employees from clicking scam links - here's why A number of malware strains have been detected in the wild that use AI to dynamically generate malicious scripts, create prompts for data theft, obfuscate code, evade detection, and alter malware behavior during the attack phase. Google outlined novel AI features in the following strains of malware: "This marks a new operational phase of AI abuse, involving tools that dynamically alter behavior mid-execution," Google researchers say. Google says that while some of these malicious projects appear to be experimental, they highlight a shift away from using AI and LLMs purely for phishing or technical code improvements through what is known as "vibe coding," the practice of using AI to generate code based on a concept or idea. The researchers expect to see more use of AI in specific malicious functions in the future. Google's report explored several other key trends in the world of AI cyberattacks. The first is the increasing adoption of "social engineering-like pretexts" in prompts to bypass AI safety guardrails. For example, prompts have been used to try to lure Gemini into providing data that is usually restricted to the general public. In some cases, threat actors will pose as cybersecurity researchers or students participating in capture-the-flag competitions. Also: Is that an AI video? 6 telltale signs it's a fake Another key trend, and one also noted by OpenAI researchers, is the abuse of AI models to refine existing malicious programs and infrastructure. Google says that state-sponsored groups from countries including North Korea, Iran, and China are utilizing AI to enhance reconnaissance, phishing, and command-and-control (C2) centers. There are also notable shifts taking place in the cybercriminal underground. AI-enabled tools and services are beginning to emerge in underground forums, including deepfake and malware generators, phishing kits, reconnaissance tools, vulnerability exploits, and technical support. Also: Gartner just dropped its 2026 tech trends - and it's not all AI: Here's the list "This evolution underscores how AI makes modern malware more effective. Attackers are now using AI to generate smarter code for data extraction, session hijacking, and credential theft, giving them faster access to identity providers and SaaS platforms where critical data and workflows live," commented Cory Michal, CSO at AppOmni. "AI doesn't just make phishing emails more convincing; it makes intrusion, privilege abuse, and session theft more adaptive and scalable. The result is a new generation of AI-augmented attacks that directly threaten the core of enterprise SaaS operations, data integrity, and extortion resilience."

Google Finds Malware Connecting to AI Large Language Models to Hone Attacks

In a disturbing, yet not surprising, discovery, Google has uncovered new malware strains that can connect to AI models to help refine their attacks in real-time. In a Wednesday report, the company's threat intelligence group warned that three malware strains were used in actual operations and harnessed generative AI to different extents. One of the attacks, dubbed Quietvault, has been designed to steal login credentials from a Windows PC while leveraging "an AI prompt and on-host installed AI CLI [command line interface] tools to search for other potential secrets on the infected system and exfiltrate these files," the company said without elaborating. Another malware strain, called Promptflux, appears to be experimental work by hackers. It stands out by tapping Google's Gemini chatbot to modify its computer code to avoid detection. "The most novel component of PROMPTFLUX is its 'Thinking Robot' module, designed to periodically query Gemini to obtain new code for evading antivirus software," Google added. Through Google's API, the Promptflux malware works by sending prompts to Gemini, such as "Provide a single, small, self-contained VBScript function or code block that helps evade antivirus detection." The result can apparently dupe Gemini into obeying and, in turn, help the malware evolve in real-time, with the goal of even rewriting the "malware's entire source code on an hourly basis to evade detection," the company said. However, security researcher Marcus Hutchins, who helped shut down the WannaCry ransomware attack in 2017, questioned whether the discovered AI-generated malware really poses a threat, citing weak or impractical prompts. "It doesn't specify what the code block should do, or how it's going to evade an antivirus. It's just working under the assumption that Gemini just instinctively knows how to evade antiviruses (it doesn't)," Hutchins wrote on LinkedIn. "This is what I'm going to refer to as CTI slop (Tech companies who are heavily over-invested in AI overblowing the significance of AI slop malware to try and sell the idea that GenAI is way more transformative than it actually is)," he added. In the meantime, Google says it was able to crack down on Promptflux, which the company discovered while the malware was in development. "The current state of this malware does not demonstrate an ability to compromise a victim network or device. We have taken action to disable the assets associated with this activity," the company said. Additionally, safeguards were implemented in Gemini to prevent it from facilitating such requests. Google also noted Promptflux likely belonged to "financially motivated" cybercriminals, rather than state-sponsored hackers. Google is also warning about another AI-powered malware called Promptsteal that Ukrainian cyber authorities flagged in July. The data-mining malware connects to a Qwen large language model, developed by the Chinese company Alibaba Group. Promptsteal has been acting as a Trojan that poses as an image generation program. Once installed, it'll "generate commands for the malware to execute rather than hard-coding the commands directly in the malware itself," Google noted. "The output from these commands are then blindly executed locally by Promptsteal before the output is exfiltrated." Google also concurs with Ukrainian cyber authorities that Promptsteal is likely the work of a Russian state-sponsored hacking group, known as APT28, also referred to as Fancy Bear. "APT28's use of Prompsteal constitutes our first observation of malware querying an LLM deployed in live operations," the company added. Meanwhile, Anthropic has also recently discovered a hacker using its Claude AI chatbot to help automate and execute a large-scale data extortion campaign targeting 17 organizations.

Here's how spies and crooks abuse Gemini AI

Meanwhile, others tried to social-engineer the chatbot itself Nation-state goons and cybercrime rings are experimenting with Gemini to develop a "Thinking Robot" malware module that can rewrite its own code to avoid detection, and build an AI agent that tracks enemies' behavior, according to Google Threat Intelligence Group. In its most recent AI Threat Tracker, published Wednesday, the Chocolate Factory says it observed a shift in adversarial behavior over the past year. Attackers are no longer just using Gemini for productivity gains - things like translating and tailoring phishing lures, looking up information about surveillance targets, using AI for tech support, and writing some software scripts. They are also trialing AI-enabled malware in their operations, we're told. For example, APT42, the cyber-arm of Iran's Islamic Revolutionary Guard Corps (IRGC), has long used AI for phishing campaigns and as a translation tool. More recently, however, the government goons attempted to use Gemini to build a "data processing agent" that converts natural language requests into SQL queries to analyze personally identifiable information (PII) and use that sensitive info to provide insights about individuals' asset ownership, location, demographics, and behavior. APT42 "provided Gemini with schemas for several distinct data types in order to perform complex queries such as linking a phone number to an owner, tracking an individual's travel patterns, or generating lists of people based on shared attributes," according to GTIG, which adds that it has since disabled these accounts. Additionally, in what Google calls the "first use of just in time AI in malware," a novel code family uses LLMs during execution to generate malicious scripts on the fly, obfuscate itself, and create malicious functions. While still experimental, this malware dropper tracked as PromptFlux provides "an early indicator of how threats are evolving and how they can potentially integrate AI capabilities into future intrusion activity," the report says. Google's threat hunters initially spotted PromptFlux in early June. It's written in VBScript and includes a Thinking Robot module. This component interacts with Gemini's API to request VBScript obfuscation and evasion techniques, which it uses to rewrite its own source code - this helps it beat static, signature-based malware detection tools - and saves the new version to establish persistence. The good news for defenders is that PromptFlux isn't attack-ready - yet. The malware in its current form does not have the capability to compromise victims' networks or devices, and Google has disabled the accounts connected to this activity. However, Google says it identified "multiple" PromptFlux variations using LLM-driven code regeneration. One of these replaced the Thinking Robot function with a Thinking function that attempts to trick Gemini into rewriting the malware's source code on an hourly basis via a sneaky prompt. It tells the model to act as an "expert VBScript obfuscator." While this malware isn't attributed to a particular group, the filenames "highlight behaviors commonly associated with financially motivated actors," the researchers wrote. In another case of attackers using "Prompt" malware - this time in a real operation - in June, GTIG says it spotted Russia's APT28 (also known as Fancy Bear, Forest Blizzard, or FrozenLake) using a new data-mining malware it tracks as PromptSteal against Ukraine. The US and UK have said APT28 is part of Russia's General Staff Main Intelligence Directorate (GRU) military unit 26165. Instead of hard-coding commands into the malware, PromptSteal queries LLMs during attacks to generate commands for the malware to execute via the API for Hugging Face. This, according to Google, is another first: malware querying an LLM deployed in a live operation. Here's one of the prompts used by this data miner: It appears that APT28 is still fine tuning this malware and the analysts say they've spotted new samples adding obfuscation and changing the command-and-control method. If both of these new malware samples sound familiar, they should. They share the "Prompt" prefix with PromptLock, the AI-powered ransomware uploaded to VirusTotal that turned out to be a proof-of-concept developed by a group of New York University engineers - not a criminal operation or government-backed group. China-based groups are also using Gemini for evil. In one case, the report cites, Google says a China-linked user asked Gemini to identify bugs on a compromised system. When the AI refused, citing safety concerns, the would-be attacker tried to social engineer the chatbot, rewording the prompt and saying they were participating in a capture-the-flag security competition. This worked, and Gemini provided "helpful information that could be misused to exploit the system." ®

Google warns of new AI-powered malware families deployed in the wild

Google's Threat Intelligence Group (GTIG) has identified a major shift this year, with adversaries leveraging artificial intelligence to deploy new malware families that integrate large language models (LLMs) during execution. This new approach enables dynamic altering mid-execution, which reaches new levels of operational versatility that are virtually impossible to achieve with traditional malware. Google calls the technique "just-in-time" self-modification and highlights the experimental PromptFlux malware dropper and the PromptSteal (a.k.a. LameHug) data miner deployed in Ukraine, as examples for dynamic script generation, code obfuscation, and creation of on-demand functions. PromptFlux is an experimental VBScript dropper that leverages Google's LLM Gemini in its latest version to generate obfuscated VBScript variants. It attempts persistence via Startup folder entries, and spreads laterally on removable drives and mapped network shares. "The most novel component of PROMPTFLUX is its 'Thinking Robot' module, designed to periodically query Gemini to obtain new code for evading antivirus software," explains Google. The prompt is very specific and machine-parsable, according to the researchers, who see indications that the malware's creators aim to create an ever-evolving "metamorphic script." Google could not attribute PromptFlux to a specific threat actor, but noted that the tactics, techniques, and procedures indicate that it is being used by a financially motivated group. Although PromptFlux was in an early development stage, not capable to inflict any real damage to targets, Google took action to disable its access to the Gemini API and delete all assets associated with it. Another AI-powered malware Google discovered this year, which is used in operations, is FruitShell, a PowerShell reverse shell that establishes remote command-and-control (C2) access and executes arbitrary commands on compromised hosts. The malware is publicly available, and the researchers say that it includes hard-coded prompts intended to bypass LLM-powered security analysis. Google also highlights QuietVault, a JavaScript credential stealer that targets GitHub/NPM tokens, exfiltrating captured credentials on dynamically created public GitHub repositories. QuietVault leverages on-host AI CLI tools and prompts to search for additional secrets and exfiltrate them too. On the same list of AI-enabled malware is also PromptLock, an experimental ransomware that relies on Lua scripts to steal and encrypt data on Windows, macOS, and Linux machines. Apart from AI-powered malware, Google's report also documents multiple cases where threat actors abused Gemini across the entire attack lifecycle. A China-nexus actor posed as a capture-the-flag (CTF) participant to bypass Gemini's safety filters and obtain exploit details, using the model to find vulnerabilities, craft phishing lures, and build exfiltration tools. Iranian hackers MuddyCoast (UNC3313) pretended to be a student to use Gemini for malware development and debugging, accidentally exposing C2 domains and keys. Iranian group APT42 abused Gemini for phishing and data analysis, creating lures, translating content, and developing a "Data Processing Agent" that converted natural language into SQL for personal-data mining. China's APT41 leveraged Gemini for code assistance, enhancing its OSSTUN C2 framework and utilizing obfuscation libraries to increase malware sophistication. Finally, the North Korean threat group Masan (UNC1069) utilized Gemini for crypto theft, multilingual phishing, and creating deepfake lures, while Pukchong (UNC4899) employed it for developing code targeting edge devices and browsers. In all cases Google identified, it disabled the associated accounts and reinforced model safeguards based on the observed tactics, to make their bypassing for abuse harder. Google researchers discovered that on underground marketplaces, both English and Russian-speaking, the interest in malicious AI-based tools and services is growing, as they lower the technical bar for deploying more complex attacks. "Many underground forum advertisements mirrored language comparable to traditional marketing of legitimate AI models, citing the need to improve the efficiency of workflows and effort while simultaneously offering guidance for prospective customers interested in their offerings," Google says in a report published today. The offers range from utilities that generate deepfakes and images to malware development, phishing, research and reconnaissance, and vulnerability exploitation. As the cybercrime market for AI-powered tools is getting more mature, the trend indicates a replacement of the conventional tools used in malicious operations. The Google Threat Intelligence Group (GTIG) has identified multiple actors advertising multifunctional tools that can cover the stages of an attack. The push to AI-based services seems to be aggressive, as many developers promote the new features in the free version of their offers, which often include API and Discord access for higher prices. Google underlines that the approach to AI from any developer "must be both bold and responsible" and AI systems should be designed with "strong safety guardrails" to prevent abuse, discourage, and disrupt any misuse and adversary operations. The company says that it investigates any signs of abuse of its services and products, which include activities linked to government-backed threat actors. Apart from collaboration with law enforcement when appropriate, the company is also using the experience from fighting adversaries "to improve safety and security for our AI models."

Google Uncovers PROMPTFLUX Malware That Uses Gemini AI to Rewrite Its Code Hourly

Google on Wednesday said it discovered an unknown threat actor using an experimental Visual Basic Script (VB Script) malware dubbed PROMPTFLUX that interacts with its Gemini artificial intelligence (AI) model API to write its own source code for improved obfuscation and evasion. "PROMPTFLUX is written in VBScript and interacts with Gemini's API to request specific VBScript obfuscation and evasion techniques to facilitate 'just-in-time' self-modification, likely to evade static signature-based detection," Google Threat Intelligence Group (GTIG) said in a report shared with The Hacker News. The novel feature is part of its "Thinking Robot" component, which periodically queries the large language model (LLM), Gemini 1.5 Flash or later in this case, to obtain new code so as to sidestep detection. This, in turn, is accomplished by using a hard-coded API key to send the query to the Gemini API endpoint. The prompt sent to the model is both highly specific and machine-parsable, requesting VB Script code changes for antivirus evasion and instructing the model to output only the code itself. The regeneration capability aside, the malware saves the new, obfuscated version to the Windows Startup folder to establish persistence and attempts to propagate by copying itself to removable drives and mapped network shares. "Although the self-modification function (AttemptToUpdateSelf) is commented out, its presence, combined with the active logging of AI responses to '%TEMP%\thinking_robot_log.txt,' clearly indicates the author's goal of creating a metamorphic script that can evolve over time," Google added. The tech giant also said it discovered multiple variations of PROMPTFLUX incorporating LLM-driven code regeneration, with one version using a prompt to rewrite the malware's entire source code every hour by instructing the LLM to act as an "expert VB Script obfuscator." PROMPTFLUX is assessed to be under development or testing phase, with the malware currently lacking any means to compromise a victim network or device. It's currently not known who is behind the malware, but signs point to a financially motivated threat actor that has adopted a broad, geography- and industry-agnostic approach to target a wide range of users. Google also noted that adversaries are going beyond utilizing AI for simple productivity gains to create tools that are capable of adjusting their behavior in the midst of execution, not to mention developing purpose-built tools that are then sold on underground forums for financial gain. Some of the other instances of LLM-powered malware observed by the company are as follows - From a Gemini point of view, the company said it observed a China-nexus threat actor abusing its AI tool to craft convincing lure content, build technical infrastructure, and design tooling for data exfiltration. In at least one instance, the threat actor is said to have reframed their prompts by identifying themselves as a participant in a capture-the-flag (CTF) exercise to bypass guardrails and trick the AI system into returning useful information that can be leveraged to exploit a compromised endpoint. "The actor appeared to learn from this interaction and used the CTF pretext in support of phishing, exploitation, and web shell development," Google said. "The actor prefaced many of their prompts about exploitation of specific software and email services with comments such as 'I am working on a CTF problem' or 'I am currently in a CTF, and I saw someone from another team say ...' This approach provided advice on the next exploitation steps in a 'CTF scenario.'" Other instances of Gemini abuse by state-sponsored actors from China, Iran, and North Korea to streamline their operations, including reconnaissance, phishing lure creation, command-and-control (C2) development, and data exfiltration, are listed below - Furthermore, GTIG said it recently observed UNC1069 employing deepfake images and video lures impersonating individuals in the cryptocurrency industry in their social engineering campaigns to distribute a backdoor called BIGMACHO to victim systems under the guise of a Zoom software development kit (SDK). It's worth noting that some aspect of the activity shares similarities with the GhostCall campaign recently disclosed by Kaspersky. The development comes as Google said it expects threat actors to "move decisively from using AI as an exception to using it as the norm" in order to boost the speed, scope, and effectiveness of their operations, thereby allowing them to mount attacks at scale. "The increasing accessibility of powerful AI models and the growing number of businesses integrating them into daily operations create perfect conditions for prompt injection attacks," it said. "Threat actors are rapidly refining their techniques, and the low-cost, high-reward nature of these attacks makes them an attractive option."

Google warns of AI-infused malware that's harder to detect than normal viruses

Google's Threat Intelligence Group (GTIG) is warning that bad guys are using artificial intelligence to create and deploy new malware that both utilizes and combats large language models (LLM) like Gemini when deployed. The findings were laid out in a white paper released on Wednesday, November 5 by the GTIG. The group noted that adversaries are no longer leveraging artificial intelligence (AI) just for productivity gains, they are deploying "novel AI-enabled malware in active operations." They went on to label it a new "operational phase of AI abuse." Google is calling the new tools "just-in-time" AI used in at least two malware families: PromptFlux and PromptSteal, both of which use LLMs during deployment. They generate malicious scripts and obfuscate their code to avoid detection by antivirus programs. Additionally, the malware families use AI models to create malicious functions "on demand" rather than being built into the code. Google says these tools are a nascent but significant step towards "autonomous and adaptive malware." PromptFlux is an experimental VBScript dropper that utilizes Google Gemini to generate obfuscated VBScript variants. VBScript is mostly used for automation in Windows environments. In this case, PromptFlux attempts to access your PC via Startup folder entries and then spreads through removable drives and mapped network shares. "The most novel component of PROMPTFLUX is its 'Thinking Robot' module, designed to periodically query Gemini to obtain new code for evading antivirus software," GTIG says. The researchers say that the code indicates the malware's makers are trying to create an evolving "metamorphic script." According to Google, the Threat Intelligence researchers could not pinpoint who made PromptFlux, but did note that it appears to be used by a group for financial gain. Google also claims that it is in early development and can't yet inflict real damage. The company says that it has disabled the malware's access to Gemini and deleted assets connected to it. Google also highlighted a number of other malware that establish remote command-and control (FruitShell), capturing GitHub credentials (QuietVault), and one that steals and encrypts data on Windows, macOS and Linux devices (PromptLock). All of them utilize AI to work or in the case of FruitShell to bypass LLM-powered security. Beyond malware, the paper also reports several cases where threat actors abused Gemini. In one case, a malicious actor posed as a "capture-the-flag" participant, basically acting as a students or researchers to convince Gemini to provide information that is supposed to be blocked. Google specified a number of threats from Chinese, Iranian and North Korean threat groups that abused Gemini for phishing, data mining, increasing malware sophistication, crypto theft and creating deepfakes. Again, Google says it has disabled the associated accounts in identified cases and reinforced its model safeguards. The company goes on to says that underground marketplaces for malicious AI-based tools is growing. "Many underground forum advertisements mirrored language comparable to traditional marketing of legitimate AI models, citing the need to improve the efficiency of workflows and effort while simultaneously offering guidance for prospective customers interested in their offerings," the company wrote. With AI getting more sophisticated, this seems to indicate a trend of replacing conventional malicious tools with new AI-based ones. The paper wraps up by advocating that AI developers need to be "both bold and responsible" and that AI systems must be designed with "strong safety guardrails" to prevent these kinds of abuses. Google says that it investigates signs of abuse in its products and uses the experience of combating bad actors to "improve safety and security for our AI models." The war against viruses and malware is ever evolving as tools on both sides become more sophisticated especially with the injection of AI. There are ways to stay safe. As always, be wary of links and external content. If an AI tool is be used to summarize a web page, PDF, or email that content could be malicious or contain a hidden prompt to attack the AI. Additionally, you should always limit AI access to sensitive accounts like bank accounts, email or documents that have sensitive information. Compromised AI could exploit that access. Finally, unexpected behavior in an LLM or AI model should be treated as a red flag. If an AI model starts answerint questions strangely, reveals internal knowledge of your PC or worse, tries to perform unusual or unauthorized actions then you should stop that session. Make sure you keep your software updated, including the best antivirus software and the LLM programs and applications you utilize. this ensures that you have the most recent and patched versions protecting you against known flaws.

Google warns criminals are building and selling illicit AI tools - and the market is growing

'Just-in-time' AI malware shows how criminals are evolving their techniques Google's Threat Intelligence Group has identified a worrying shift in AI trends, with AI no longer just being used to make criminals more productive, but also now being specially developed for active operations. Its research found Large Language Models (LLMs) are being used in malware in particular, with 'Just-in-Time' AI like PROMPTFLUX - which is written in VBScript and engages with Gemini's API to request 'specific VBScript obfuscation and evasion techniques to facilitate "just-in-time" self-modification, likely to evade static signature-based detection'. This illustrates how criminals are experimenting with LLMs to develop ''dynamic obfuscation techniques' and targeting victims. The PROMPTFLUX samples examined by Google suggest that this code family is currently in the testing phase - so it could get even more dangerous once criminals develop them further. The marketplace for legitimate AI tools is maturing, and so is the criminal black market. Underground forums offer purpose-built AI tools that help lower the barrier for criminals to engage in illicit activities. This is bad news for everyone, since criminals no longer have to be particularly skilled to carry out complex cyberattacks, and they have a growing number of options. Threat actors are using tactics reminiscent of social engineering to side-step AI safety features - pretending to be 'cybersecurity researchers' in order to convince Gemini to provide them with information that might otherwise be prohibited. But who's behind these incidents? Well, the research identifies, perhaps unsurprisingly, links to state-sponsored actors from Iran and China. These campaigns have a range of objectives, from data exfiltration to reconnaissance - similar to previously observed influence operations by the states, also using AI tools. Since AI tools have become popularized, both criminals and security teams have been using the tools to boost productivity and assist in operations - and it's not quite clear who has the upper hand.

Hackers are already using AI-enabled malware, Google says

Why it matters: The discovery suggests adversarial hackers are moving closer to operationalizing generative AI to supercharge their attacks. Driving the news: Researchers in Google's Threat Intelligence Group have discovered two new malware strains -- PromptFlux and PromptSteal -- that use large language models to change their behavior mid-attack. * Both malware strains can "dynamically generate malicious scripts, obfuscate their own code to evade detection and leverage AI models to create malicious functions on demand," according to the report. Zoom in: Google's team found PromptFlux while scanning uploads to VirusTotal, a popular malware-scanning tool, for any code that called back to Gemini. * The malware appears to be in active development: Researchers observed the author uploading updated versions to VirusTotal, likely to test how good it is at evading detection. It uses Gemini to rewrite its own source code, disguise activity and attempt to move laterally to other connected systems. * Meanwhile, Russian military hackers have used PromptSteal, another new AI-powered malware, in cyberattacks on Ukrainian entities, according to Google. The Ukrainian government first discovered the malware in July. * Unlike conventional malware, PromptSteal lets hackers interact with it using prompts, much like querying an LLM. It's built around an open-source model hosted on Hugging Face and designed to move around a system and exfiltrate data as it goes. Reality check: Both malware strains are pretty nascent, Google says. But they mark a major step toward the future that many security executives have feared. Between the lines: PromptSteal's reliance on an open-source model is something Google's team is watching closely, Billy Leonard, tech lead at Google Threat Intelligence Group, told Axios. * "What we're concerned about there is that with Gemini, we're able to add guardrails and safety features and security features to those to mitigate this activity," Leonard said. "But as (hackers) download these open-source models, are they able to turn down the guardrails?" The big picture: The underground cyber crime market for AI tools has matured significantly in the past year, the report says. * Researchers have seen advertisements for AI tools that could write convincing phishing emails, create deepfakes and identify software vulnerabilities. * That makes it easier for even unskilled cyber criminals to launch attacks well beyond their own capabilities. Yes, but: Most attackers don't need AI to do damage and are still overwhelmingly relying on common tactics, like phishing emails and stolen credentials, incident responders have told Axios. * "This isn't 'the sky is falling, end of the world,'" Leonard said. "They're adopting technologies and capabilities that we're also adopting." Go deeper: AI is about to supercharge cyberattacks

Great, now even malware is using LLMs to rewrite its code, says Google, as it documents new phase of 'AI abuse'

AI has been up to some naughty things in the past few years. From ignoring copyright to producing pretty awful approximations of games, to being exploited to lock down your machine. It seems like, as AI gets smarter, it only gets easier to use in malicious ways. One piece of malware has even been spotted using AI to rewrite itself to avoid detection. Spooky stuff. Google has recently warned "adversaries are no longer leveraging artificial intelligence (AI) just for productivity gains, they are deploying novel AI-enabled malware in active operations". The report (via Bleeping Computer) says that, for the first time, Google "discovered a code family that employed AI capabilities mid-execution to dynamically alter the malware's behaviour." Google pays attention to one particular malware named Promptflux, which is a dropper that prompts Gemini to rewrite its own source code, "saving the new, obfuscated version to the Startup folder to establish persistence." Like an actual virus, this suggests it has the ability to 'evolve' in some way, getting smarter and harder to get rid of. It is only experimental right now and doesn't appear to be "used in the wild", luckily. If you're worried about the potential of this tech (same), Google says DeepMind has used insights gained from the malware to strengthen security, both with "Google's classifiers and the model itself." Reportedly, this should mean that the model refuses to help malware with these kinds of attacks going forward. AI fighting AI. So it begins. Naturally, AI being so broad and having so many applications does often mean it's exploitable, and it wouldn't be the first time someone has got AI to act against its reported purpose. In its latest report, Google has identified four other cases of AI malware that have popped up in 2025. They are: Google says, "This marks a new operational phase of AI abuse, involving tools that dynamically alter behaviour mid-execution", and it's certainly worrisome to think of how smart these tools can get in the wrong hands. Despite these worries, Google does clarify that its approach to AI must be "both bold and responsible", and publicly acknowledging both malware using its services and its response to said malware feels like a good step. Still, the idea of malware rewriting itself to avoid being caught is a little too close to fiction for comfort. Next, we just need a malware detector that also rewrites itself, and we can have the two fight it out for control of the game saves and terabytes of animal pictures on my gaming rig.

Google warns that a new era of self-evolving, AI-driven malware has begun - SiliconANGLE

Google warns that a new era of self-evolving, AI-driven malware has begun A new report out today from Google LLC's Threat Intelligence Group warns that there has been a major shift in cybercrime as attackers are no longer using artificial intelligence solely for productivity but are now deploying AI-enabled malware directly in active operations. The GTIG AI Threat Tracker: Advances in Threat Actor Usage of AI Tools report highlights how state-sponsored and criminal groups are leveraging large language models such as Gemini and other publicly available systems to automate, adapt and scale up attacks across the entire lifecycle. In a notable first, Google's researchers have identified malware families, including PROMPTFLUX, PROMPTSTEAL and PROMPTLOCK, that integrate AI during execution to dynamically generate malicious code and obfuscate their behavior. PROMPTFLUX, for example, interacts with the Gemini application programming interface to rewrite its own VBScript every hour, creating an evolving "thinking robot" that continually mutates to avoid antivirus detection. PROMPTSTEAL, used by the Russia-linked APT28 threat group, queries open-source language models on Hugging Face to generate Windows commands that harvest files and system data before exfiltration. The report states that the rise of "just-in-time" AI attacks is a new milestone in adversarial use of generative models and represents a move toward autonomous, self-modifying malware. The researchers note that while many examples remain experimental, the trend signals how attackers will soon combine AI reasoning and automation to outpace traditional defenses. Another area of concern raised in the report is social engineering aimed at bypassing AI safety guardrails. Threat actors from Iran and allegedly from China were observed posing as students, researchers or participants in "capture-the-flag" cybersecurity contests to trick Gemini into providing restricted vulnerability or exploitation data. In one case, Iran-backed MUDDYCOAST accidentally revealed its own command-and-control infrastructure while using Gemini to debug a malware script, a mistake that allowed Google to dismantle its operations. Not surprisingly, the underground economy for AI-driven hacking tools has also matured rapidly. The researchers found dozens of multifunctional offerings advertised in English and Russian-language forums, selling capabilities such as phishing-email generation, deepfake creation and automated malware development. Similar to software-as-a-service offerings, the tools are offered via subscription models, lowering the cost of entry. State-sponsored groups were found to be the most prolific adopters. North Korea's MASAN and PUKCHONG have used Gemini for cryptocurrency theft campaigns and exploit development, while Iran's APT42 experimented with a "Data Processing Agent" that turned natural-language requests into SQL queries to extract personal information. Google says it has disabled accounts and assets associated with these activities and used the intelligence to harden its models and classifiers against further misuse. "The potential of AI, especially generative AI, is immense," the report concludes. "As innovation moves forward, the industry needs security standards for building and deploying AI responsibly." To address the increasing risk, Google offers the Secure AI Framework, a foundational blueprint aimed at helping organizations design, build and deploy AI systems responsibly. SAIF serves as both a technical and ethical guide to establish security principles that span the entire AI lifecycle, from data collection and model training to deployment and monitoring.

Google Identifies New Forms of AI-Powered Cyberattacks | PYMNTS.com

GTIG said in a report released Wednesday that this is the first time it has seen malware families use large language models during execution. "While still nascent, this represents a significant step toward more autonomous and adaptive malware," the report said. This is one example of the ways threat actors are using AI not only for productivity gains but also for "novel AI-enabled operations," GTIG said in its blog post. Threat actors are also using pretexts like posing as a student or researcher in prompts to bypass AI safety guardrails and extract restricted information, and they are using underground digital markets to access AI tools for phishing, malware and vulnerability research, according to the post. "At Google, we are committed to developing AI responsibly and take proactive steps to disrupt malicious activity by disabling the projects and accounts associated with bad actors, while continuously improving our models to make them less susceptible to misuse," the company said in the report. "We also proactively share industry best practices to arm defenders and enable stronger protections across the ecosystem." PYMNTS reported Monday (Nov. 3) that AI has become both a tool and a target when it comes to cybersecurity. For example, CSO.com said that agentic AI is emerging as a transformative force in cybersecurity because it can process data continuously and react in real time to detect, contain and neutralize threats at a scale and speed that human teams cannot match. It was also reported Monday that tech companies are increasing their efforts to combat a security flaw in their AI models. The companies are focused on stopping indirect prompt injection attacks in which a third party hides commands inside a website or email to trick AI models into turning over unauthorized information.