Google's AI Bug Hunter 'Big Sleep' Uncovers 20 Security Vulnerabilities in Open Source Software

Google says its AI-based bug hunter found 20 security vulnerabilities | TechCrunch

Google's AI-powered bug hunter has just reported its first batch of security vulnerabilities. Heather Adkins, Google's vice president of security, announced Monday that its LLM-based vulnerability researcher Big Sleep found and reported 20 flaws in various popular open source software. Adkins said that Big Sleep, which is developed by the company's AI department DeepMind as well as its elite team of hackers Project Zero, reported its first-ever vulnerabilities, mostly in open source software such as audio and video library FFmpeg and image editing suite ImageMagick. Given that the vulnerabilities are not fixed yet, we don't have details of their impact or severity, as Google does not yet want to provide details, which is a standard policy when waiting for bugs to be fixed. But the simple fact that Big Sleep found these vulnerabilities is significant, as it shows these tools are starting to get real results, even if there was a human involved in this case. "To ensure high quality and actionable reports, we have a human expert in the loop before reporting, but each vulnerability was found and reproduced by the AI agent without human intervention," Google's spokesperson Kimberly Samra told TechCrunch. Royal Hansen, Google's vice president of engineering, wrote on X that the findings demonstrate "a new frontier in automated vulnerability discovery." LLM-powered tools that can look for and find vulnerabilities are already a reality. Other than Big Sleep, there's RunSybil, and XBOW, among others. XBOW has garnered headlines after it reached the top of one of the U.S. leaderboards at bug bounty platform HackerOne. It's important to note that in most cases, these reports have a human at some point of the process to verify that the AI-powered bug hunter found a legitimate vulnerability, as is the case with Big Sleep. Vlad Ionescu, co-founder and chief technology officer at RunSybil, a startup that develops AI-powered bug hunters, told TechCrunch that Big Sleep is a "legit" project, given that it has "good design, people behind it know what they're doing, Project Zero has the bug finding experience and DeepMind has the firepower and tokens to throw at it." There is obviously a lot of promise with these tools, but also significant downsides. Several people who maintain different software projects have complained of bug reports that are actually hallucinations, with some calling them the bug bounty equivalent of AI slop. "That's the problem people are running into, is we're getting a lot of stuff that looks like gold, but it's actually just crap," Ionescu previously told TechCrunch.

Google's new AI-powered bug hunting tool finds major issues in open source software

The first batch of 20 vulnerabilities it has spotted have been announced Google's AI-powered tool designed to find bugs, Big Sleep, has reported its first batch of 20 security vulnerabilities in open source software. Developed by AI and security teams from Google's DeepMind and Project Zero, the first vulnerabilities were found in the likes of FFmpeg and ImageMagick, however details of those vulnerabilities remain undisclosed until they have been patched. Google says Big Sleep marks a significant step forward in app security, with AI capable of autonomously uncovering and reporting vulnerabilities more effectively than human security workers. Each of the 20 bugs was found and reproduced autonomously by Big Sleep, though Google notes that a human expert does review the findings before making reports public - with human review important to temper worries over false positives or hallucinated bugs by ensuring the issues are worthy of being reported to their respective developers. Finer details like CVE IDs, technical explanations and proofs of concept are withheld for now under Google's 90-day policy to give developers time to patch the vulnerabilities without attackers getting in first. "By November 2024, Big Sleep was able to find its first real-world security vulnerability, showing the immense potential of AI to plug security holes before they impact users," President of Global Affairs Kent Walker boasted in a blog post. VP for Security Engineering, Heather Adkins, announced the news in an X post: "Today as part of our commitment to transparency in this space, we are proud to announce that we have reported the first 20 vulnerabilities discovered using our AI-based "Big Sleep" system powered by Gemini." Google keeps a full list of vulnerabilities, which currently includes the first 20, separated into high, medium and low impact issues. Google plans a full technical briefing at the upcoming Black Hat USA and DEF CON 33 events, and will donate anonymized training data to the Secure AI Framework so other researchers can benefit from the tech.

Google unveils Big Sleep AI for vulnerability detection

Google's AI-powered bug hunter, Big Sleep, developed by DeepMind and Project Zero, has identified 20 security vulnerabilities in open-source software, marking its initial public reporting of flaws. Heather Adkins, Google's vice president of security, disclosed on Monday that Big Sleep, an LLM-based vulnerability researcher, detected these flaws across various popular open-source software applications. The identified vulnerabilities primarily impact systems such as FFmpeg, an audio and video library, and ImageMagick, an image-editing suite. Details regarding the specific impact or severity of these vulnerabilities have not been released due to Google's standard protocol of withholding information until fixes are implemented. Kimberly Samra, a Google spokesperson, clarified the process involved in these discoveries. Samra stated, "To ensure high quality and actionable reports, we have a human expert in the loop before reporting, but each vulnerability was found and reproduced by the AI agent without human intervention." This procedure confirms that while human verification occurs, the initial detection and reproduction of the vulnerability are performed autonomously by the AI agent. Royal Hansen, Google's vice president of engineering, characterized these findings as demonstrating "a new frontier in automated vulnerability discovery" in a post on X. Big Sleep is not the only LLM-powered tool designed for vulnerability detection; other notable examples include RunSybil and XBOW. XBOW has gained attention for reaching the top position on a U.S. leaderboard within the bug bounty platform HackerOne. Similar to Big Sleep, many of these AI-powered bug hunters incorporate a human verification step to confirm the legitimacy of detected vulnerabilities. Vlad Ionescu, co-founder and chief technology officer at RunSybil, a startup specializing in AI-powered bug hunters, described Big Sleep as a "legit" project. He attributed this assessment to its "good design, people behind it know what they're doing, Project Zero has the bug finding experience and DeepMind has the firepower and tokens to throw at it." While the potential of these AI tools for identifying security flaws is evident, there are also noted drawbacks. Several maintainers of various software projects have reported receiving bug reports that are later identified as hallucinations, drawing parallels to "AI slop" in the context of bug bounties. Ionescu previously commented on this issue, stating, "That's the problem people are running into, is we're getting a lot of stuff that looks like gold, but it's actually just crap."