AI-Generated Bug Reports Plague Open Source Projects, Frustrating Developers

Open source projects drown in bad bug reports penned by AI

Python security developer-in-residence decries use of bots that 'cannot understand code' Software vulnerability submissions generated by AI models have ushered in a "new era of slop security reports for open source" - and the devs maintaining these projects wish bug hunters would rely less on results produced by machine learning assistants. Seth Larson, security developer-in-residence at the Python Software Foundation, raised the issue in a blog post last week, urging those reporting bugs not to use AI systems for bug hunting. "Recently I've noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects," he wrote, pointing to similar findings from the Curl project in January. "These reports appear at first glance to be potentially legitimate and thus require time to refute." Larson argued that low-quality reports should be treated as if they're malicious. As if to underscore the persistence of these concerns, a Curl project bug report posted on December 8 shows that nearly a year after maintainer Daniel Stenberg raised the issue, he's still confronted by "AI slop" - and wasting his time arguing with a bug submitter who may be partially or entirely automated. In response to the bug report, Stenberg wrote: Spammy, low-grade online content existed long before chatbots, but generative AI models have made it easier to produce the stuff. The result is pollution in journalism, web search, and of course social media. For open source projects, AI-assisted bug reports are particularly pernicious because they require consideration and evaluation from security engineers - many of them volunteers - who are already pressed for time. Larson told The Register that while he sees relatively few low-quality AI bug reports - fewer than ten each month - they represent the proverbial canary in the coal mine. "Whatever happens to Python or pip is likely to eventually happen to more projects or more frequently," he warned. "I am concerned mostly about maintainers that are handling this in isolation. If they don't know that AI-generated reports are commonplace, they might not be able to recognize what's happening before wasting tons of time on a false report. Wasting precious volunteer time doing something you don't love and in the end for nothing is the surest way to burn out maintainers or drive them away from security work." Larson argued that the open source community needs to get ahead of this trend to mitigate potential damage. "I am hesitant to say that 'more tech' is what will solve the problem," he said. "I think open source security needs some fundamental changes. It can't keep falling onto a small number of maintainers to do the work, and we need more normalization and visibility into these types of open source contributions. "We should be answering the question: 'how do we get more trusted individuals involved in open source?' Funding for staffing is one answer - such as my own grant through Alpha-Omega - and involvement from donated employment time is another." While the open source community mulls how to respond, Larson asks that bug submitters not submit reports unless they've been verified by a human - and don't use AI, because "these systems today cannot understand code." He also urges platforms that accept vulnerability reports on behalf of maintainers to take steps to limit automated or abusive security report creation. ®

Open source software users are being hit by AI-written junk bug reports

Reading them all hits maintainer time and energy, report warns Security report triage worker Seth Larson has revealed many open source project maintainers are being hit by "low-quality, spammy, and LLM-hallucinated security reports." The AI-generated reports, often inaccurate and misleading, demand time and effort to review, which is taking away from the already limited time open source software developers and maintainers typically have given that they contribute on a volunteer basis. Larson added maintainers are typically discouraged from sharing their experiences or asking for help due to the security-sensitive nature of reports, making the unreliable security reports even more time-consuming. Maintainers of open source projects like Curl and Python have faced "an uptick" in such reports recently, revealed Larson, who points to Curl maintainer Daniel Stenberg's post of a similar nature. Responding to a recent bug report, Stenberg criticized the reported for submitting an AI-generated vulnerability claim without verification, adding that this sort of behavior adds to the already stretched workload of developers. Stenberg, who is a maintainer for Curl, said: "We receive AI slop like this regularly and at volume. You contribute to unnecessary load of curl maintainers and I refuse to take that lightly and I am determined to act swiftly against it... You submitted what seems to be an obvious AI slop 'report' where you say there is a security problem, probably because an AI tricked you into believing this." While the problem of false reports like this is nothing new, artificial intelligence has seemingly worsened it. AI-generated bug reports are already proving to be draining on maintainers' time and energy, but Larson said that continued false reports could discourage developers from wanting to contribute to open source projects altogether. To address this issue, Larson is calling on bug reports to verify their submissions manually before reporting, and to avoid using AI for vulnerability detection in the first place. Reporters who can provide actionable solutions rather than simply highlighting vague issues can also prove their worth to maintainers. For maintainers, Larson says they should not respond to suspected AI-generated reports to same themselves time, and ask reporters to justify their claims if in doubt.

Useless AI generated security reports are frustrating open-source maintainers

Facepalm: Generative AI services are neither intelligent nor capable of providing a meaningful addition to open-source development efforts. A security expert who has had enough of "spammy," hallucinated bug listings is venting his frustration, asking the FOSS community to sidestep AI-generated reports. Generative AI models have already proven powerful tools in the hands of cyber-criminals and fraudsters. However, hucksters can also use them to spam open-source projects with useless bug reports. According to Seth Larson, the number of "extremely" low-quality, spammy, and LLM-hallucinated security reports has recently increased, forcing maintainers to waste their time on things of low intelligence. Larson is a security developer at the Python Software Foundation who also volunteers on "triage teams" tasked with vetting security reports for popular open-source projects such as CPython, pip, urllib3, Requests, and others. In a recent blog post, the developer denounces a new and troublesome trend of sloppy security reports created with generative AI systems. These AI reports are insidious because they appear as potentially legitimate and worth checking out. As Curl and other projects have already pointed out, they are just better-sounding crap but crap nonetheless. Thousands of open-source projects are affected by this issue, while maintainers aren't encouraged to share their findings because of the sensitive nature of security-related development. "If this is happening to a handful of projects that I have visibility for, then I suspect that this is happening on a large scale to open source projects," Larson said. Hallucinated reports waste volunteer maintainers' time and result in confusion, stress, and much frustration. Larson said that the community should treat low-quality AI reports as malicious, even if this is not the original intent of the senders. He had valuable advice for platforms, reporters, and maintainers currently dealing with an uptick in AI-hallucinated reports. The community should employ CAPTCHA and other anti-spam services to prevent the automated creation of security reports. Meanwhile, bug reporters should not use AI models to detect security vulnerabilities in open-source projects. Large language models don't understand anything about code. Finding legitimate security flaws requires dealing with "human-level concepts" such as intent, common usage, and context. Maintainers can save themselves from a lot of trouble by responding to apparent AI reports with the same effort put forth by the original senders, which is "near zero." Larson acknowledges that many vulnerability reporters act in good faith and usually provide high-quality reports. However, an "increasing majority" of low-effort, low-quality reports ruin it for everyone involved in development.

Bogus AI-Generated Bug Reports Are Driving Open Source Developers Nuts

Just like social media, open source maintainers are facing an avalanche of junk. Artificial intelligence is not just flooding social media with garbage, it's also apparently afflicting the open-source programming community. And in the same way, fact-checking tools like X's Community Notes struggle to refute a deluge of false information, contributors to open-source projects are lamenting the time wasted evaluating and debunking bug reports created using AI code-generation tools. The Register reported today on such concerns raised by Seth Larson in a blog post recently. Larson is a security developer-in-residence at the Python Software Foundation who says that he has noticed an uptick in "extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects." "These reports appear at first glance to be potentially legitimate and thus require time to refute," Larson added. It could potentially be a big problem for open-source projects (i.e. Python, WordPress, Android) that power much of the internet, because they're often maintained by small groups of unpaid contributors. Legitimate bugs in ubiquitous code libraries can be dangerous because they have such a potentially wide impact zone if exploited. Larson said he's only seeing a relatively small number of AI-generated junk reports, but the number is increasing. Another developer, Daniel Sternberg, called out a bug submitter for wasting his time with a report he believed was generated using AI: You submitted what seems to be an obvious AI slop 'report' where you say there is a security problem, probably because an AI tricked you into believing this. You then waste our time by not telling us that an AI did this for you and you then continue the discussion with even more crap responses â€" seemingly also generated by AI. Code generation is an increasingly popular use case for large language models, though many developers are still torn on how useful they truly are. Programs like GitHub Copilot or ChatGPT's own code generator can be quite effective at producing scaffolding, the basic skeleton code to get any project started. They can also be useful for finding functions in a programming library a developer might not be intimate with. But as with any language model, they will hallucinate and produce incorrect code. Code generators are probability tools that guess what you want to write next based on the code you have given them and what they have seen before. Developers still need to fundamentally understand the programming language they're working with and know what they're trying to build; the same way essays written by ChatGPT need to be reviewed and modified manually. Platforms like HackerOne offer bounties for successful bug reports, which may encourage some individuals to ask ChatGPT to search a codebase for flaws and then submit erroneous ones the LLM returns. Spam has always been around on the internet, but AI is making it a lot easier to generate. It seems possible that we're going to find ourselves in a situation that demands more technology like CAPTCHAs for login screens are used to combat this. An unfortunate situation and a big waste of time for everyone.