AI coding tools boost output tenfold but create massive backlogs and security vulnerabilities

The Effects of AI-Generated Code Tearing Through Corporations Is Actually Kind of Funny

Can't-miss innovations from the bleeding edge of science and tech Corporations are rapidly embracing AI to churn out mountains of code. Outwardly, this is presented as a revolution in productivity. But a behind the scenes look in The New York Times paints a slightly different, and somewhat comic, picture. Beleaguered programmers are being saddled with more code than what they know what to do with, while their employers struggle to find the best way to get them to check all the AI's hastily written work. One financial services company, for example, saw its coding output increase tenfold after embracing the popular AI tool Cursor -- creating an epic backlog of one million lines of code that needs to be reviewed, according to Joni Klippert, CEO of the security startup StackHawk, which works with the financial firm. And the code glut isn't something that can be ignored. Left unchecked, bad code -- regardless of whether it's AI-generated or human-written -- can gum up software and cause security flaws. Amazon and Meta both recently experienced disruptions after AI tools took unauthorized actions, and those are just the ones we've heard about. "The sheer amount of code being delivered, and the increase in vulnerabilities, is something they can't keep up with," Klippert told the NYT. The accelerated output created a "lot of stress" in other departments, like sales and marketing support, she added. We're now at an interesting inflection point of AI's impact in the workplace. It's been used to justify whittling down workforces across the globe, with one report finding that AI was cited in the announcements of more than 54,000 layoffs last year. This year included major names in tech: Jack Dorsey's fintech firm Block and software giant Atlassian laid off thousands of employees while touting pivots to AI. Yet, at the same time that jobs are being eliminated, AI is also creating more work that would be best done by another human. Someone has to test the AI code, and traditionally it'd be the guy who wrote it -- but nowadays they're too busy prompting an AI agent. Who's supposed to pick up the slack is unclear. "There are not enough application security engineers on the planet to satisfy what just American companies need," Joe Sullivan, an adviser to Costanoa Ventures, told the NYT. Moreover, AI may actually be making programmers' jobs harder. Software engineers have admitted that being expected to produce more code while having to constantly supervise their AI tools is accelerating them towards burnout -- a phenomenon that's been documented in emerging research into the topic. One ongoing study dubbed this mental health toll AI "brain fry." Companies are still grappling with how to address the code glut. "The blessing and the curse is that now everyone inside your company becomes a coder," Michele Catasta, the president and head of AI at the startup Replit, told the NYT. Sachin Kamdar of the AI agent startup Elvix took a hardline approach: all code must be reviewed by a human, because it'd be harder to fix down the line if no one understood what the AI cooked up in the first place. "It's just going to break something, and they're not going to know why it broke," he told the NYT. Another solution is throwing more AI at the problem. Anthropic and OpenAI have released AI agents designed to review code. And in December, Cursor, the provider of the much hyped AI coding tool, bought the startup Graphite, which builds an AI code reviewing platform.

AI has turbocharged coding, but stirred a slop problem of its own

AI coding tools were supposed to make software development faster and easier. They did, but maybe a little too well. People are writing code faster than ever before, and this has created a whole new set of problems for companies. According to The New York Times, one financial services company started using Cursor, an AI coding tool, and went from producing 25,000 to 250,000 lines of code per month. That sounds like a win, but it created a backlog of one million lines of unreviewed code. "The sheer amount of code being delivered, and the increase in vulnerabilities, is something they can't keep up with," said Joni Klippert, CEO of StackHawk, a security startup working with the firm. Recommended Videos The problem has spread across Silicon Valley. Companies are now producing more code than they have the people to review, and that gap is becoming a security risk. So, what's the problem? The role responsible for catching errors in AI-generated code is called an application security engineer. There aren't nearly enough of them. "There are not enough application security engineers on the planet to satisfy what just American companies need," said Joe Sullivan, an adviser to Costanoa Ventures. It's not just a staffing problem either. AI coding tools work better on personal laptops than on secure company servers, which means engineers are downloading entire codebases onto personal devices. If a laptop goes missing, so does a lot of sensitive data. Is more AI really the answer? Predictably, Silicon Valley thinks so. Companies like Anthropic, OpenAI, and Cursor are already building AI-powered review tools to help catch errors in AI-generated code. Cursor even acquired a code-reviewing startup to build this into its product. As Cursor's head of engineering put it, "The software development factory kind of broke. We're trying to rearrange the parts in some sense." I have my doubts. Yes, AI will eventually be able to catch errors in code, but human review will still be necessary before releasing final production. Recently, an AI code caused an Amazon outage, resulting in over 100,000 lost orders and 1.6 million errors. No company wants to see that happen, and I am not sure AI code reviewers are the answer.

Anyone can code with AI. But it might come with a hidden cost.

Anyone can code using AI. But it might come with a hidden cost. Over the past year, AI systems have become so advanced that users without significant coding or computer science experience can now spin up websites or apps simply by giving instructions to a chatbot. Yet with the rise of AI systems powerful enough to translate the instructions into tomes of code, experts and software engineers are torn over whether the technology will lead to an explosion of bloated, error-riddled software or instead supercharge security efforts by reviewing code faster and more effectively than humans. "AI systems don't make typos in the way we make typos," said David Loker, head of AI for CodeRabbit, a company that helps software engineers and organizations review and improve the quality of their code. "But they make a lot of mistakes across the board, with readability and maintainability of the code chief among them." Coding has long been an art and a science. Since the days of coding computer systems by punch cards in the mid-20th century, conveying computing instructions has been a challenge of elegance and efficiency for computer scientists. But inside today's leading AI companies, most coding is performed by AI systems themselves, with human software engineers functioning more as coaches or high-level architects rather than in-the-weeds mechanics. Anthropic's head of Claude Code, Boris Cherny, said on X that AI has written 100% of his code since at least December. "I don't even make small edits by hand," Cherny said. The rise of AI-assisted coding -- also called vibe coding -- is simultaneously allowing people who have never coded before to unleash their creativity and enabling experienced software engineers to dramatically expand the amount of code they write. "The initial push of all this was developer productivity," Loker told NBC News. "It was about increasing the throughput in terms of feature generation, the ability to build fast and ship things." Though AI-coding systems have become significantly more capable even since November, they often fail to understand entire repositories of code as fully as experienced human developers. For example, Loker said, "AI coding systems might duplicate functionality in multiple different locations because they didn't find that that function already existed, so they re-create it over and over and over again." "Now you end up with a sprawling problem. If you update a function in one spot and you don't update it in the other, you have different business logic in different areas that don't line up. You're left wondering what's going on." With AI coding systems supercharging the amount of code being created, experts wonder whether code will be the next victim of the AI slop onslaught. The concept of AI slop was originally popularized in 2024 as AI systems became capable and pervasive enough to start churning out volumes of low-quality, unwanted AI outputs -- from AI-generated photos to unhelpful AI-powered search results. On one hand, AI coding systems are producing vast amounts of serviceable but imperfect code. On the other hand, those same systems are quickly getting better at reviewing their own code and finding security vulnerabilities. For example, in late January, the rise of AI code slop forced leading developer Daniel Stenberg to shutter a popular effort to find bugs in a popular software system. Stenberg wrote on his blog that "the never-ending slop submissions take a serious mental toll to manage and sometimes also a long time to debunk. Time and energy that is completely wasted while also hampering our will to live." Yet on Thursday, Stenberg said the flood "has transitioned from an AI slop tsunami into more of a ... plain security report tsunami. Less slop but lots of reports. Many of them [are] really good." Companies are quickly realizing that boosted quantity does not automatically increase quality -- in fact, the opposite is often true, according to Jack Cable, CEO and co-founder of the cybersecurity consulting firm Corridor. "Even if [a large language model] is better at writing code line by line, if it's writing 20 times as much code as a human would be, there is significantly more code to be reviewed," Cable said. "It's no longer a challenge to produce tons and tons of code, but companies, if they're doing their job right, still need to be reviewing that code from a functionality perspective, a quality perspective and also a security perspective." AI coding agents are producing "an explosion in complexity," he added. "And if there's one thing we know about software, it's that with increased complexity comes increased attack surface and vulnerability." In January, developer and entrepreneur Matt Schlicht said he used AI coding systems to create a social network for AI systems called Moltbook, now owned by Meta. Yet security researchers soon identified critical security vulnerabilities in Moltbook's software that exposed human users' credentials, which they ascribed to its AI-coded roots. One of those ethical hackers and researchers, Jamieson O'Reilly, told NBC News that the rise of AI coding agents threatened to create security vulnerabilities by giving coding novices significant public exposure without commensurate security expertise. "People often believe that AI coding agents will build things per the best security standards," O'Reilly said. "That's just not the case. AI is knocking down decades of security silos that were built up to protect users, and it's being traded for convenience as these AI systems evolve." Daniel Kang, a professor of computer science at the University of Illinois Urbana-Champaign and an expert on security vulnerabilities created by AI coding agents, agreed that AI coding systems are likely to give new users a false sense of safety. "Even if you assume that the rate of security vulnerabilities in any given chunk of code is constant, the number of vulnerabilities will go up dramatically because people who don't know the first thing about computer security, and even experienced programmers who don't treat security as a top priority, are going to be producing more code," Kang said. To try to quantify the growing phenomenon, researchers at Georgia Tech have launched a Vibe Security Radar. Since August, the team has identified over 70 critical software vulnerabilities that are most likely due to AI coding, with a significant increase in the past two months. An AI startup called Arcade recently launched a tool for developers to monitor the sloppiness of their code. CodeRabbit also released a report in December finding that AI-generated code has 70% more errors than human-written code and that the AI-generated errors are more serious than human-generated errors, though Loker, of CodeRabbit, cautioned that those results might be slightly out of date given how quickly today's AI systems are evolving. While much software is proprietary and "closed-source," or hidden from public sight, many other projects, like Mozilla's Firefox browser or the Linux operating system, are open-source and rely on community members to submit suggestions to improve the software. By lowering the barriers to submit suggestions to the open-source software packages, AI-assisted coding has flooded many of the community-led initiatives with low-quality code over the past few months. "A lot of package maintainers we talk to are inundated by slop," Loker said. "It's just completely poorly written. It's not even well thought-out, doesn't fit in and contains various other pieces of nonsense." The barrage of AI-mediated code is forcing one of the most popular hosts of code repositories, GitHub, to rethink its approach to open-source software maintenance. And on Friday, GitHub's chief operating officer said overall platform activity in 2026 is roughly on pace to surge 14 times above 2025 levels. Yet, as Stenberg said, the new AI-fueled fire might also be best fought with other AI systems, as AI-powered programs to review and refine code become increasingly popular. Noting that CodeRabbit's own systems are AI-powered, Loker said: "A code-review system that's automated is now really, really necessary in most companies that are adopting these systems. We don't have to sell people anymore as much on the idea that quality is an issue. Our partners have been using AI to code long enough now that they are seeing the detrimental side effects." Cherny, of Anthropic, is betting that rapid improvements in AI systems' coding abilities will help solve the emerging chasms in code quality and reliability. "My bet is that there will be no slopcopolypse because the model will become better at writing less sloppy code and at fixing existing code issues," Cherny wrote in late January. Regardless of the growing cottage industry of code-review systems, Kang, of the University of Illinois, is adamant that coders -- new and old -- can guard their systems against code slop by embracing age-old cybersecurity fundamentals. "If you apply all the best practices and you do all of the correct things, then you can actually be better off than before AI systems," he said. Yet Kang is pessimistic that users will actually adopt adequate security practices given rabid AI adoption. As a result, he is bearish about the long-term effects of code slop: "It's going to blow up. It's definitely going to be really nasty." "The question is just how and when, and that's what I'm worried about."