How are developers actually using GenAI-powered coding tools now that some of the initial hype has faded?
Our annual survey of more than 65,000 developers captures a portrait of the global developer community: who you are, how you learn, what kind of job you have, the tools and technologies you use, and more. We previously summarized some of the results in this blog post, but we wanted to dive deeper into certain insights the survey uncovered.
This year, we were interested to know how developers are using GenAI-powered coding tools now that some of the initial hype has faded and you've had time to incorporate these tools into your workflows and understand their advantages and drawbacks.
How developers feel about AI coding tools and how much they trust their output was something else we wanted to understand. Finally, we wanted to know how much time developers spend searching for answers at work and how AI might help them surface information more quickly and efficiently. These questions are all relevant to our job at Stack Overflow: giving developers a hub for trustworthy, community-vetted knowledge.
Here's what we learned.
Most developers are using AI tools, but they don't necessarily trust them
According to this year's survey, 76% of developers are using or planning to use AI coding tools, up from 70% last year. That's no surprise: naturally, as more tools have become more widely available, greater numbers of developers have used them. Almost one in four devs has no plans to use AI coding tools this year, whether because those tools aren't available to them, they don't see a use case in their current work, because they lack trust in AI coding tools, or for some other reason.
Most respondents (72%) this year said their attitude toward these tools was favorable or very favorable. That's down from 77% of respondents with favorable or very favorable feelings toward AI coding tools in 2023. This drop suggests that as more developers have begun using these tools, more of them have been disappointed in the quality or reliability of their output.
And developers' trust in AI remains largely an open question. This year, about 42% of our respondents told us affirmatively that they trust the accuracy of AI output in their workflows, compared with 31% who say they don't trust the output. Another 27% said they neither trusted nor distrusted AI output. For these folks, the jury is still out, perhaps because they haven't had a chance to form a strong opinion one way or the other.
Developers see the benefits of AI tools -- for certain tasks
Developers clearly recognize the benefits of AI coding tools, according to our survey. Asked to cite the biggest benefit of these tools, developers responded with:
Increasing productivity (81%)Speeding up learning (62%)Improving efficiency (58%)Improving accuracy in coding (30%)
That said, developer confidence in AI tools to perform complex tasks has a long way to go. Nearly half of professional developers (45%) said AI tools are bad or very bad at handling complex tasks.
Interestingly, professional developers also agree that these shortcomings aren't attributable to user error or a lack of training. Only 30% of professional developers blamed user error or lack of training for the challenges they encounter with AI tools at work. That's less than half the percentage of those who simply don't trust the AI tool.
When asked for their top concerns about AI within an organization or team, 66% of our respondents cited a distrust of the output of AI tools, while 63% said the tools lack crucial context necessary to understanding their organization's codebase, internal architecture, and institutional knowledge.
Despite these reservations about AI's shortcomings, developers largely agree that AI tools will become more integrated with their workflows for documenting (81%), testing (80%), and writing code (76%) over the next year. In other words, these tools aren't going anywhere.
Even with AI, developers struggle to find the knowledge they need
It's clear from our survey that even with the assistance of AI-powered coding tools, developers can't always find the information they need when they need it. Tracking down technical and/or institutional knowledge often requires meandering side quests that disrupt your workflow and hamper your productivity. We found that:
Waiting for answers disrupts developers' workflows, and devs can't always find the accurate, timely answers they need to do their jobs well. More than half of respondents (53%) agree or strongly agree that waiting on answers disrupts their workflow, while less than half feel that they can easily surface the up-to-date information they need to do their jobs.Developers spend a lot of time looking for answers to their questions. More than 60% of respondents reported spending 30 minutes or more a day searching for solutions, with one in four spending 60 minutes or more every day looking for answers. Half an hour or an hour might not seem like a huge chunk of your day (though if it doesn't, I'm frankly jealous), but it adds up rapidly. Every moment you spend looking for answers is a minute you can't spend doing something else -- more productive, more creative, or just more fun.Developers spend a lot of time answering other people's questions -- often repeatedly. Three out of four developers find themselves answering questions they've answered before, while close to half (47%) spend 30 minutes or more a day answering questions. Again, an hour or half an hour might not seem like all that much, but it adds up. And as any seasoned developer can tell you, the most knowledgeable team members who are the best at answering others' questions get tapped to do it again and again, cutting into time they could be spending on more fulfilling or creative projects.Knowledge silos make developers less productive. Knowledge silos are situations in which one person or one team has crucial information that hasn't been shared throughout the organization. For 30% of developers, knowledge silos impact their productivity 10 times a week or even more often -- that's an average of twice a day for a five-day workweek. See what we mean about these little things adding up?
As we've written before, the field of software engineering rewards a culture of continuous learning (and sharing of knowledge). Looking for answers to your questions and answering others' queries are integral parts of a developer's job, and most devs know it. They enjoy it! But they want to be able to engage in this part of their job without it derailing their other projects or wreaking havoc with their ability to focus.
That's why our paid platform now includes AI features designed to surface relevant, trustworthy knowledge in the natural course of developer workflows: we think (and, survey says, you largely agree) that AI has enormous potential to increase your coding productivity and efficiently, help you learn new skills and languages more quickly, and generate tests to help assess and improve software quality.
