Readers Prefer AI-Generated Stories Over Human Writing, Study of 1,682 Adults Reveals

Most people prefer AI-written stories - and can't tell when they are made by humans | New Scientist

AI-generated literature may have an unfair reputation for poor quality. Despite a cultural backlash to AI-generated content, people seem to prefer it - and if they're lied to and told that it was created by a human, their opinion rises even higher. Sydney Sears and Deena Weisberg at Villanova University in Pennsylvania recruited 1682 adults - selected to be representative of the US in terms of their gender, age, and race - and asked them to read a story. Half the participants read a story written by a human, and half read one written by AI. Each of these groups was again split in two, with half told that their story was written by a human and half that their story was written by AI, regardless of its true origin. The human-created stories were sourced from a literary journal, while the AI-generated ones were created using ChatGPT 4.0. Those who read an AI-generated story ranked its quality as 1.54, on average, when asked to score it on several qualities on a scale ranging from -3 to 3. Meanwhile, participants who read a human-written story gave it an average score of just 0.97. Participants who read an AI-generated story also ranked it as more absorbing, based on several qualities on a similar scale, with an average score of 1.42. Those who read a human-written story, meanwhile, gave an average score of just 1. In all cases, the story was rated slightly higher if the participant was told it was written by a human. The researchers also ran two further studies, where several hundred people were given an AI-generated story and a human-written story and asked to identify which was which. In the first test, only 39 per cent made a correct identification, while in the second, 52 per cent were correct. The researchers have no explanation for the difference, except that the second test took place several months after the first; society may be adapting to better understand AI output. In any case, they claim that it shows people are no better than chance at distinguishing human work from AI output. Claire Hardaker at Lancaster University, UK, says that humans are having to come to terms with the uncomfortable idea that we aren't any better than robots at writing short stories. She says that in her own research on testing people's ability to identify AI-generated speech, text and music, it is the music that angers them most of all when they are tricked. "I think there's a huge sensitivity [around AI generating art]," she says. "When you have AI that is continually improving at the speed that it's improving at, there is going to come a point where every single model can produce Shakespeare-like output. What does that mean for humans?" While AI is becoming increasingly capable with every passing month, the concept of AI-generated literature and art is creating a backlash. Earlier this year, a short story given the Commonwealth prize and published in Granta was accused by some of being AI-generated - something which the competition explicitly forbade. An investigation was inconclusive. Last year, artists criticised an auction of AI-generated art at Christie's because they saw the work as "mass theft", due to models being trained on existing images. Rodney Jones at the University of Reading, UK, says that part of the preference for AI-generated writing in this case may be down to researchers taking stories from a literary journal, rather than choosing a more accessible and popular genre. "People like expected things. People like easy-to-process things. Real human writing, real kind of human literary writing, is, frankly, not very popular, because it's hard work," says Jones. Although ChatGPT was asked to create new stories for the experiment that had traits similar to those of the human-written stories, Jones thinks it would have produced a simpler, easier-to-read version by default. Weisberg agrees that AI produces more "vanilla" content than humans do, and ascribes the perceived difference in quality to this. She plans to run experiments in the future on different types of writing to see if AI retains its lead. "I think AI will be able to accomplish some truly amazing feats of creativity, but I think AI creativity is just different from human creativity," says Weisberg. "I think it will be able to do something that we will recognise as having artistic value, but it might be measured on a different scale."

Most People Prefer AI Writing, but That's Because It's Trained on Us - CNET

You probably think you prefer writing created by a human. But a new study is proving that you may not actually. You may just like things that sound human. Led by Cambridge University Press and researchers at Villanova University, the study asked its 1,682 participants aged 18 to 81 to read a few fictional stories to determine whether they could tell if they were AI-generated, and then asked them to rate the quality of each story. The study consisted of three separate experiments. In part one, all participants were given a story - some told it was AI-generated, others told it was human-written - and then rated its quality and engagement. In the second and third experiments, the participants were divided into two groups: one with 424 participants and the other with 481. Each person was given one human-written and one ChatGPT-generated story, though it was not disclosed which was which, and they were asked to identify which story they believed was authored by ChatGPT. They found that the participants rated stories generated by ChatGPT highest, especially when they were intentionally and incorrectly told the story had been written by a human. Researchers also found that many (over 60% in experiment two and over 48% in experiment three) couldn't correctly identify which content was AI-generated. Deena Weisberg, senior author of the study and professor in the Department of Psychological & Brain Sciences at Villanova University, said this shows a bias toward "narratives written by real people." "We assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience, which leads people to underestimate AI's capabilities," she said. "I'd say that public assumptions about AI's capabilities are increasingly out of date." Why do people actually prefer AI writing? There is a growing portfolio of evidence that people like AI-generated writing, including short stories, essays and poetry. A 2023 study led by Renee Richardson Gosline of MIT found similar results, concluding that AI-generated content and augmented AI (the AI made the final decision with human output as a reference) were seen as higher quality than that produced by human experts and augmented human experts (the human makes the final decision with AI output as a reference). Weisberg speculates that this is likely due to AI-written stories being more direct, explicit and simple, whereas human-written stories tend to be more subtle or complex and leave room for interpretation. For instance, a snippet from one of the AI-generated stories given to the participants reads: "So now, as I sit by the pond, the autumn leaves falling gently around me, I think of my mother, of the stories she shared, of the wisdom she imparted. The koi continue their dance beneath the water's surface, oblivious to the change they have witnessed in my life. And in their simple, unchanging world, I find a sense of comfort, a reminder that life, in all its complexity and uncertainty, goes on." Comparatively, a human-generated story has a portion that reads: "Perhaps, I think, we are all experiencing a kind of birth after all, she laboring and I midwifing, her husband playing the role of nervous father, not sure what he'll do after the sacred event occurs, not sure what to do now. We swim in uneasy circles around each other, and all the while the golden sun continues to rise and set, keeping its own mysterious time." The AI-generated text as a whole is much more direct, uses simpler sentences, doesn't rely on anecdotes or literary devices like similes and metaphors, and has much shorter paragraphs. In essence, it's not that the participants have bad taste or that the large language model powering the chatbot has any skill a human writer lacks; it's more that AI writing often incorporates tropes, syntax, language and styles that people already like. This outcome may not be all that surprising when we're reminded that these AI systems are trained on writing we, as humans, have enjoyed and consumed for decades. "We hear a lot about the crisis in literacy, shrinking attention spans and the influence of platforms, such as TikTok," Weisberg said. "But I don't think we can say that our findings in this study can be entirely explained by contemporary media consumption." "It's more that these technologies amplify existing tendencies, rather than creating them."

AI-generated tales preferred to human-written ones, study finds

Readers gave the highest ratings to stories created by artificial intelligence (AI), believing they had been written by humans, according to new research. The study, published by Cambridge University Press, tested to see whether people aged 18 to 81 could tell the difference between human-written and AI-generated short stories. Led by researchers at Villanova University (VU), USA, it found that participants could not tell which was which. Senior author Dr Deena Weisberg, from the department of psychological and brain sciences at VU, said: "I'd say that public assumptions about AI's capabilities are increasingly out of date." The study also examined how participants rated the appeal and quality of each story and whether they could identify AI-generated content. "We assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience, which leads people to underestimate AI's capabilities," Weisberg added. Researchers used three fictional short stories written by published authors, alongside three generated by ChatGPT. In their first experiment, 1,682 participants read a story and were either correctly or incorrectly told it had been written by AI or a human. They then rated how engaging it was and its quality. In a second and third experiment, 424 and 481 people received one human-written story and one by AI, and were not told the author. After reading, they were asked to identify them. Weisberg said: "In our study, participants gave the highest ratings to stories that they had been told were written by humans but which were actually written by AI. "I'd argue that this reveals a bias towards narratives written by real people." Researchers also found that people who said they had greater AI literacy were better able to identify AI-written text, but participants who said they had greater expertise in literature were not. Participants were also asked to rate their experience with fictional literature and using AI platforms, with AI-generated stories receiving a higher rating for quality. "AI writing tends to be clearer, more direct and easier to process," said Weisberg. "By contrast, human-written stories are often more subtle and complex. "Perhaps people generally prefer predictability, because difficult or subtle material requires more brain power." Do you have a story suggestion for Cambridgeshire? Contact us below. Follow Cambridgeshire news on BBC Sounds, Facebook, Instagram and X.

Study finds readers preferred AI-generated stories

A handful of short stories were put in front of readers with one simple question: Which ones were best? The top-rated stories were generated by ChatGPT, beating fiction written by human authors published in a literary journal. People are increasingly encountering writing created by both humans and AI, often without knowing which is which. Readers in this test could not reliably identify the author, and they gave stories higher ratings when they believed a person had written them. The research comes from Villanova University and appears in Judgment and Decision Making, a Cambridge University Press journal published for SJDM. The findings add to a growing body of evidence that people judge writing differently once they believe it was created by AI. Readers compared AI and human stories Sydney Sears and Deena Skolnick Weisberg built the study around six stories. Three were fiction published by human authors in a reputable literary journal or short story collection. The other three were ChatGPT versions of the same stories. Every participant was an adult, ranging in age from 18 to 81. In the first experiment, 1,682 readers each read a single story. Half were told the truth about who wrote it. Half were told the opposite. They then rated the story's quality and how absorbed they felt while reading it. Absorption was measured with an edited Story World Absorption Scale, covering attention, transportation, emotional engagement, and mental imagery. AI stories earned higher ratings The AI stories were rated higher on both measures. Readers called them better written, and said they were pulled in further. The label mattered too. The very highest ratings went to AI-generated stories that participants had been told a human wrote. Deena Weisberg is the senior author and a researcher in the Department of Psychological and Brain Sciences at Villanova. She reads the result as a bias rather than a verdict on the writing. "In our study, participants gave the highest ratings to stories that they had been told were written by humans but which were actually written by AI," says Weisberg. "I'd argue that this reveals a bias toward narratives written by real people. We assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience, which leads people to underestimate AI's capabilities." "I'd say that public assumptions about AI's capabilities are increasingly out of date," she says. Can people spot AI writing? The second and third experiments handed each reader one human story and one AI-generated story, with no hint about origins. The task was to say which was which. In the second experiment, readers picked correctly 39.93% of the time. That is worse than a coin flip, meaning they were steered wrong rather than merely confused. In the third, accuracy reached 51.97%, statistically no different from chance. Between them, 905 adults took the test. Weisberg expected those errors to run in one direction. They did not. "I was most surprised by the mistakes that participants made when they were asked to guess which story was human-written and which was AI-generated. I had expected that the mistakes would be in one direction, specifically that people would tend to say that AI-generated writing was human-written. But that's not what we found," Weisberg told Earth.com. Readers relied on the wrong cues "Participants were equally likely to get the question wrong by saying that an AI-written story was written by a human and by saying that a human-written story was written by AI. What that suggests to me is that our participants were using the wrong metrics to judge which story was which." "Participants may have come into the study with some ideas about what characterizes human writing and AI writing, but those ideas seemed to be wrong," she says. Experience with literature offered no edge. Readers who described themselves as well-read in fiction were no better at spotting the machine. AI familiarity improved detection One thing did predict success, and it was familiarity with AI itself. In the second experiment, each one-point rise in self-reported AI expertise raised the odds of a correct guess by 14%. The third experiment used the Artificial Intelligence Literacy Scale, a previously validated measure. There, a one-point gain raised the odds by 33%. That advantage appears to come from pattern recognition rather than literary taste. The tells are small, stylistic, and learnable. "One thing to note was that familiarity with AI systems appeared to help people recognize the patterns typical of AI-generated writing, such as em dashes and sentence structures such as 'it's not just X, it's Y,'" says Weisberg. "That suggests that improving AI literacy would be one way to help people to navigate the new AI-enabled world that we're living in." Fiction gives readers less practice Asked what a reader could practically train themselves to notice, Weisberg points to plain exposure. "The main thing that people can do is familiarize themselves with what AI-generated output looks like... In general, we found that participants with higher self-reported levels of experience with AI systems were better at telling the difference between human-written and AI-generated stories. The more you get to know the style of AI writing, the easier it is to spot," she says. There is a catch in that advice. Most people meet machine writing in work reports and student essays, not in fiction. "...my impression is that AI systems are mostly being asked to generate nonfiction or expository writing, like reports for work or student essays. If that's true, that means that people generally have less familiarity with AI-generated creative writing, which may make it harder to spot," says Weisberg. Clear writing influenced ratings Weisberg's explanation for the preference is simple. AI writing tends to be clean, and clean prose is easy to read. "AI writing tends to be clearer, more direct and easier to process. By contrast, human-written stories are often more subtle and complex," says Weisberg. "For example, the AI versions of our stories usually stated their themes explicitly, rather than allowing readers to infer meaning from the characters' words and actions." Large language models absorb the statistical patterns of the text they are trained on and reproduce them. Explicit, tidy prose is one of those patterns. "We hear a lot about the crisis in literacy, shrinking attention spans and the influence of platforms such as TikTok. But I don't think we can say that our findings in this study can be entirely explained by contemporary media consumption," says Weisberg. "It's more that these technologies amplify existing tendencies, rather than creating them. Perhaps people generally prefer predictability, because difficult or subtle material requires more brain power." Where AI writing goes next The study rests on three story pairs and one AI system. Other models, other genres, and longer work could all behave differently. The experiments also ran on an earlier version of ChatGPT. Weisberg does not expect a newer one to swing the result back toward human authors. "The models have gotten more advanced in what they're able to do, so if anything, they now should be able to generate short stories that seem even more human-like," she says. Readers also judged the stories once, in a single sitting. Nothing here says a machine can write a novel that people return to for years. Broader impacts of AI writing What the results do show is more limited, but no less relevant. Readers cannot reliably tell the difference, and the label alone moves their judgment. That gap matters beyond fiction. AI writing assistants already shift opinions in social media posts, and chatbots draw more empathy than doctors in patient conversations. The same tools have pushed invented citations into peer-reviewed papers. In each case, readers judged the text before knowing where it came from. The study is published in the journal Judgment and Decision Making. -- - Like what you read? Subscribe to our newsletter for engaging articles, exclusive content, and the latest updates. Check us out on EarthSnap, a free app brought to you by Eric Ralls and Earth.com.

AI-generated stories rated better quality than human-written ones, study finds

Researcher says simpler writing by AI is easier to read and digest, but this does not mean human authors are obsolete From Homer's Odyssey to Agatha Christie's crime fiction, humans have produced plenty of tales to grace the bookshelf. But research suggests artificial intelligence may now be providing some competition. In a study published in the journal Judgment and Decision Making, 1,682 adults were asked to read one of six short stories, three of which were written by humans and three by ChatGPT. Each AI story had a similar theme to one of the human-authored works. The team told participants whether their story was written by a human or AI, but this information was not always correct. The researchers then asked participants to rate how absorbing and engaging they found the story, and its quality. Participants who read an AI-generated story rated it as more absorbing and of higher quality than those who read a story written by a human. However, participants gave higher ratings to stories they had been told were written by people. Dr Deena Skolnick Weisberg, a senior author of the research from Villanova University in Pennsylvania, said: "AI systems can already generate short stories that are seen as being at least as good as - if not better than - human-written stories. We should update our views of AI's abilities accordingly." However, Weisberg, who is herself a creative writer, said that did not mean writing should be left to AI, noting that novels produced by tech were probably going to be different from those written by humans. "We may need to make room for AI-generated novels, and for AI/human co-written novels, but that doesn't mean that there's no longer space for us to appreciate the process of human creativity," she said. "Humans write as a form of creative self-expression or to challenge ourselves, or to make sense of our experiences. The fact that AI can generate human-like stories doesn't change any of that." The results also suggest attitudes to AI matter: participants with more positive attitudes toward AI gave higher ratings to stories they were told were produced by ChatGPT than those with a less positive outlook. The researchers carried out two further experiments in which 905 adults were given two of the six short stories - both on the same theme and only one of which was penned by a person. They were then asked to guess which was written by a human and which by AI. In one experiment, about 40% guessed correctly, a result that is worse than chance, while in the other experiment 52% guessed correctly. "There is no general evidence that participants were able to distinguish human-written from AI-generated stories," the authors write. The researchers found that while participants' self-reported expertise with fiction was not linked to how accurately they guessed, greater expertise with AI systems was. "AI writing has a particular style that we can learn to recognise with practice - just as we can learn to recognise different human authors' unique styles," Weisberg said. Weisberg suggested that the preference for stories labelled as being written by humans was linked to a desire for authenticity, but that people actually preferred AI-generated stories because they were easier to read and digest. However, Luke Kennard, a professor of film and creative writing at the University of Birmingham, who was not involved in the work, emphasised that AI was highly contentious. "Because we're human, we tend to personify AI and imagine it as some kind of glowing robot with a benign and neutral face, when it's really just predictive code based on a massive stolen database of actual writing, physically based in vast, expensive and destructive datacentres with catastrophic repercussions for the surrounding communities," he said. Kennard also stressed that AI was not a "tool" similar to spellcheck or a thesaurus, but an existential threat. "Of course it's capable of spitting out a coherent and even enjoyable poem with the right prompting and finessing. Given that it's been trained on thousands of years of human work, it would be embarrassing if it wasn't," he said. "I suppose the question we should be asking is: is it worth it? And the answer is absolutely, categorically, it is not."

Study finds readers rate AI-written stories higher, but still trust the "human" label more

Researchers also found that readers could barely differentiate between AI-generated and human-written stories, though those familiar with AI tools guessed somewhat better. If you think AI writing is easy to spot, a new study suggests you may be overestimating your ability to tell the difference. Researchers from Villanova University have found that readers struggled to tell AI-generated stories from human-written ones and frequently rated the AI versions higher. The study, led by Dr. Deena Skolnick Weisberg and published in the journal Judgment and Decision Making (via The Guardian), asked more than 1,600 participants to rate one of six short stories, three written by a human and the rest generated by ChatGPT, on quality and engagement. Participants were told who wrote their story, though that label wasn't always accurate. AI stories scored higher, but the "human" label still won points Readers who got an AI-generated story rated it as more absorbing and higher quality than readers who got a human-written one. At the same time, stories labeled as human-written scored better than those labeled AI, regardless of who actually wrote them. Weisberg says the results show AI has become far more capable of producing readable, engaging fiction than most people assume. But that doesn't mean human authors have become irrelevant, she added. Most readers couldn't spot the difference In a follow-up test, participants were given one human-written and one AI-written story on the same theme and asked to identify which was which. They largely failed, guessing correctly only about as often as random chance would predict. The one group that performed noticeably better was people who described themselves as more familiar with AI systems, suggesting that spotting AI writing may be a learned skill rather than an instinct. Recommended Videos The Commonwealth Short Story Prize ran into this exact problem earlier this year, when readers online flagged three of its five winning entries as AI-generated, and an AI-detection tool backed them up. The prize's own publisher ran a separate check, and it came back inconclusive. If professional judges and readers with a reputation on the line can't settle the question, expecting casual readers to spot the difference is asking too much.

Is AI Telling Better Stories Than Us? Why Human Writers May Be Less Appealing

Not only are people unable to tell the difference between stories written by humans or artificial intelligence, they actually prefer it when the stories are created by AI. And that's especially true when they are told that the stories had human authors, according to a new study from Villanova University. The study consisted of multiple experiments, all of which indicated a preference for AI. In the first, 1,682 participants were told to rate both the quality of six short stories they were given and how "engaging" they were. They were told, either correctly or incorrectly, who or what had authored each one. Not only were AI-generated stories found to be higher-quality (by 6%) and more engaging (by 8%), they were rated "even more highly," researchers concluded, when participants believed that they were written by humans (by an additional 3%). Deena Weisberg, the senior author of the study and an associate professor from Villanova's Department of Psychological & Brain Sciences, said in a press release that the finding "reveals a bias towards narratives written by real people." That may be because "we assume creative writing requires uniquely human qualities, such as emotional understanding and lived experience," she said, explaining that it shows how "public assumptions about AI's capabilities are increasingly out of date." Furthermore, people were only able to identify AI-generated content roughly 40% and 52% of the time in two subsequent experiments, each with over 400 participants. People who said they were AI-literate had more success, while people who claimed to have a background in literature were less accurate in their guesses. "Familiarity with AI systems appeared to help people recognize the patterns typical of AI-generated writing, such as em dashes and sentence structures such as 'it's not just X, it's Y,'" Weisberg said. "That suggests that improving AI literacy would be one way to help people to navigate the new AI-enabled world that we're living in." Cameron Jones, assistant professor of psychology at Stony Brook University tells TIME that the study's findings are not necessarily surprising, considering the history of research going back years that shows people's diminishing ability to tell between artificial intelligence and humans. He also points to the trajectory of his own studies, which look at whether people can differentiate between them in conversation. In those scenarios, he says, "The person actually gets to quiz the model and ask follow-up questions." Even then, they had trouble distinguishing between human and generative content partners. Jones says that this is because the large language models driving the content responses are trained to appeal to users. "We basically get the model to generate little bits of text, and then we get people to read them, and they give a thumbs up if they like what the model's saying and they give a thumbs down if they don't like what the model's saying," he says. It's no surprise that people prefer the results, he explains, when "we're optimizing these models' outputs against our preferences." It can oversimplify output, but it leads to writing with higher overall appeal -- whereas humans might write something truly exceptional that only caters to select tastes. "Real humans are weird and idiosyncratic, and they're very different from one another," Jones says. "But models can kind of learn to be this kind of milquetoast everyman who appeals to everybody." While the findings align with rising exposure to artificial intelligence, along with algorithmic reward systems for broadly approachable content on platforms like TikTok, Weisberg does not think that the current digital landscape is to blame for the results. She says, "Technologies amplify existing tendencies, rather than creating them." Plus, she adds, the human-authored content might have simply challenged readers more. "AI writing tends to be clearer, more direct and easier to process," she said. And it's reasonable for participants to find themselves more deeply engaged with content that offers greater predictability and less friction. In other words, Weisberg says, "Difficult or subtle material requires more brainpower."

Villanova University study: AI fiction scores higher, but human labels win trust

Villanova University has published a new study in Judgment and Decision Making with a result plenty of writers probably won't enjoy: in blind ratings, machine-written short stories scored higher than human-written ones on both absorbing and quality. The responses came from more than 1,600 readers. A follow-up test with more than 900 participants found that readers could identify the machine-written story at about the rate you'd expect from random guessing, though text generator experience were better at spotting it. Lead researcher Dr. Deena Skolnick Weisberg said one reason may be that simpler, easier-to-process prose helps these stories move more smoothly on the page. That changed once labels were added. When a story was described as human-written, it got higher scores no matter who wrote it. That lines up with a 2024 Bynder survey in which 56% of readers picked the machine-written article when reading blind, and with separate 2026 findings suggesting that news produced by a person using software was seen as more trustworthy than copy generated entirely by a machine. If you follow AI writing, this study is worth reading because it gets at the same issue now showing up in publishing disputes, including the Commonwealth Short Story Prize controversy over entries flagged as machine-generated. You can read the study in Judgment and Decision Making.