Study Reveals Disconnect Between Stated Preferences and Actual Behavior in AI-Generated vs. Human-Written Stories

People say they prefer stories written by humans over AI-generated works, yet new study suggests that's not quite true

https://theconversation.com/people-say-they-prefer-stories-written-by-humans-over-ai-generated-works-yet-new-study-suggests-thats-not-quite-true-251347 People say they prefer a short story written by a human over one composed by artificial intelligence, yet most still invest the same amount of time and money reading both stories regardless of whether it is labeled as AI-generated. That was the main finding of a study we conducted recently to test whether this preference of humans over AI in creative works actually translates into consumer behavior. Amid the coming avalanche of AI-generated work, it is a question of real livelihoods for the millions of people worldwide employed in creative industries. To investigate, we asked OpenAI's ChatGPT 4 to generate a short story in the style of the critically acclaimed fiction author Jason Brown. We then recruited a nationally representative sample of over 650 people and offered participants US$3.50 to read and assess the AI-generated story. Crucially, only half the participants were told that the story was written by AI, while the other half was misled into believing it was the work of Jason Brown. After reading the first half of the AI-generated story, participants were asked to rate the quality of the work along various dimensions, such as whether they found it predictable, emotionally engaging, evocative and so on. We also measured participants' willingness to pay in order to read to the end of the story in two ways: how much of their study compensation they'd be willing to give up, and how much time they'd agree to spend transcribing some text we gave them. So, were there differences between the two groups? The short answer: yes. But a closer analysis reveals some startling results. To begin with, the group that knew the story was AI-generated had a much more negative assessment of the work, rating it more harshly on dimensions like predictability, authenticity and how evocative it is. These results are largely in keeping with a nascent but growing body of research that shows bias against AI in areas like visual art, music and poetry. Nonetheless, participants were ready to spend the same amount of money and time to finish reading the story whether or not it was labeled as AI. Participants also did not spend less time on average actually reading the AI-labeled story. When asked afterward, almost 40% of participants said they would have paid less if the same story was written by AI versus a human, highlighting that many are not aware of the discrepancies between their subjective assessments and actual choices. Why it matters Our findings challenge past studies showing people favor human-produced works over AI-generated ones. At the very least, this research doesn't appear to be a reliable indicator of people's willingness to pay for human-created art. The potential implications for the future of human-created work are profound, especially in market conditions in which AI-generated work can be orders of magnitude cheaper to produce. Even though artificial intelligence is still in its infancy, AI-made books are already flooding the market, recently prompting the authors guild to instate its own labeling guidelines. Our research raises questions whether these labels are effective in stemming the tide. What's next Attitudes toward AI are still forming. Future research could investigate whether there will be a backlash against AI-generated creative works, especially if people witness mass layoffs. After all, similar shifts occurred in the wake of mass industrialization, such as the arts and crafts movement in the late 19th century, which emerged as a response to the growing automation of labor. A related question is whether the market will segment, where some consumers will be willing to pay more based on the process of creation, while others may be interested only in the product. Regardless of how these scenarios play out, our findings indicate that the road ahead for human creative labor might be more uphill than previous research suggested. At the very least, while consumers may hold beliefs about the intrinsic value of human labor, many seem unwilling to put their money where their beliefs are. The Research Brief is a short take about interesting academic work.

People say they prefer stories written by humans over AI-generated works, yet new study suggests that's not quite true

People say they prefer a short story written by a human over one composed by artificial intelligence, yet most still invest the same amount of time and money reading both stories regardless of whether it is labeled as AI-generated. That was the main finding of a study we conducted recently to test whether this preference of humans over AI in creative works actually translates into consumer behavior. Amid the coming avalanche of AI-generated work, it is a question of real livelihoods for the millions of people worldwide employed in creative industries. To investigate, we asked OpenAI's ChatGPT 4 to generate a short story in the style of the critically acclaimed fiction author Jason Brown. We then recruited a nationally representative sample of over 650 people and offered participants US$3.50 to read and assess the AI-generated story. Crucially, only half the participants were told that the story was written by AI, while the other half was misled into believing it was the work of Jason Brown. After reading the first half of the AI-generated story, participants were asked to rate the quality of the work along various dimensions, such as whether they found it predictable, emotionally engaging, evocative and so on. We also measured participants' willingness to pay in order to read to the end of the story in two ways: how much of their study compensation they'd be willing to give up, and how much time they'd agree to spend transcribing some text we gave them. So, were there differences between the two groups? The short answer: yes. But a closer analysis reveals some startling results. To begin with, the group that knew the story was AI-generated had a much more negative assessment of the work, rating it more harshly on dimensions like predictability, authenticity and how evocative it is. These results are largely in keeping with a nascent but growing body of research that shows bias against AI in areas like visual art, music and poetry. Nonetheless, participants were ready to spend the same amount of money and time to finish reading the story whether or not it was labeled as AI. Participants also did not spend less time on average actually reading the AI-labeled story. When asked afterward, almost 40% of participants said they would have paid less if the same story was written by AI versus a human, highlighting that many are not aware of the discrepancies between their subjective assessments and actual choices. Why it matters Our findings challenge past studies showing people favor human-produced works over AI-generated ones. At the very least, this research doesn't appear to be a reliable indicator of people's willingness to pay for human-created art. The potential implications for the future of human-created work are profound, especially in market conditions in which AI-generated work can be orders of magnitude cheaper to produce. Even though artificial intelligence is still in its infancy, AI-made books are already flooding the market, recently prompting the authors guild to instate its own labeling guidelines. Our research raises questions whether these labels are effective in stemming the tide. What's next Attitudes toward AI are still forming. Future research could investigate whether there will be a backlash against AI-generated creative works, especially if people witness mass layoffs. After all, similar shifts occurred in the wake of mass industrialization, such as the arts and crafts movement in the late 19th century, which emerged as a response to the growing automation of labor. A related question is whether the market will segment, where some consumers will be willing to pay more based on the process of creation, while others may be interested only in the product. Regardless of how these scenarios play out, our findings indicate that the road ahead for human creative labor might be more uphill than previous research suggested. At the very least, while consumers may hold beliefs about the intrinsic value of human labor, many seem unwilling to put their money where their beliefs are.