By Martin Abel and Reed Johnson
While much recent research has focused on how people say they prefer creative works made by humans over AI, we don’t yet know whether these stated preferences translate into consumer behavior. And, as the saying goes, talk is cheap. But for the millions of people worldwide employed in creative industries, the concern of whether people will be willing to pay more for human-made art is more than an idle academic question. Amidst the coming avalanche of AI-generated work, it is a question of real livelihoods—and more broadly, a question of what will remain ours in this most human of endeavors.
An experiment with AI-generated creative writing
To investigate these questions, we turned to AI-generated creative writing. We asked OpenAI’s GPT-4 to generate a short story in the style of the critically acclaimed author Jason Brown. We then recruited a nationally representative sample of over 650 people and asked participants to read and assess this story. Crucially, only half the participants were told that the story was written by AI, while the other half were misled into believing it was the work of Jason Brown. This design allows us to isolate the effect of (perceived) authorship and to test if consumers value human over AI writing.
After reading the first half of the AI-generated story, participants were asked to rate the quality of the work along various dimensions, such as whether they found it predictable, emotionally engaging, atmospheric, etc. We also measured participants’ willingness to pay in order to read to the end of the story. To capture a fuller picture of participants’ investment in the story, we included both monetary measures (giving up a portion of their pay for participating) and time measures (their willingness to work doing a routine transcription task).
Subjective assessments vs. actual behavior
So, were there differences between the readers who knew the work was AI-generated versus those who didn’t? The short answer: yes. But a closer analysis reveals some startling results.
To begin with, the group that knew the story was AI-generated had a much more negative assessment of the work, rating it more harshly on dimensions like predictability, authenticity, and atmosphere. These results are largely in keeping with a nascent but growing body of research that shows bias against AI in areas like visual art, music, or poetry. It seems that, at least at present, consumers reflexively ascribe lower judgments of quality to AI-labeled work.
However, notwithstanding these perceptions of lower quality, participants were ready to invest the same amount of money and working time to finish reading the story, whether or not it was labeled as AI. Participants also did not spend less time actually reading the AI-labeled story. When asked afterwards, almost 40% of participants said they would have paid less if the same story had been written by AI versus a human, highlighting that many are not aware of the discrepancies between their subjective assessments and actual choices.
Implications for the future of the creative industry
These findings provide timely evidence that widely documented professed AI biases may not be reliable indicators of willingness to pay for human creative labor. The potential implications for the future of human-created work are profound, especially in market conditions where AI-generated work can be orders of magnitude cheaper to produce. With the technology still in its infancy, AI-made books are already flooding the market, recently prompting the Authors Guild to instate its own labeling guidelines. However, our research raises questions about whether these labels are effective in stemming the tide.
True, attitudes toward AI are still forming, and we may well see a backlash against AI-generated creative works. After all, similar shifts occurred in the wake of mass industrialization, such as the Arts and Crafts Movement. This response may take the form of market segmentation, where some consumers will be willing to pay more based on the process of creation, while others may be interested only in the product.
Regardless of how these scenarios play out, our findings indicate that the road ahead for human creative labor might be more uphill than previous research suggests. At the very least, we see that, while consumers may hold beliefs about the intrinsic value of human labor, many seem unwilling to put their money where their beliefs are.