Understanding beliefs, preferences, and motivations is central to social sciences, yet eliciting these dimensions can be challenging. A new IZA dicussion paper by Vincenzo Galasso, Tommaso Nannicini, and Debora Nozza explores how the mode of response—spoken versus written—impacts the quality of open-ended survey answers. Through randomized controlled trials across three survey topics (AI, public policy, and international relations), the researchers highlight the distinct advantages of audio responses.
Oral answers are longer and more personal
Analyzing 7,766 open-ended responses, the study reveals that participants providing audio responses offered longer answers with more personal reflections compared to written responses. While oral responses tended to have simpler lexical structures, they conveyed more information overall. Using large language models (LLMs) to assess informativeness, the researchers found that spoken answers scored higher in providing nuanced reasoning and personal experiences.
Oral responses stood out for their spontaneity and emotional engagement, often reflecting the respondent’s immediate thoughts and feelings. This contrasts with written responses, which tend to be more filtered and less personal. The study demonstrates the value of integrating audio surveys into research, particularly for capturing subjective experiences and individual reasoning processes.
Implications for survey design and policymaking
These findings suggest that audio responses can provide researchers and policymakers with deeper insights into public attitudes, preferences, and motivations. However, challenges such as higher attrition rates among audio respondents and potential biases from unfamiliarity with voice recording need to be addressed. Future research could explore ways to optimize audio survey methods and incorporate tools like emotion recognition to further enhance data quality.