Generative AI for surveys on payment apps: AI views on privacy and technology
Summary
Focus
The study examines whether artificial intelligence (AI) can simulate how people think about mobile payment apps, especially regarding privacy risks and perceived benefits. Using OpenAI's ChatGPT, we generate artificial survey responses and compare them with real survey data from the Netherlands. The goal is to see if AI can reproduce realistic human attitudes, such as some people using apps despite privacy concerns while others avoid them. This highlights the broader question of whether AI can act as a substitute or support tool for traditional surveys.
Contribution
The paper demonstrates a new use of generative AI in survey research. It shows that AI can mimic key behavioural patterns found in real data, such as differences between users and non-users and varying privacy attitudes. It also introduces a structured method for designing AI-based surveys using personas and prompts. Importantly, it highlights how AI can help researchers test ideas, design surveys and generate preliminary insights quickly and at low cost.
Findings
The results show that AI can broadly replicate human patterns: users see more benefits and fewer risks, while privacy-concerned individuals are more cautious. However, AI responses lack diversity and tend to overemphasise privacy concerns, producing biased and less varied results than real surveys. Changes in prompt design also significantly affect outcomes. Overall, AI is useful as a complementary tool, but it cannot fully replace human surveys due to limited variability and potential bias.
Abstract
This study uses ChatGPT to simulate survey responses about payment apps, focusing on privacy and perceived benefits. By designing prompts that mirror real user characteristics, the generated responses align with findings from a Dutch survey, especially when grouped by privacy concern. Privacy-concerned agents view apps less favorably, while users show more positive attitudes than non-users, even without such traits in the prompt. However, ChatGPT fails to match the real survey's response variability and tends to overstate privacy concerns. These results indicate that generative AI can complement but not replace human surveys for studying perceptions of payment tools.
JEL Classification: M31, C83, C45, D12, L86
Keywords: ChatGPT, generative artificial agents, privacy paradox, Westin index, survey, payments
