Abstract
Conducting electronic surveys in remote areas can be challenging due to the lack of the required Internet infrastructure. Traditional phone surveys are typically used in such cases since mobile phones have become pervasive. However, this process can be time-consuming since a human is required to conduct the session and they must then upload responses to a database. We propose using Large Language Models (LLMs) to process an audio recording of the phone session to extract the responses and store them in a database. In our proof of concept humans were used to conduct the survey but further automation is planned whereby the phone session itself is carried out by a robot. We use OpenAI’s Whisper for the speech-to-text process and we then pass the text to a Large Language Model (GPT-4) which is prompted to extract the responses. The responses are then uploaded to a database. Finally we use an LLM to provide answers to questions about the survey responses. For multiple choice questions we obtained an accuracy score of 97%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Achiam, J., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)
Baevski, A., Zhou, Y., Mohamed, A., Auli, M.: wav2vec 2.0: a framework for self-supervised learning of speech representations. In: Advances in Neural Information Processing Systems, vol. 33, 12449–12460 (2020)
Chang, Y., et al.: A survey on evaluation of large language models. arXiv preprint arXiv:2307.03109 (2023)
Cheng, C., Lau, Y., Chan, L., Luk, J.W.: Prevalence of social media addiction across 32 nations: meta-analysis with subgroup analysis of classification schemes and cultural values. Addict. Behav. 117, 106845 (2021)
De Koning, R., et al.: Survey fatigue during the Covid-19 pandemic: an analysis of neurosurgery survey response rates. Front. Surg. 8, 690680 (2021)
Demombynes, G., Gubbins, P., Romeo, A.: Challenges and opportunities of mobile phone-based data collection: evidence from South Sudan. World Bank Policy Research Working Paper (2013)
Fei, J., et al.: Automated chat application surveys using WhatsApp: evidence from panel surveys and a mode experiment. Econstor (2022)
Feng, Y., Vanam, S., Cherukupally, M., Zheng, W., Qiu, M., Chen, H.: Investigating code generation performance of chat-GPT with crowdsourcing social data. In: Proceedings of the 47th IEEE Computer Software and Applications Conference, pp. 1–10 (2023)
Floridi, L., Chiriatti, M.: GPT-3: its nature, scope, limits, and consequences. Mind. Mach. 30, 681–694 (2020)
Gourlay, S., Kilic, T., Martuscelli, A., Wollburg, P., Zezza, A.: High-frequency phone surveys on Covid-19: good practices, open questions. Food Policy 105, 102153 (2021)
Jansen, B.J., Jung, S., Salminen, J.: Employing large language models in survey research. Nat. Lang. Process. J. 4, 100020 (2023)
Kasirye, F.: Errors in survey research and their threat to validity and reliability. International Islamic University, Selangor, Malaysia (2021)
Kasneci, E., et al.: Chatgpt for good? On opportunities and challenges of large language models for education. Learn. Individ. Differ. 103, 102274 (2023)
Kastelic, K.H., Eckman, S., Kastelic, J.G., Mcgee, K.R., et al.: High frequency mobile phone surveys of households to assess the impacts of Covid-19 (vol. 2): guidelines on sampling design. Policy Commons (2020)
Kelfve, S., Kivi, M., Johansson, B., Lindwall, M.: Going web or staying paper? The use of web-surveys among older people. BMC Med. Res. Methodol. 20, 1–12 (2020)
Kocoń, J., et al.: ChatGPT: jack of all trades, master of none. Inf. Fusion 101861 (2023)
Olson, K., et al.: Transitions from telephone surveys to self-administered and mixed-mode surveys: AAPOR task force report. J. Surv. Stat. Methodol. 9(3), 381–411 (2021)
Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision. In: International Conference on Machine Learning, pp. 28492–28518. PMLR (2023)
Schaeffer, N.C., Dykema, J.: Advances in the science of asking questions. Ann. Rev. Sociol. 46, 37–60 (2020)
Sinkowitz-Cochran, R.L.: Survey design: to ask or not to ask? That is the question. Clin. Infect. Dis. 56(8), 1159–1164 (2013)
Wang, J., Dong, Y.: Measurement of text similarity: a survey. Information 11(9), 421 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tewari, T., Hosein, P. (2024). Automating the Conducting of Surveys Using Large Language Models. In: Fred, A., Hadjali, A., Gusikhin, O., Sansone, C. (eds) Deep Learning Theory and Applications. DeLTA 2024. Communications in Computer and Information Science, vol 2172. Springer, Cham. https://doi.org/10.1007/978-3-031-66705-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-66705-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-66704-6
Online ISBN: 978-3-031-66705-3
eBook Packages: Computer ScienceComputer Science (R0)