Abstract
We present an interface for automatic transcription and simultaneous visualization of spoken narratives on a timeline. We discuss the results of initial laboratory testing of the interface and interviews with prospective users of the approach in two different domains. Speech transcription and entity recognition errors inherent in machine-learning-based approaches to natural language processing place special requirements on the system-user interaction. We outline the design principles for this kind of interface based on the results of testing and interviews with potential users. The presented approach differs from other state-of-the-art text-to-visualization systems in that it constructs a visualization from a speech narrative, while automatically identifying data of interest, as opposed to building plot visualization for user-specified data in structured form based on user’s spoken instructions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ilievski, A., Dojchinovski, D., Gusev, M.: Interactive voice assisted home healthcare systems. In: Proceedings of the 9th Balkan Conference on Informatics, Sofia Bulgaria, pp. 1–5. ACM (2019). https://doi.org/10.1145/3351556.3351572
Bhatt, V., Li, J., Maharjan, B.: DocPal: a voice-based EHR assistant for health practitioners. In: 2020 IEEE International Conference on E-health Networking, Application & Services (HEALTHCOM), pp. 1–6 (2021). https://doi.org/10.1109/HEALTHCOM49281.2021.9399013
Maddigan, P., Susnjak, T.: Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Models (2023). http://arxiv.org/abs/2302.02094
Luo, Y., Tang, N., Li, G., Tang, J., Chai, C., Qin, X.: Natural language to visualization by neural machine translation. IEEE Trans. Visual Comput. Graphics 28, 217–226 (2022). https://doi.org/10.1109/TVCG.2021.3114848
Tang, J., Luo, Y., Ouzzani, M., Li, G., Chen, H.: Sevi: speech-to-visualization through neural machine translation. In: Proceedings of the 2022 International Conference on Management of Data, pp. 2353–2356. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3514221.3520150
Gil, Y., Selman, B.: A 20-Year Community Roadmap for Artificial Intelligence Research in the US (2019). http://arxiv.org/abs/1908.02624
Babaian, T., Lucas, W., Xu, J., Topi, H.: Usability through system-user collaboration. In: Winter, R., Zhao, J.L., Aier, S. (eds.) DESRIST 2010. LNCS, vol. 6105, pp. 394–409. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13335-0_27
Topi, H., Lucas, W., Babaian, T.: Identifying usability issues with an ERP implementation. In: Proceedings of the International Conference on Enterprise Information Systems (ICEIS-2005), pp. 128–133 (2005)
Grosz, B.J.: Beyond mice and menus. Proc. Am. Philos. Soc. 149, 529–523 (2005)
Lucas, W., Babaian, T.: The collaborative critique: an inspection method for expert evaluation of user interfaces. Int. J. Hum.-Comput. Interact. 31, 843–859 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Babaian, T. (2023). Exploring Design Principles for Speech-to-Visualization Data Entry Interfaces. In: Stephanidis, C., Antona, M., Ntoa, S., Salvendy, G. (eds) HCI International 2023 Posters. HCII 2023. Communications in Computer and Information Science, vol 1834. Springer, Cham. https://doi.org/10.1007/978-3-031-35998-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-35998-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35997-2
Online ISBN: 978-3-031-35998-9
eBook Packages: Computer ScienceComputer Science (R0)