skip to main content
10.1145/3640792.3675713acmconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

‘Talking with your Car’: Design of Human-Centered Conversational AI in Autonomous Vehicles

Published: 22 September 2024 Publication History

Abstract

The Development of Fully Autonomous Vehicles (AVs) would fundamentally change the nature of in-vehicle user interactions, behaviors, needs, and activities. Passengers free from driving would expect to undertake diverse Non-Driving-Related Tasks to keep themselves occupied. Introducing Conversational Artificial Intelligence (CAI) in Level 5 AVs could improve the in-vehicle user experience (UX). To explore this, firstly, we identify what roles and relationships can CAI play towards end-users of AVs through end-user interviews and thematic analysis. Secondly, we examine how end-users qualitatively assess the embodied UX of the CAI roles and relationships through guided brainstorming, post simulator interaction experiments employing Wizard of Oz setup and Participant Enactment methods. Results show that Tour Guide, Mentor, and Storyteller were the most preferred CAI roles, and that Human-CAI relationships are maintained if the CAI mediates in-vehicle user activities, interactions, sharing of vehicle control, and deep conversations. We discuss the research implications and propose design guidelines.

References

[1]
Francesco Biondi, Ignacio Alvarez, and Kyeong-Ah Jeong. 2019. Human–vehicle cooperation in automated driving: A multidisciplinary review and appraisal. International Journal of Human–Computer Interaction 35, 11 (2019), 932–946.
[2]
Hayet Brabra, Marcos Báez, Boualem Benatallah, Walid Gaaloul, Sara Bouguelia, and Shayan Zamanirad. 2021. Dialogue management in conversational systems: a review of approaches, challenges, and opportunities. IEEE Transactions on Cognitive and Developmental Systems 14, 3 (2021), 783–798.
[3]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[4]
Dishman-E. Verplank W. Burns, C. and B. Lassiter. 1994. Actors, hairdos & videotape—informance design. n Conference companion on Human factors in computing systems (1994), 119–120.
[5]
Fabio Catania, Pietro Crovari, Micol Spitale, and Franca Garzotto. 2019. Automatic Speech Recognition: Do Emotions Matter?. In 2019 IEEE International Conference on Conversational Data & Knowledge Engineering (CDKE). IEEE, 9–16.
[6]
Ana Paula Chaves and Marco Aurelio Gerosa. 2021. How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction 37, 8 (2021), 729–758.
[7]
Zhitong Cui, Hebo Gong, Yanan Wang, Chengyi Shen, Wenyin Zou, and Shijian Luo. 2021. Enhancing interactions for in-car voice user interface with gestural input on the steering wheel. In 13th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 59–68.
[8]
Rebecca Currano, So Yeon Park, Dylan James Moore, Kent Lyons, and David Sirkin. 2021. Little road driving HUD: Heads-up display complexity influences drivers’ perceptions of automated vehicles. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama Japan). ACM, New York, NY, USA.
[9]
N Dahlbäck, A Jönsson, and L Ahrenberg. 1993. Wizard of Oz studies: why and how. In Proceedings of the 1st international conference on Intelligent user interfaces. 193–200.
[10]
Pieter Desmet and Steven Fokkinga. 2020. Beyond Maslow’s pyramid: Introducing a typology of thirteen fundamental needs for human-centered design. Multimodal Technol. Interact. 4, 3 (July 2020), 38.
[11]
Debargha Dey and Jacques Terken. 2017. Pedestrian interaction with vehicles: roles of explicit and implicit communication. In Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applications. 109–113.
[12]
Paul Dourish. 2001. Where the action is: the foundations of embodied interaction. MIT press.
[13]
Emanuel Felipe Duarte, Yusseli Lizeth Méndez Mendoza, Maria Jêsca Nobre de Queiroz, and M Cecília C Baranauskas. 2022. Embodiment in interactive installations: results from a systematic literature review. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems. 1–13.
[14]
Mustafa Ergen 2019. What is artificial intelligence? Technical considerations and future perception. Anatolian J. Cardiol 22, 2 (2019), 5–7.
[15]
Lex Fridman. 2018. Human-Centered Autonomous Vehicle Systems: Principles of Effective Shared Autonomy. arxiv:1810.01835 [cs.AI]
[16]
Dennis A Gioia, Kevin G Corley, and Aimee L Hamilton. 2013. Seeking qualitative rigor in inductive research: Notes on the Gioia methodology. Organizational research methods 16, 1 (2013), 15–31.
[17]
G Gomez-Beldarrain, W Van Der Maden, S Huang, and E Kim. 2023. Identifying meaningful user experiences with autonomous products: a case study in fundamental user needs in fully autonomous vehicles. In IASDR2023: Milan. Milan, Italy.
[18]
Evelien Heyselaar and Tibor Bosse. 2019. Using theory of mind to assess users’ sense of agency in social chatbots. In International workshop on chatbot research and design. Springer, 158–169.
[19]
Jennifer Hill, W Randolph Ford, and Ingrid G Farreras. 2015. Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations. Computers in human behavior 49 (2015), 245–250.
[20]
Kate S Hone and Robert Graham. 2000. Towards a tool for the subjective assessment of speech system interfaces (SASSI). Natural Language Engineering 6, 3-4 (2000), 287–303.
[21]
Myounghoon Jeon, Andreas Riener, Jason Sterkenburg, Ju-Hwan Lee, Bruce N Walker, and Ignacio Alvarez. 2018. An international survey on automated and electric vehicles: Austria, Germany, South Korea, and USA. In Digital Human Modeling. Applications in Health, Safety, Ergonomics, and Risk Management: 9th International Conference, DHM 2018, Held as Part of HCI International 2018, Las Vegas, NV, USA, July 15-20, 2018, Proceedings 9. Springer, 579–587.
[22]
Iris Jestin, Joel Fischer, Maria Jose Galvez Trigo, David Large, and Gary Burnett. 2022. Effects of wording and gendered voices on acceptability of voice assistants in future autonomous vehicles. In Proceedings of the 4th Conference on Conversational User Interfaces. 1–11.
[23]
Sofia Jorlöv, Katarina Bohman, and Annika Larsson. 2017. Seating positions and activities in highly automated cars–a qualitative study of future automated driving scenarios. In International research conference on the biomechanics of impact. IRCOBI, 13–22.
[24]
Euiyoung Kim, Sara Beckman, Ki-Hun Kim, and Sicco Santema. 2022. Designing for dynamic stability in an uncertain world: A media content study of the aviation industry. (2022).
[25]
Hyang Sook Kim, Sol Hee Yoon, Meen Jong Kim, and Yong Gu Ji. 2015. Deriving future user experiences in autonomous vehicle. In Adjunct proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications. 112–117.
[26]
Pradnya Kulkarni, Ameya Mahabaleshwarkar, Mrunalini Kulkarni, Nachiket Sirsikar, and Kunal Gadgil. 2019. Conversational AI: An overview of methodologies, applications & future scope. In 2019 5th International conference on computing, communication, control and automation (ICCUBEA). IEEE, 1–7.
[27]
Knut Kvale, Olav Alexander Sell, Stig Hodnebrog, and Asbjørn Følstad. 2019. Improving conversations: lessons learnt from manual analysis of chatbot dialogues. In International workshop on chatbot research and design. Springer, 187–200.
[28]
David R Large, Gary Burnett, Davide Salanitri, Anneka Lawson, and Elizabeth Box. 2019. A Longitudinal simulator study to explore drivers’ behaviour in level 3 automated vehicles. In Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 222–232.
[29]
David R Large and Gary E Burnett. 2013. Drivers’ preferences and emotional responses to satellite navigation voices. International journal of vehicle noise and vibration 9, 1-2 (2013), 28–46.
[30]
David R Large, Leigh Clark, Annie Quandt, Gary Burnett, and Lee Skrypchuk. 2017. Steering the conversation: A linguistic exploration of natural language interactions with a digital assistant during simulated driving. Applied ergonomics 63 (2017), 53–61.
[31]
Pontus Larsson, Justyna Maculewicz, Johan Fagerlönn, and Max Lachmann. 2019. Auditory displays for automated driving—challenges and opportunities. In The 25th International Conference on auditory display (ICAD 2019), Vol. 52. 299–305.
[32]
Cheong-Jae Lee, Sang-Keun Jung, Kyung-Duk Kim, Dong-Hyeon Lee, and Gary Geun-Bae Lee. 2010. Recent approaches to dialog management for spoken dialog systems. Journal of Computing Science and Engineering 4, 1 (2010), 1–22.
[33]
Seul Chan Lee, Chihab Nadri, Harsh Sanghavi, and Myounghoon Jeon. 2020. Exploring user needs and design requirements in fully automated vehicles. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–9.
[34]
Christine Liebrecht and Charlotte van Hooijdonk. 2020. Creating humanlike chatbots: What chatbot developers could learn from webcare employees in adopting a conversational human voice. In Chatbot Research and Design: Third International Workshop, CONVERSATIONS 2019, Amsterdam, The Netherlands, November 19–20, 2019, Revised Selected Papers 3. Springer, 51–64.
[35]
Peter Lloyd, Senthil Chandrasegaran, Euiyoung Kim, Jonathan Cagan, Maria Yang, and Kosa Goucher-Lambert. 2022. Designing dialogue: Human-AI collaboration in design processes. (2022).
[36]
Seng W Loke. 2019. Cooperative automated vehicles: A review of opportunities and challenges in socially intelligent vehicles beyond networking. IEEE Transactions on Intelligent Vehicles 4, 4 (2019), 509–518.
[37]
Giuseppe Lugano. 2017. Virtual assistants and self-driving cars. In 2017 15th International Conference on ITS Telecommunications (ITST). IEEE, 1–5.
[38]
B. Martin and B. Hanington. 2012. Universal Methods of Design: 100 Ways to Research Complex Problems, Develop Innovative Ideas, and Design Effective Solutions. Rockport Publishers. https://books.google.nl/books?id=uZ8uzWAcdxEC
[39]
John McCarthy. 2004. What is Artificial Intelligence? (01 2004).
[40]
Roger K Moore. 2019. Talking with robots: Opportunities and challenges. arXiv preprint arXiv:1912.00369 (2019).
[41]
Clare Mutzenich, Szonya Durant, Shaun Helman, and Polly Dalton. 2021. Updating our understanding of situation awareness in relation to remote operators of autonomous vehicles. Cognitive research: principles and implications 6 (2021), 1–17.
[42]
Clifford Nass, Jonathan Steuer, and Ellen R Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems. 72–78.
[43]
Yu-Leung Ng and Zhihuai Lin. 2022. Exploring conversation topics in conversational artificial intelligence–based social mediated communities of practice. Computers in Human Behavior 134 (2022), 107326.
[44]
Dennis Orth, Nadja Schömig, Christian Mark, Monika Jagiellowicz-Kaufmann, Dorothea Kolossa, and Martin Heckmann. 2017. Benefits of personalization in the context of a speech-based left-turn assistant. In Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applications. 193–201.
[45]
Se Hyeon Park and Seul Chan Lee. 2022. Which Voice Do You want To Hear From Your Automated Vehicle? User Preference on In-Vehicle Intelligent Agent Voice in Automated Vehicles. In Adjunct Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 91–93.
[46]
Michael Quinn Patton. 2002. Two decades of developments in qualitative inquiry: A personal, experiential perspective. Qualitative social work 1, 3 (2002), 261–283.
[47]
Nicole Perterer, Susanne Meerwald-Stadler, Sandra Trösterer, Alexander Meschtscherjakov, and Manfred Tscheligi. 2018. Follow Me: Exploring Strategies and Challenges for Collaborative Driving. In Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 176–187.
[48]
Jun Quan, Meng Yang, Qiang Gan, Deyi Xiong, Yiming Liu, Yuchen Dong, Fangxin Ouyang, Jun Tian, Ruiling Deng, Yongzhi Li, 2021. Integrating pre-trained model into rule-based dialogue management. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 16097–16099.
[49]
Florian Roider, Sonja Rümelin, Bastian Pfleging, and Tom Gross. 2017. The effects of situational demands on gaze, speech and gesture input in the vehicle. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 94–102.
[50]
Dirk Rothenbücher, Jamy Li, David Sirkin, Brian Mok, and Wendy Ju. 2016. Ghost driver: A field study investigating the interaction between pedestrians and driverless vehicles. In 2016 25th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, 795–802.
[51]
Abdullahi B. Saka, Lukumon O. Oyedele, Lukman A. Akanbi, Sikiru A. Ganiyu, Daniel W.M. Chan, and Sururah A. Bello. 2023. Conversational artificial intelligence in the AEC industry: A review of present status, challenges and opportunities. Advanced Engineering Informatics 55 (2023), 101869. https://doi.org/10.1016/j.aei.2022.101869
[52]
Elizabeth B-N Sanders and Pieter Jan Stappers. 2012. Convivial toolbox: Generative research for the front end of design. Bis.
[53]
Anuschka Schmitt, Naim Zierau, Andreas Janson, and Jan Marco Leimeister. 2021. Voice as a contemporary frontier of interaction design. In European Conference on Information Systems (ECIS).-Virtual.
[54]
Katie Seaborn, Norihisa P Miyake, Peter Pennefather, and Mihoko Otake-Matsuura. 2021. Voice in human–agent interaction: A survey. ACM Computing Surveys (CSUR) 54, 4 (2021), 1–43.
[55]
Helena Strömberg, Ingrid Pettersson, and Wendy Ju. 2020. Enacting metaphors to explore relations and interactions with automated driving systems. Design Studies 67 (2020), 77–101.
[56]
Nina Svenningsson and Montathar Faraon. 2019. Artificial intelligence in conversational agents: A study of factors related to perceived humanness in chatbots. In Proceedings of the 2019 2nd Artificial Intelligence and Cloud Computing Conference. 151–161.
[57]
Pinyan Tang, Xu Sun, and Shi Cao. 2020. Investigating user activities and the corresponding requirements for information and functions in autonomous vehicles of the future. International Journal of Industrial Ergonomics 80 (2020), 103044. https://doi.org/10.1016/j.ergon.2020.103044
[58]
Pinyan Tang, Xu Sun, and Shi Cao. 2020. Investigating user activities and the corresponding requirements for information and functions in autonomous vehicles of the future. International Journal of Industrial Ergonomics 80 (2020), 103044.
[59]
David Callisto Valentine, Iskander Smit, and Euiyoung Kim. 2021. DESIGNING FOR CALIBRATED TRUST: EXPLORING THE CHALLENGES IN CALIBRATING TRUST BETWEEN USERS AND AUTONOMOUS VEHICLES. Proceedings of the Design Society 1 (2021), 1143–1152. https://doi.org/10.1017/pds.2021.114
[60]
Varad Vishwarupe, Shrey Maheshwari, Aseem Deshmukh, Shweta Mhaisalkar, Prachi M Joshi, and Nicole Mathias. 2022. Bringing humans at the epicenter of artificial intelligence: A confluence of AI, HCI and human centered computing. Procedia Computer Science 204 (2022), 914–921.
[61]
Froukje Sleeswijk Visser, Pieter Jan Stappers, Remko Van der Lugt, and Elizabeth BN Sanders. 2005. Contextmapping: experiences from practice. CoDesign 1, 2 (2005), 119–149.
[62]
Manhua Wang, Seul Chan Lee, Genevieve Montavon, Jiakang Qin, and Myounghoon Jeon. 2022. Conversational voice agents are preferred and Lead to better driving performance in conditionally automated vehicles. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 86–95.
[63]
Yang Xing, Chao Huang, and Chen Lv. 2020. Driver-Automation Collaboration for Automated Vehicles: A Review of Human-Centered Shared Control. In 2020 IEEE Intelligent Vehicles Symposium (IV). 1964–1971. https://doi.org/10.1109/IV47402.2020.9304755
[64]
Yang Xing, Chen Lv, Dongpu Cao, and Peng Hang. 2021. Toward human-vehicle collaboration: Review and perspectives on human-centered collaborative automated driving. Transportation Research Part C: Emerging Technologies 128 (2021), 103199. https://doi.org/10.1016/j.trc.2021.103199

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AutomotiveUI '24: Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
September 2024
438 pages
ISBN:9798400705106
DOI:10.1145/3640792
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 September 2024

Permissions

Request permissions for this article.

Check for updates

Badges

  • Honorable Mention

Author Tags

  1. Conversational AI Roles and Relationships
  2. Design Guidelines
  3. Human-Artificial Intelligence interaction
  4. Human-Autonomous Vehicle interaction
  5. User Experience Evaluation

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AutomotiveUI '24
Sponsor:

Acceptance Rates

Overall Acceptance Rate 248 of 566 submissions, 44%

Upcoming Conference

AutomotiveUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 349
    Total Downloads
  • Downloads (Last 12 months)349
  • Downloads (Last 6 weeks)62
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media