skip to main content
10.1145/3581961.3609874acmconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
Work in Progress

Providing Explainability in Safety-Critical Automated Driving Situations through Augmented Reality Windshield HMIs

Published: 18 September 2023 Publication History

Abstract

Research on automotive user interfaces has highlighted the importance of transparent and explainable user interfaces, especially in safety-critical situations where automation failures can lead to (fatal) accidents. In this work, we investigated the use of augmented reality (AR) windshield displays as a means to improve explainability in safety-critical driving situations. To this end, we conducted a mixed-methods study (VR-based driving simulation combined with a semi-structured interview) with N = 24 participants and evaluated two concepts, containing (1) first-person information about vehicle behavior and (2) additional information about other traffic objects. Results showed that confidence and situational awareness were significantly higher when additional information was presented. The highest scores were achieved when participants were shown "how & why" information.

References

[1]
Oliver Carsten and Marieke Martens. 2019. How can humans understand their automated cars? HMI principles, problems and solutions. Cognition, Technology & Work 21 (02 2019). https://doi.org/10.1007/s10111-018-0484-0
[2]
Mark Colley, Benjamin Eder, Jan Ole Rixen, and Enrico Rukzio. 2021. Effects of Semantic Segmentation Visualization on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 155, 11 pages. https://doi.org/10.1145/3411764.3445351
[3]
Mark Colley, Svenja Krauss, Mirjam Lanzer, and Enrico Rukzio. 2021. How Should Automated Vehicles Communicate Critical Situations? A Comparative Analysis of Visualization Concepts. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 3, Article 94 (sep 2021), 23 pages. https://doi.org/10.1145/3478111
[4]
Mark Colley, Max Rädler, Jonas Glimmann, and Enrico Rukzio. 2022. Effects of Scene Detection, Scene Prediction, and Maneuver Planning Visualizations on Trust, Situation Awareness, and Cognitive Load in Highly Automated Vehicles. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 2, Article 49 (jul 2022), 21 pages. https://doi.org/10.1145/3534609
[5]
Simon Danner, Alexander Feierle, Carina Manger, and Klaus Bengler. 2021. Context-Adaptive Availability Notifications for an SAE Level 3 Automation. Multimodal Technologies and Interaction 5 (03 2021), 16. https://doi.org/10.3390/mti5040016
[6]
Na Du, Jacob Haspiel, Qiaoning Zhang, Dawn Tilbury, Anuj Pradhan, X. Jessie Yang, and Lionel Robert. 2019. Look Who’s Talking Now: Implications of AV’s Explanations on Driver’s Trust, AV Preference, Anxiety and Mental Workload. Transportation Research Part C Emerging Technologies 104 (05 2019), 428–442. https://doi.org/10.1016/j.trc.2019.05.025
[7]
David W. Eby, Lisa J. Molnar, Liang Zhang, Renée M. St. Louis, Nicole Zanier, Lidia P. Kostyniuk, and Sergiu C Stanciu. 2016. Use, perceptions, and benefits of automotive technologies among aging drivers.
[8]
Mica Endsley. 1995. Endsley, M.R.: Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors Journal 37(1), 32-64. Human Factors: The Journal of the Human Factors and Ergonomics Society 37 (03 1995), 32–64. https://doi.org/10.1518/001872095779049543
[9]
Brittany E. Holthausen, Philipp Wintersberger, Bruce N. Walker, and Andreas Riener. 2020. Situational Trust Scale for Automated Driving (STS-AD): Development and Initial Validation. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 40–47. https://doi.org/10.1145/3409120.3410637
[10]
Jeamin Koo, Jungsuk Kwac, Martin Steinert, Larry Leifer, and Clifford Nass. 2014. Why did my car just do that? Explaining semi-autonomous driving actions to improve driver understanding, trust, and performance. International Journal on Interactive Design and Manufacturing (IJIDeM) 9 (01 2014). https://doi.org/10.1007/s12008-014-0227-2
[11]
Alexander Kunze, Steve J. Summerskill, Russell Marshall, and Ashleigh J. Filtness. 2019. Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces. Ergonomics 62 (2019), 345 – 360.
[12]
S. Langlois and B. Soualmi. 2016. Augmented Reality versus Classical HUD to Take over from Automated Driving: An Aid to Smooth Reactions and to Anticipate Maneuvers. In 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC) (Rio de Janeiro, Brazil). IEEE Press, New York City, NY, USA, 8 pages. https://doi.org/10.1109/ITSC.2016.7795767
[13]
John D. Lee and Katrina A. See. 2004. Trust in Automation: Designing for Appropriate Reliance. Human Factors 46, 1 (2004), 50–80. https://doi.org/10.1518/hfes.46.1.50 _30392 15151155.
[14]
Todd Litman. 2020. Autonomous Vehicle Implementation Predictions: Implications for Transport Planning.Technical Report. Victoria Transport Policy Institute. http://www.vtpi.org/avip.pdf
[15]
Lia Morra, Fabrizio Lamberti, F. Gabriele Pratticó, Salvatore La Rosa, and Paolo Montuschi. 2019. Building Trust in Autonomous Vehicles: Role of Virtual Reality Driving Simulators in HMI Design. IEEE Transactions on Vehicular Technology 68, 10 (2019), 9438–9450. https://doi.org/10.1109/TVT.2019.2933601
[16]
D Omeiza, H Webb, M Jirotka, and L Kunze. 2021. Explanations in autonomous driving: a survey. IEEE Transactions on Intelligent Transportation Systems 23, 8 (2021), 10142 – 10162.
[17]
On-Road Automated Driving (ORAD) committee. 2018. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles.
[18]
Jakob Benedikt Peintner, Carina Manger, and Andreas Riener. 2022. “Can You Rely on Me?” Evaluating a Confidence HMI for Cooperative, Automated Driving. In Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY, USA, 340–348. https://doi.org/10.1145/3543174.3546976
[19]
Luke Petersen, Lionel Robert, X. Jessie Yang, and Dawn Tilbury. 2019. Situational Awareness, Driver’s Trust in Automated Driving Systems and Secondary Task Performance. https://doi.org/10.2139/ssrn.3345543
[20]
Bastian Pfleging, Maurice Rang, and Nora Broy. 2016. Investigating User Needs for Non-Driving-Related Activities during Automated Driving. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (Rovaniemi, Finland) (MUM ’16). Association for Computing Machinery, New York, NY, USA, 91–99. https://doi.org/10.1145/3012709.3012735
[21]
L.P. Robert. 2019. Are automated vehicles safer than manually driven cars?, 687–688 pages. https://doi.org/10.1007/s00146-019-00894-y
[22]
Tobias Schneider, Joana Hois, Alischa Rosenstein, Sabiha Ghellal, Dimitra Theofanou-Fülbier, and Ansgar R.S. Gerlicher. 2021. ExplAIn Yourself! Transparency for Positive UX in Autonomous Driving. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 161, 12 pages. https://doi.org/10.1145/3411764.3446647
[23]
Tim Schrills, Mourad Zoubir, Mona Bickel, Susanne Kargl, and Thomas Franke. 2021. Are Users in the Loop? Development of the Subjective Information Processing Awareness Scale to Assess XAI.
[24]
Brad Templeton. 2021. Fatal Tesla Crash In Texas: What We Know And What We Don’t. YouTube. https://www.forbes.com/sites/bradtempleton/2021/04/22/fatal-tesla-crash-in-texas-what-we-know-and-what-we-dont/?sh=69c6b6a634bbRetrieved: 2022-05-09.
[25]
Gesa Wiegand, Malin Eiband, Maximilian Haubelt, and Heinrich Hussmann. 2020. “I’d like an Explanation for That!”Exploring Reactions to Unexpected Autonomous Driving. In 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services. Association for Computing Machinery, New York, NY, USA, Article 36, 11 pages. https://doi.org/10.1145/3379503.3403554
[26]
Gesa Wiegand, Matthias Schmidmaier, Thomas Weber, Yuanting Liu, and Heinrich Hussmann. 2019. I Drive - You Trust: Explaining Driving Behavior Of Autonomous Cars. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312817
[27]
Philipp Wintersberger, Tamara von Sawitzky, Anna-Katharina Frison, and Andreas Riener. 2017. Traffic Augmentation as a Means to Increase Trust in Automated Driving Systems. In Proceedings of the 12th Biannual Conference on Italian SIGCHI Chapter (Cagliari, Italy) (CHItaly ’17). Association for Computing Machinery, New York, NY, USA, Article 17, 7 pages. https://doi.org/10.1145/3125571.3125600
[28]
Ingo Wolf. 2016. The Interaction Between Humans and Autonomous Agents. Springer Berlin Heidelberg, Berlin, Heidelberg, 103–124. https://doi.org/10.1007/978-3-662-48847-8_6

Cited By

View all
  • (2024)TimelyTale: A Multimodal Dataset Approach to Assessing Passengers' Explanation Demands in Highly Automated VehiclesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785448:3(1-60)Online publication date: 9-Sep-2024
  • (2024)Explaining Away Control: Exploring the Relationship between Explainable AI and Passengers' Desire for Control in Automated VehiclesAdjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3641308.3685040(155-160)Online publication date: 22-Sep-2024
  • (2024)I Don't Know What Happens, But… Do I Really Need that Explanation?Adjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3641308.3680517(310-311)Online publication date: 22-Sep-2024

Index Terms

  1. Providing Explainability in Safety-Critical Automated Driving Situations through Augmented Reality Windshield HMIs

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    AutomotiveUI '23 Adjunct: Adjunct Proceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
    September 2023
    382 pages
    ISBN:9798400701122
    DOI:10.1145/3581961
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 September 2023

    Check for updates

    Author Tags

    1. Augmented Reality
    2. Automated Driving
    3. User Interfaces
    4. User Study
    5. Windshield Displays

    Qualifiers

    • Work in progress
    • Research
    • Refereed limited

    Funding Sources

    • German Federal Ministry of Education and Research (BMBF)

    Conference

    AutomotiveUI '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 248 of 566 submissions, 44%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)136
    • Downloads (Last 6 weeks)10
    Reflects downloads up to 10 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)TimelyTale: A Multimodal Dataset Approach to Assessing Passengers' Explanation Demands in Highly Automated VehiclesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785448:3(1-60)Online publication date: 9-Sep-2024
    • (2024)Explaining Away Control: Exploring the Relationship between Explainable AI and Passengers' Desire for Control in Automated VehiclesAdjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3641308.3685040(155-160)Online publication date: 22-Sep-2024
    • (2024)I Don't Know What Happens, But… Do I Really Need that Explanation?Adjunct Proceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3641308.3680517(310-311)Online publication date: 22-Sep-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media