Abstract
For a successful market introduction of Level 3 Automated Driving Systems (L3 ADS), a careful evaluation of Human-Machine Interfaces (HMIs) is necessary. This paper describes an empirical evaluation of a checklist that has been previously developed for the use in heuristic expert assessments, demonstrating that an ADS HMI that meets the guidelines as defined in the checklist scores higher in several measures of usability and acceptance than an HMI that does not meet the checklist requirements. Therefore, N = 57 participants completed two 15-min drives with an L3 ADS in a driving simulator. They experienced two variations of a L3 ADS HMI that differed in the degree of complying with the checklist. Inferential statistics showed that acceptance and usability measures differed substantially between the two experimental HMIs. Behavioral observations of interaction performance also demonstrate that non-compliance with the checklist items lowers the usability of the L3 system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
SAE: Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems (J3016R) (2018)
Nunes, A., Reimer, B., Coughlin, J.F.: People must retain control of autonomous vehicles. Nature 556(7700), 169–171 (2018). https://doi.org/10.1038/d41586-018-04158-5
RESPONSE Consortium: Code for practice for the design and evaluation of ADAS: RESPONSE 3: A PReVENT Project (2006)
Jarosch, O., Kuhnt, M., Paradies, S., et al.: It’s out of our hands now! Effects of non-driving related tasks during highly automated driving on drivers’ fatigue. In: Driving Assessment Conference, pp. 319–325 (2017)
Forster, Y., Naujoks, F., Neukum, A.: Increasing anthropomorphism and trust in automated driving functions by adding speech output. In: Intelligent Vehicles Symposium (IV) IEEE. Redondo Beach, California, USA (2017)
Hergeth, S., Lorenz, L., Krems, J.F.: Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Hum. Factors 59(3), 457–470 (2017)
Feldhütter, A., Segler, C., Bengler, K.: Does shifting between conditionally and patially automated driving lead to a loss of mode awareness? In: International Conference on Applied Human Factors and Ergonomics, Los Angeles, CA, USA (2017)
Naujoks, F., Mai, C., Neukum, A.: The effect of urgency take-over requests during highly automated driving under distraction conditions. Adv. Hum. Aspects Transp. 7(Part I), 431 (2014)
Forster, Y., Hergeth, S., Naujoks, F., et al.: Learning to use automation: behavioral changes in interaction with automated driving systems (2019). (Submitted and under review)
Forster, Y., Hergeth, S., Naujoks, F., et al.: The development of learning and mental models in interaction with driving automation: a simulator study (2019). (Submitted and under review)
Naujoks, F., Wiedemann, K., Schömig, N., et al.: Towards guidelines and verification methods for automated vehicle HMIs. Transp. Res. Part F Traffic Psychol. Behav. 60, 121–136 (2019)
Naujoks, F., Hergeth, S., Wiedemann, K., et al.: Use cases for assessing, testing, and validating the human machine interface of automated driving systems. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 62nd edn., pp. 1873–1877. Sage publications, Los Angeles (2018)
Naujoks, F., Purucker, C., Neukum, A., et al.: Controllability of partially automated driving functions–does it matter whether drivers are allowed to take their hands off the steering wheel? Transp. Res. Part F Traffic Psychol. Behav. 35, 185–198 (2015)
Stevens, A., Quimby, A., Board, A., et al.: Design Guidelines for safety in-vehicle information systems. TRL Limited (2002)
Forster, Y., Naujoks, F., Neukum, A.: Your turn or my turn? Design of a human-machine interface for conditional automation. In: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 253–260 (2016)
AdaptIVe Consortium: Final human factors recommendations (D3.3) (2017)
Tullis, T.S., Boynton, T.L., Hersh, H.: Readability of fonts in the windows environment. In: Conference Companion on Human Factors in Computing Systems (1995)
Crash Avoidance Metric Partnership: Automated vehicles research for enhanced safety (2016)
ISO: Road vehicles—ergonomic aspects of transport information and control systems—calibration tasks for methods which assess driver demand due to the use of in-vehicle systems (14198) (2012)
Naujoks, F., Forster, Y., Wiedemann, K., et al.: Improving usefulness of automated driving by lowering primary task interference through HMI design. J. Adv. Transp. (2017)
Forster, Y., Hergeth, S., Naujoks, F., et al.: How usability can save the day: methodological considerations for making automated driving a success story. In: Donmez, B., Walker, B.N., Fröhlich, K. (eds.) Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (2018)
Brooke, J.: SUS - a quick and dirty usability scale. Usability evaluation in industry 194(189) (1996)
van der Laan, J.D., Heino, A., de Waard, D.: A simple procedure for the assessment of acceptance of advanced transport telematics. Transp. Res. Part C Emerg. Technol. 5(1), 1–10 (1997)
Lewis, J.R., Sauro, J.: The factor structure of the system usability scale. In: International Conference on Human Centered Design (2009)
Forster, Y., Naujoks, F., Neukum, A.: Your turn or my turn? Design of a human-machine interface for conditional automation. In: Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. ACM (2016)
Bazilinskyy, P., Eriksson, A., Petermeijer, B., et al.: Usefulness and satisfaction of take-over requests for highly automated driving. In: Road Safety & Simulation International Conference (RSS 2017), The Hague, Netherlands (2017)
Neukum, A., Lübbecke, T., Krüger, H.-P., et al.: ACC-Stop&Go: Fahrerverhalten an funktionalen Systemgrenzen. In: 5. Workshop Fahrerassistenzsysteme-FAS (2008)
Lüdtke, O., Robitzsch, A., Trautwein, U., et al.: Umgang mit fehlenden Werten in der psychologischen Forschung. Psychologische Rundschau 58(2), 103–117 (2007). https://doi.org/10.1026/0033-3042.58.2.103
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Forster, Y., Hergeth, S., Naujoks, F., Krems, J.F., Keinath, A. (2020). Empirical Validation of a Checklist for Heuristic Evaluation of Automated Vehicle HMIs. In: Stanton, N. (eds) Advances in Human Factors of Transportation. AHFE 2019. Advances in Intelligent Systems and Computing, vol 964. Springer, Cham. https://doi.org/10.1007/978-3-030-20503-4_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-20503-4_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20502-7
Online ISBN: 978-3-030-20503-4
eBook Packages: EngineeringEngineering (R0)