Skip to main content
SpringerLink
Log in

Standardization of User Experience Evaluation: Theory, Method and Promotion of High-Quality Development of Chinese Intelligent Vehicle Industry

  • Conference paper
  • First Online:
HCI International 2023 – Late Breaking Papers (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14057))

Included in the following conference series:

  • 347 Accesses

Abstract

User experience (UX) evaluation can improve product quality, which has been confirmed in many studies. In recent years, the Chinese government has continued to promote the digitalization, intelligence and standardization of automobiles and the high-quality development of the intelligent automobile industry. However, there is still space for progress in production standardization and quality standardization, and mechanisms have not yet been designed to comprehensively measure the UX of intelligent vehicles. Therefore, the purpose of this paper is to develop a comprehensive framework for the evaluation of UX in intelligent vehicle human-machine interface (HMI) from the perspective of literature research and industry research. The framework was determined after a review of related literatures to analysis and summary existing evaluation approaches and was analyzed and organized according to the following components: UX evaluation object, the type of evaluation performer, HMI UX metrics, the most frequently applied evaluation methods, and UX influence factors. Through this method, it is possible for developers to obtain the theoretical knowledge and practical experience required for the evaluation of UX of HMI of intelligent vehicles, to carry out the evaluation to ensure the consistency of product quality, and then promote the high-quality development of the intelligent vehicle industry.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Key points of automobile standardization in 2022. http://www.springer.com/lncs https://www.miit.gov.cn/jgsj/zbys/qcgy/art/2022/art_b703341da41a49c682a7c878c868dcbb.html Accessed 06 Feb 2023

  2. Opinions on Strengthening Management of Intelligent Connected Vehicle Manufacturers and Product Access. https://www.miit.gov.cn/zwgk/zcwj/wjfb/yj/art/2021/art_bf9a399907d94b8c9ac9f79f2045fe14.html. Accessed 06 Feb 2023

  3. Yang, Z., Zeng, J., Huang, X., Lu, Y.: Research on man-machine interface design based on intelligent vehicle. In: Rau, PL.P. (eds) HCII 2022, LNCS, vol. 13314, pp. 276–285. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06053-3_19

  4. Maurer, S., Erbach, R., Kraiem, I., Kuhnert, S., Grimm, P., Rukzio, E.: Designing a guardian angel: giving an automated vehicle the possibility to override its driver. In: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 341–350. Association for Computing Machinery, New York (2018)

    Google Scholar 

  5. Rezvani, T., Driggs-Campbell, K., Sadigh, D., Sastry, S.S., Seshia, S.A., Bajcsy, R.: Towards trustworthy automation: user interfaces that convey internal and external awareness. In: 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC), pp. 682–688. IEEE, New York (2016)

    Google Scholar 

  6. Tran, C., Bark, K., Ng-Thow-Hing, V.: A left-turn driving aid using projected oncoming vehicle paths with augmented reality. In: Proceedings of the 5th international conference on automotive user interfaces and interactive vehicular applications, pp. 300–307. Association for Computing Machinery, New York (2013)

    Google Scholar 

  7. Politis, I., Brewster, S., & Pollick, F.: Language-based multimodal displays for the handover of control in autonomous cars. In: Proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications, pp. 3–10. Association for Computing Machinery, New York (2015)

    Google Scholar 

  8. Carmona, J., Guindel, C., Garcia, F., de la Escalera, A.: EHMI: review and guidelines for deployment on autonomous vehicles. Sensors 21(9), 1–21 (2021)

    Article  Google Scholar 

  9. Wang, L., Zhong, L.: Human Engineering, 1st edn. Hunan University Publisher, Hunan (2011)

    Google Scholar 

  10. Bengler, K., Rettenmaier, M., Fritz, N., Feierle, A.: From HMI to HMIs: towards an HMI framework for automated driving. Information 11(2), 1–17 (2020)

    Article  Google Scholar 

  11. Tan, Z., et al.: Human–machine interaction in intelligent and connected vehicles: a review of status quo, issues, and opportunities. IEEE Trans. Intell. Transp. Syst. 23(9), 13954–13975 (2021)

    Article  Google Scholar 

  12. Olaverri-Monreal, C., Hasan, A.E., Bulut, J., Körber, M., Bengler, K.: Impact of in-vehicle displays location preferences on drivers’ performance and gaze. IEEE Trans. Intell. Transp. Syst. 15(4), 1770–1780 (2014)

    Article  Google Scholar 

  13. Lau, M., Jipp, M., Oehl, M.: One solution fits all? Evaluating different communication strategies of a light-based external human-machine interface for differently sized automated vehicles from a pedestrian’s perspective. Accid. Anal. Prev. 171, 106641 (2022)

    Article  Google Scholar 

  14. Dey, D., et al.: Taming the eHMI jungle: A classification taxonomy to guide, compare, and assess the design principles of automated vehicles’ external human-machine interfaces. Transp. Res. Interdisc. Perspect. 7, 100174 (2020)

    Google Scholar 

  15. Deb, S., Strawderman, L.J., Carruth, D.W.: Investigating pedestrian suggestions for external features on fully autonomous vehicles: a virtual reality experiment. Transport. Res. F: Traffic Psychol. Behav. 59, 135–149 (2018)

    Article  Google Scholar 

  16. Chang, H., Li, L.: Smart and seamless: investigating user needs and recognition for smartphone-automobile interactive features. In: Krömker, H. (ed.) HCII 2020. LNCS, vol. 12212, pp. 217–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50523-3_15

    Chapter  Google Scholar 

  17. ISO 9241–11:2018(en) Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en. Accessed 06 Feb 2023

  18. Norman, D., Miller, J., Henderson, A.: What you see, some of what’s in the future, and how we go about doing it: HI at Apple Computer. In: Conference companion on Human factors in computing systems, p. 155. ACM Press, New York (1995)

    Google Scholar 

  19. Berni, A., Borgianni, Y.: Making order in user experience research to support its application in design and beyond. Appl. Sci. 11(15), 6981 (2021)

    Article  Google Scholar 

  20. Hassenzahl, M., Tractinsky, N.: User experience-a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)

    Article  Google Scholar 

  21. Forlizzi, J., Ford, S.: The building blocks of experience: an early framework for interaction designers. In: Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques, pp. 419–423. Association for Computing Machinery, New York (2000)

    Google Scholar 

  22. Roto, V., Law, E.L.C., Vermeeren, A., Hoonhout, J.: 10373 Abstracts Collection--Demarcating User eXperience. In: Dagstuhl Seminar Proceedings. Taylor & Francis, London (2011)

    Google Scholar 

  23. Voinescu, A., Morgan, P.L., Alford, C., Caleb-Solly, P.: The utility of psychological measures in evaluating perceived usability of automated vehicle interfaces–a study with older adults. Transport. Res. F: Traffic Psychol. Behav. 72, 244–263 (2020)

    Article  Google Scholar 

  24. García-Díaz, J.M., García-Ruiz, M.A., Aquino-Santos, R., Edwards-Block, A.: Evaluation of a driving simulator with a visual and auditory interface. In: Collazos, C., Liborio, A., Rusu, C. (eds.) Human Computer Interaction, LNCS, vol. 8278, pp. 131–139. Springer, Cham (2013)

    Chapter  Google Scholar 

  25. Lewis, J.R.: The system usability scale: past, present, and future. Int. J. Hum. Comput. Inter. 34(7), 577–590 (2018)

    Article  Google Scholar 

  26. Albers, D., et al.: Usability evaluation—advances in experimental design in the context of automated driving human–machine interfaces. Information 15(5), 240 (2020)

    Article  Google Scholar 

  27. Detjen, H., Salini, M., Kronenberger, J., Geisler, S., Schneegass, S.: Towards transparent behavior of automated vehicles: design and evaluation of HUD concepts to support system predictability through motion intent communication. In: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction, pp. 1–12. Association for Computing Machinery, New York (2021)

    Google Scholar 

  28. Holländer, K., Colley, A., Mai, C., Häkkilä, J., Alt, F., Pfleging, B.: Investigating the influence of external car displays on pedestrians’ crossing behavior in virtual reality. In: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1–11. Association for Computing Machinery, New York (2019)

    Google Scholar 

  29. Riegler, A., Aksoy, B., Riener, A., Holzmann, C.: Gaze-based interaction with windshield displays for automated driving: impact of dwell time and feedback design on task performance and subjective workload. In: 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 151–160. Association for Computing Machinery, New York (2020)

    Google Scholar 

  30. Xu, J., Chen, J., Liu, Z.: Research on active interaction and user experience of community intelligent vehicle system. In: 2021 International Symposium on Artificial Intelligence and its Application on Media (ISAIAM), pp. 43–50. IEEE (2021)

    Google Scholar 

  31. Su, Y., Tan, Z., Dai, N.: Changes in usability evaluation of human-machine interfaces from the perspective of automated vehicles. In: Ahram, T.Z., Falcão, C.S. (eds.) AHFE 2021, LNCS, vol. 275, pp. 886–893. Springer, Cham (2021)

    Google Scholar 

  32. Xie, Y., Murphey, Y.L., Kochhar, D.S.: Personalized driver workload estimation using deep neural network learning from physiological and vehicle signals. IEEE Trans. Intell. Veh. 5(3), 439–448 (2019)

    Article  Google Scholar 

  33. Weidner, F., Broll, W.: Smart S3D TOR: intelligent warnings on large stereoscopic 3D dashboards during take-overs. In: Proceedings of the 8th ACM International Symposium on Pervasive Displays, pp. 1–7. Associa-tion for Computing Machinery, New York (2019)

    Google Scholar 

  34. Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann, Massachusetts (2016)

    Google Scholar 

  35. Hartson, H.R., Andre, T.S., Williges, R.C.: Criteria for evaluating usability evaluation methods. Int. J. Hum. Comput. Inter. 13(4), 373–410 (2001)

    Article  Google Scholar 

  36. Chen, F., Terken, J.: Automotive Interaction Design: From Theory to Practice, 1st edn. China machine press, Beijing (2022)

    Google Scholar 

  37. Vermeeren, A.P., Law, E.L.C., Roto, V., Obrist, M., Hoonhout, J., Väänänen-Vainio-Mattila, K.: User experience evaluation methods: current state and development needs. In: Proceedings of the 6th Nordic conference on human-computer interaction: Extending boundaries, NordiCHI 2010, pp. 521–530. Association for Computing Machinery, New York (2020)

    Google Scholar 

  38. Rohrer, C.: When to use which user-experience research methods. Nielsen Norman Group (2014)

    Google Scholar 

  39. Naujoks, F., Forster, Y., Wiedemann, K., & Neukum, A.: A human-machine interface for cooperative highly automated driving. In: Stanton, N., Landry, S., Di Bucchianico, G., Vallicelli, A. (eds) Advances in Human Aspects of Transportation, Advances in Intelligent Systems and Computing, vol. 484, pp. 585–595. Springer, Cham. (2017). https://doi.org/10.1007/978-3-319-41682-3_49

  40. Helander, M.G.: Handbook of Human-Computer Interaction. Hunan University Publisher, Elsevier (2014)

    Google Scholar 

  41. Hartson, R., Pyla, P.S.: The UX Book: Process and guidelines for ensuring a quality user experience, Elsevier (2012)

    Google Scholar 

  42. Cœugnet, S., Cahour, B., Kraïem, S.: A psycho-ergonomic approach of the street-crossing decision-making: toward pedestrians’ interactions with automated vehicles. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.) IEA 2018. AISC, vol. 823, pp. 132–141. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-96074-6_14

    Chapter  Google Scholar 

  43. Hock, P., Kraus, J., Babel, F., Walch, M., Rukzio, E., Baumann, M.: How to design valid simulator studies for investigating user experience in automated driving: review and hands-on considerations. In: Proceedings of the 10th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, pp. 105–117. Association for Computing Machinery, New York (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Hunan University, Changsha, 410006, Hunan, China

    Shiyan Tang, Ruilin Ouyang & Hao Tan

Authors
  1. Shiyan Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruilin Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Tan .

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Vincent G. Duffy

  2. Technische Universitat Ilmenau, Ilmenau, Germany

    Heidi Krömker

  3. Smart Future Initiative, Frankfurt am Main, Germany

    Norbert A. Streitz

  4. Kyushu University, Fukuoka, Japan

    Shin'ichi Konomi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tang, S., Ouyang, R., Tan, H. (2023). Standardization of User Experience Evaluation: Theory, Method and Promotion of High-Quality Development of Chinese Intelligent Vehicle Industry. In: Duffy, V.G., Krömker, H., A. Streitz, N., Konomi, S. (eds) HCI International 2023 – Late Breaking Papers. HCII 2023. Lecture Notes in Computer Science, vol 14057. Springer, Cham. https://doi.org/10.1007/978-3-031-48047-8_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48047-8_27

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48046-1

  • Online ISBN: 978-3-031-48047-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation