Abstract
As an important medium for human-computer information exchange, the human-computer interface (HCI) plays an important role in the human-computer-environment system that cannot be ignored. A friendly design of HCI not only enables operators to complete operation tasks efficiently, but also has the characteristics of simplicity, beauty, easy operation and low load. Therefore, how to evaluate the friendliness of HCI design and how to judge whether the evaluation process is scientific and reasonable has become an important issue in current HCI-related research. Starting from the elements of HCI evaluation process, this paper firstly summarizes and analyzes the representative evaluation purposes in HCI evaluation research, specifically including interface usability, visual performance, interface aesthetics, suitability for special people, mental load, and evaluation exploratory research, and points out the guiding role of evaluation purposes in HCI evaluation activities; Secondly, this paper categorizes and summarizes the construction of HCI evaluation index system, determination of evaluation index weights, and selection of evaluation methods involved in HCI evaluation research from the perspective of research methods, and analyzes the advantages and disadvantages of each method in conjunction with specific HCI evaluation research; Finally, by analyzing and summarizing the research literature, this paper finds that the following problems exist in HCI evaluation research: the evaluation indexes are too abstract, the objectivity and rationality of the evaluation process need to be improved, and the feedback guidance of evaluation results on HCI design needs to be improved. This paper also points out that the future research trends of HCI evaluation research are as follows: the research target will be more inclined to the HCI of mobile touch devices, the selection of evaluation indexes needs to be more specific, the evaluation process requires researchers to pay more attention to objectivity and rationality, and the evaluation results can be better used to optimize the HCI design.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wu, C., Zhang, K.: Diagnostic user interface usability evaluation (IM) (Part 1) – introduction and evaluation. Chin. Ergon. 03, 54–57 (2000)
Wu, C., Zhang, K.: Diagnostic user interface usability evaluation (IM) (Part 2) – comparison and suggestion. Chin. Ergon. 04, 35–38 (2000)
Wu, C., Zhang, K.: Human-computer interface usability evaluation method. Psychol. Sci. 06, 727–728 (2001)
Liang, H.: Based on AHM and fuzzy comprehensive evaluation of interval number in the evaluation of mobile phone interface. Packag. Eng. 33(16), 67–71 (2012)
Liu, S., Li, Y.: Study of evaluation on display interface ergonomic evaluation for intelligent electric cooker. J. Jilin Univ. (Inf. Sci. Ed.) 34(02), 266–270 (2016)
Liu, C., Guo, F., Liu, W.: The evaluation scale of the perceived usability of the elderly shopping website interface was constructed. J. Inf. Syst. 01, 49–71 (2017)
Xu, F., Zhang, L.: Research on human-factor fitness evaluation system of smart phone APP interactive interface. Chin. J. Ergon. 24(03), 77–81 (2018)
Zhu, R., Yao, J., Tang, X.: Evaluation and reaserch of app interface for user experience optimization of doctors and patients. Dev. Innov. Mach. Electr. Prod. 32(02), 45–47+92 (2019)
Liu, W., Li, L., Fu, G.: Research on usability evaluation index system of combat command software interface based on the IPO model. J. China Acad. Electron. Inf. Technol. 16(10), 1060–1066 (2021)
Wu, J., Sun, J., Li, M.: Usability evaluation of WeChat program interface design based on entropy. Packag. Eng. 42(12), 191–196+222 (2021)
Cigdem, A., Aycan, P., Mustafa, E., et al.: Usability evaluation of TV interfaces: subjective evaluation vs. objective evaluation. Int. J. Hum.-Comput. Interact. 38(7), 661–679 (2021)
Zhou, M., Wang, Z., Zheng, Y.: Study on usability evaluation of medical software interface based on eye movement tracking technology. In: International Conference on Machine Vision (2020)
Wang, S., Li, B., Zhu, Y.: Comprehensive evaluation of usability at the mobile end interface. IOP Conf. Ser. Mater. Sci. Eng. 573(1), 012–037 (2019)
Manoela, R., Rafael R.: User interface evaluation methods for elderly: a systematic review. In: 21st Symposium on Virtual and Augmented Reality (SVR), Rio de Janeiro, Brazil, pp. 84–91. IEEE (2019)
Nugraha, A., Rolando, Puspasari, M., et al.: Usability evaluation for user interface redesign of financial technology application. IOP Conf. Ser. Mater. Sci. Eng. 505(1), 012–101 (2019)
Lestari, R., Muslim, E., Moch, B.: User interface evaluation of official store for FMCG (fast moving consumer goods) products in e-commerce website using user experience approach. IOP Conf. Ser. Mater. Sci. Eng. 505(1), 012–079 (2019)
Li, H., Tian, S., Li, F., et al.: Usability evaluation of mining machinery interface based on eye movement experiment. In: Rebelo, F., Soares, M. (eds.) Advances in Ergonomics in Design. AHFE 2017: Advances in Intelligent Systems and Computing, vol. 588, pp. 591–599. Springer, Cham (2017)
Alonso, R., Mosqueira, R., Moret, B.: A systematic and generalizable approach to the heuristic evaluation of user interfaces. Int. J. Hum.-Comput. Interact. 34(12), 1169–1182 (2018)
Rim, R., Mohamed, A., Adel, M., et al.: Evaluation method for an adaptive web interface: GOMS model. In: Madureira, A., Abraham, A., Gamboa, D., Novais, P. (eds.) ISDA 2016: Intelligent Systems Design and Applications, Advances in Intelligent Systems and Computing, vol. 557, pp. 116–124. Springer, Cham (2017)
Cui, M., Zhu, L.: Usability evaluation methods of user interface based on mobile games using fuzzy methods. In: Chen, Y., Christie, M., Tan, W. (eds.) Smart Graphics, pp. 124–131. Springer International Publishing, Cham (2017)
Bessghaier, N., Souii, M.: Towards usability evaluation of hybrid mobile user interfaces. In: 2017 IEEE/ACS 14th International Conference on Computer Systems and Applications (AICCSA), Hammamet, pp. 895–900. IEEE (2017)
Mator, J.D., et al.: Usability: adoption, measurement, value. Hum. Factors 63(6), 956–973 (2021)
Allah, K.K., Ismail, N.A., Hasan, L., Leng, W.Y.: Usability evaluation of web search user interfaces from the elderly perspective. Int. J. Adv. Comput. Sci. Appl. 12(12), 647–657 (2021)
Wu, Y., Cheng, J., Kang, X.: Study of smart watch interface usability evaluation based on eye-tracking. In: Marcus, A. (ed.) DUXU 2016. LNCS, vol. 9748, pp. 98–109. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-40406-6_10
Normark, C.J.: Design and evaluation of a touch-based personalizable in-vehicle user interface. Int. J. Hum.-Comput. Interact. 31(11), 731–745 (2015)
Xiao, G., Li, X., Xiao, M.: An ease-of-use evaluation of the human-machine interface. J. Southwest China Normal Univ. (Nat. Sci. Ed.) 34(03), 98–102 (2009)
Wang, H., Xie, W., Zhao, Y.: Research on evaluation method about handing performance of man-machine interface of weapons. Command Control Simul. 35(02), 68–70+84 (2013)
Nielsen, J.: Usability Engineering. Morgan Kaufmann (1994)
Liu, W., Yuan, X., Liu, Z., et al.: Comprehensive evaluation indexes and evaluation method for man-machine compatibility of display/control interface. China Saf. Sci. J. 04, 36–39 (2004)
Wang, J., Cai, W.: Ergonomics evaluation of human-machine interface based on spatial vision characteristics. Tactical Missile Technol. 06, 107–111 (2012)
Gao, S., Zhu, L., Li, Y.: Color matching evaluation of APP user interface for elderly based on grey clustering method. Packag. Eng. 42(06), 198–205 (2021)
Ma, Y., Zhai, L., Wang, X., Liang, H.: Evaluation of the matching optimization of human-machine interface in the cab. In: 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE), Harbin, China, pp. 152–156. IEEE (2020)
Riegler, A., Holzmann, C.: Measuring visual user interface complexity of mobile applications with metrics. Interact. Comput. 30(3), 207–223 (2018)
Ngo, D.C.L., Teo, L.S., Byrne, J.G.: Modelling interface aesthetics. Inf. Sci. 152, 25–46 (2003)
Zhou, L., Xue, C., Tang, W., et al.: Aesthetic evaluation method of interface elements layout design. J. Comput.-Aided Des. Comput. Graph. 25(005), 758–766 (2013)
Zhang, N., Wang, J., Yang, Y.: Evaluation method of aesthetic image for man-machine interface form elements layout design. Mech. Sci. Technol. Aerosp. Eng. 34(10), 1594–1598 (2015)
Dai, Y., Xue, C., Wang, H.: Research on interface beauty evaluation method based on AHP. Design 08, 123–125 (2018)
Ren, X., Xue, C.: Evaluation of interface element layout of camera connect APP based on Meidu calculation. Design 04, 142–143 (2018)
Li, H.: Overview of human-machine interface Meidu research. Ind. Des. 01, 93–94 (2019)
Lv, J., Sun, W., Pan, W., et al.: Evaluation of information interface layout beauty based on cognitive characteristics. Packag. Eng. 40(18), 220–226 (2019)
Zhou, A., Zhou, C., Ooyang, J., et al.: Model of synthetic evaluation on interface stylistic beauty based on moderately standardized of index. J. Zhejiang Univ. (Eng. Sci.) 54(12), 2273–2285 (2020)
Pastushenko, O., Hynek, J., Hruška, T.: Evaluation of user interface design metrics by generating realistic‐looking dashboard samples. Expert Syst. 38(5) (2021)
Deng, L., Wang, G.: Quantitative evaluation of visual aesthetics of human-machine interaction interface layout. Comput. Intell. Neurosci. 2020, 1–14 (2020)
Kong, Q., Guo, Q.: Comprehensive evaluation method of interface elements layout aesthetics based on improved AHP. In: Rebelo, F., Soares, M.M. (eds.) Advances in Ergonomics in Design, pp. 509–520. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-94706-8_54
Yang, Y., Zhang, B., Li, X., et al.: Evaluation of navigation interface design of ASD children intervention APP based on AHP-entropy weight method. Packag. Eng. 43(12), 165–173 (2022)
Susilo, F.F., Park, J.-H., Park, J.-M.: Evaluation of touch-based interface design for the elderly based on cultural differences. In: Stephanidis, C. (ed.) HCI 2018. CCIS, vol. 851, pp. 211–219. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-92279-9_29
Krel, M., Kožuh, I., Debevc, M.: Heuristic evaluation of a mobile telecare system for older adults. In: Miesenberger, K., Kouroupetroglou, G. (eds.) Computers Helping People with Special Needs, pp. 391–398. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-94274-2_56
Trilar, J., Sobo, T., Duh, E.S.: Family-centered design: interactive performance testing and user interface evaluation of the slovenian edavki public tax portal. Sensors 21(15) (2021)
Rakhra, A.K., Mann, D.D.: Design and evaluation of individual elements of the interface for an agricultural machine. J. Agric. Saf. Health 24(1), 27–42 (2018)
Chen, Y., Yan, S., Tran, C.C.: Comprehensive evaluation method for user interface design in nuclear power plant based on mental workload. Nucl. Eng. Technol. 51(2), 453–462 (2019)
Li, M., Albayrak, A., Zhang, Yu., van Eijk, D.: Multiple factors mental load evaluation on smartphone user interface. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.) IEA 2018. AISC, vol. 827, pp. 302–315. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-96059-3_33
Li, F., Wang, B.: Evaluation methods of the relevance between PC and mobile terminal layout. Packag. Eng. 38(20), 143–149 (2017)
Li, J., Yu, S., Liu, W.: Cognitive characteristic evaluation of CNC interface layout based on eye-tracking. J. Comput.-Aided Des. Comput. Graph. 29(07), 1334–1342 (2017)
Zhou, Q., Cheng, Y., Liu, Z., Chen, Y., Li, C.: The layout evaluation of man-machine interface based on eye movement data. In: Bagnara, S., Tartaglia, R., Albolino, S., Alexander, T., Fujita, Y. (eds.) IEA 2018. AISC, vol. 824, pp. 64–75. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-96071-5_7
Sun, L., Han, B., Zhang, W.: Evaluation of mobile APP interface design for english learning based on eye movement experiment. Chin. J. Ergon. 27(02), 1–8 (2021)
Wang, J., Ma, J., Lv, S.: Evaluation of WinCC interface based on eye tracking technology. Exp. Technol. Manag. 36(06), 53–57 (2019)
Du, D., Pang, Q., Wu, Y.: Modern Comprehensive Evaluation Methods and Selected Cases, 3rd edn. Tsinghua University Press, Beijing (2021)
Lin, H., Zhang, Y.: Quality evaluation of mobile game main interface design based on information entropy. Design 07, 57–59 (2017)
Wang, M., Yu, S., Yang, Y.: Human-machine interface evaluation of aircraft cockpit based on Fuzzy AHP-GEM. J. Mach. Des. 34(02), 105–109 (2017)
Baker, N.A., Redfern, M.S.: Developing an observational instrument to evaluate personal computer keyboarding style. Appl. Ergon. 36(3), 345–354 (2005)
Song, H., Guo, M.: Study on methods and application of human-computer interface ergonomics evaluation for fighter cockpit. Aeronaut. Sci. Technol. 28(05), 28–32 (2017)
Zhang, X., Yin, Y., Feng, Y., et al.: Study on ergonomics evaluation index system for display and control interface of automobile. Energy Conserv. Environ. Prot. Transp. 12(04), 12–16 (2016)
Song, H., Xu, X.: Interface ergonomics evaluation methods and applied research for fighter cockpit. In: Long, S., Dhillon, B.S. (eds.) MMESE 2018. LNEE, vol. 527, pp. 333–340. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-2481-9_38
Yuan, S., Gao, H., Wang, W., et al.: Multi-image evaluation for human-machine interface based on Kansei engineering. Chin. J. Eng. Des. 24(05), 523–529 (2017)
Zhu, S., Qu, Y., Wang, W., et al.: Evaluation model of human-machine interface of the CNC based on improved FAHP-TOPSIS. Mach. Des. Res. 35(06), 144–148+156 (2019)
Zhao, H., He, L., Lin, L., et al.: Digital human-machine interface experience measurement based on TOPSIS. J. Mach. Des. 33(01), 120–123 (2016)
Sun, J., Jiang, T., Wang, C., et al.: Research on ergonomic evaluation indicator and methodology for interactive interface of spacecraft software. Manned Spaceflight 26(02), 208–213 (2020)
Zhao, J., Hao, J., Wang, S., et al.: Research on the human-machine interface evaluation of the tank. Microcomput. Inf. 25(19), 243–244+224 (2009)
Rao, D., Chen, Y., Lv, W., et al.: Comprehensive evaluation of human-machine interface layout in ttrain driving cab based on the multi-level fuzzy theory. Packag. Eng. 42(04), 61–69 (2021)
Song, H.: Application of the hierarchy analysis-grey ccorrelation in assessment of software interface. J. Donghua Univ. (Nat. Sci.) 03, 318–321 (2008)
Li, B., Ji, X., Ye, C.: Evaluation of DSV man-machine interface based on fuzzy AHP & grey correlation. Manuf. Autom. 36(23), 71–75 (2014)
Yan, S., Zhang, Z., Chen, N., et al.: Evaluation of human-machine interface based on human factor indicator. Chin. Ergon. 04, 26–30 (2004)
Yan, S., Li, Q., Zhang, Z., et al.: Research on subjective evaluation method in human-machine-interface based on grey theory. J. Harbin Eng. Univ. 01, 98–100+104 (2005)
Yan, S., Yu, X., Zhang, Z., et al.: Evaluation method of human-machine interface of virtual meter based on RBF network. J. Syst. Simul. 24, 5731–5735 (2007)
Qiu, Y., Zhu, Z., Mao, E., et al.: Virtual ergonomics design and evaluation of agricultural equipment cab. Trans. Chin. Soc. Agric. Eng. 27(03), 117–121 (2011)
Zhang, X., Li, K., Guo, J.: An evaluation method for software user interface. J. Shaanxi Normal Univ. (Nat. Sci. Ed.) S1, 185–187 (2005)
Xiong, S., Yao, Z.: An evaluation model for sonar ssoftware user interface. Ship Electron. Eng. 36(03), 98–101 (2016)
Index weight. https://baike.baidu.com/item/%E6%8C%87%E6%A0%87%E6%9D%83%E9%87%8D?forcehttps=1%3Ffr%3Dkg_hanyu. Accessed 11 Apr 2022
Zhang, X., Guo, J., Zhang, F.: Software user interface evaluation based on grey incidence analysis. Comput. Eng. Des. 14, 2661–2662 (2006)
Wang, B., Li, Y.: Five-level scale assignment-fuzzy comprehensive evaluation of huma-machine interface quality. Comput. Appl. Softw. 30(02), 22–25 (2013)
Luo, A., Tan, D., Zeng, Y.: Fuzzy multi-level synthetic evaluation for human computer interface. Fuzzy Syst. Math. 04, 80–86 (1999)
Yan, S., Li, Q., He, P., et al.: A comprehensive evaluation of software user interface. J. Harbin Eng. Univ. 05, 653–657 (2004)
Ding, W., Yang, G., Fang, M.: Human-computer interface synthetical evaluation based on AHP. J. Nanyang Normal Univ. 12, 72–73+85 (2007)
Zhao, C., Li, J., Ren, J., et al.: Research on evaluation method of human-machine interface of fitness equipment based on multi-factor fusion. J. Graph. 40(05), 932–935 (2019)
Zhang, R., Wang, M., Guo, Y., et al.: Design and evaluation of man-machine interface of oil field water injection system platform. J. Mach. Des. 38(07), 118–125 (2021)
Li, H., Wang, S., Li, J.: Research on human-machine interface evaluation method based on QFD-PUGH. J. Graph. 42(06), 1043–1050 (2021)
Xiao, X., Wanyan, X., Zhuang, D.: Comprehensive evaluation model of multidimensional visual coding on display interface. J. Beijing Univ. Aeronaut. Astronaut. 41(06), 1012–1018 (2015)
Zhang, R.: Evaluation of the user interface of university digital library based on the analysis of grey ideal relation. Value Eng. 33(33), 231–232 (2014)
Bao, W., Hu, Y., Zhu, L.: Fuzzy comprehensive evaluation of interface design of elderly intelligent products based on grey correlation. Hunan Packag. 33(04), 51–54 (2018)
Wang, Y., Wang, X.: Evaluation of organizational interface management of large-scale engineering projects based on entropy method and grey theory. Archit. Technol. 49(02), 213–216 (2018)
Liu, X., et al.: Human reliability evaluation based on objective and subjective comprehensive method used for ergonomic interface design. Math. Probl. Eng. 2021, 1–16 (2021)
He, S., Zhi, J.: Evaluation of train passenger interface design based on analytic hierarchy process with independent weight method. J. Southwest Jiaotong Univ. 56(04), 897–904 (2021)
Xia, C., Jiang, K.: Weight assignment of evaluation indexes for human-machine interface based on reliability coefficient. J. Shenyang Univ. Technol. 35(01), 53–57 (2013)
Wang, F., Zhou, X.: Research on APP evaluation based on visually impaired user experience. Sci. Technol. Innov. 02, 103–104 (2021)
Wang, Y., Wang, X., Wu, T., et al.: Interface design evaluation of mobile phone music APP based on eye-tracking experiment. Sci. Technol. Eng. 18(09), 266–271 (2018)
Yan, H., Wang, J., Wang, W.: Eye movement experiment analysis and evaluation of shopping app interface design. Ind. Des. Res. 00, 206–211 (2018)
Li, X., Shou, Z., Zhang, T., et al.: Applicability evaluation model of customizable user interface based on rough set-FAHP. J. Guilin Univ. Electron. Technol. 38, 279–284 (2018)
He, S., Zhi, J., Du, Y., et al.: A review of research on man-machine design evaluation of equipment driving interface. Mach. Des. Res. 35(05), 97–103 (2019)
Eliseo, M.A., Casac, B.S., Gentil, G.R.: A comparative study of video content user interfaces based on heuristic evaluation. In: 2017 12th Iberian Conference on Information Systems and Technologies (CISTI), Lisbon, Portugal, pp. 1–6. IEEE (2017)
Wang, Y., Li, P.: Research on evaluation of digital man-machine interface of nuclear power plant based on personnel performance. Atomic Energy Sci. Technol. 55(03), 534–543 (2021)
Hellianto, G.R., Iqbal, B.M., Komarudin: Ergonomics evaluation of game interface design provisions on various computer based monitor screens. In: Proceedings of the 2017 International Conference on Industrial Design Engineering-ICIDE 2017, Dubai, United Arab Emirates, pp. 38–44. ACM Press (2017)
Muslim, E., Lestari, R.A., Hazmy, A.I., Alvina, S.: User interface evaluation of mobile application KRL access using user experience approach. IOP Conf. Ser. Mater. Sci. Eng. 508, 012110 (2019)
Wang, D., Qu, Y.: Research on human-computer interface evaluation of allegation system based on EEG indicators. J. Ordnance Equipment Eng. 42(11), 196–203 (2021)
Guo, F., Qian, X.: Human Factor Engineering, 1st edn. China Machine Press, Beijing (2005)
Grey system theory. https://baike.baidu.com/item/%E7%81%B0%E8%89%B2%E7%B3%BB%E7%BB%9F%E7%90%86%E8%AE%BA/2262621?fr=aladdin. Accessed 22 Apr 2022
Xia, C., Li, Q., Liu, Y.: Research on evaluation method in human-machine interface based on grey interval clustering. J. Jiamusi Univ. (Nat. Sci. Ed.) 38(02), 37–39 (2020)
Jin, X., Mao, E., Song, Z.: Study on qualitative classification evaluation method of vehicle cab human-machine interface. In: Advances in Man-Machine-Environment Systems Engineering, vol. 07 (2005)
Wang, Lu., Xue, Q., Hao, J., Yu, H.: Research of human-machine interface evaluation based on Cogtool. In: Yamamoto, S., Mori, H. (eds.) HCII 2019. LNCS, vol. 11569, pp. 371–391. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22660-2_27
Guo, Y.: Theory and Method of Comprehensive Evaluation, 1st edn. Science Press, Beijing (2002)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, X., Liu, M., Gong, L., Gu, Y., Shidujaman, M. (2023). A Review of Human-Computer Interface Evaluation Research Based on Evaluation Process Elements. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14011. Springer, Cham. https://doi.org/10.1007/978-3-031-35596-7_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-35596-7_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35595-0
Online ISBN: 978-3-031-35596-7
eBook Packages: Computer ScienceComputer Science (R0)