Abstract
Evaluating the utility of Human-Machine Systems’ User Interfaces is not trivial. Several evaluation methods can be used to investigate if the behaviour of the user interface complies with best practices of Human-Machine Interface Design. Even when is possible to agree on which methods to use to conduct the evaluation, defining the utility requires evaluating the interface under analysis toward the company’s goals, or mission. This paper investigates how the utility, perceived by end users of interfaces, can be captured by a research instrument, as well as be represented by a structured approach based on Usability evaluation and Balanced Score Cards methodology. This is an alternative demarche for accessing the Usability of a Software System, and the main goal is helping designers and administrators to maintain and improve their systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Benyon, D.: Designing interactive systems: a comprehensive guide to HCI, UX and interaction design (2014)
Fernandez, A., Insfran, E., Abrahão, S.: Usability evaluation methods for the web: a systematic mapping study. Inf. Softw. Technol. 53(8), 789–817 (2011)
Okhovati, M., Karami, F., Khajouei, R.: Exploring the usability of the central library websites of medical sciences universities. J. Librarianship Inform. Sci. 49(3), 246–255 (2017)
Djamasbi, S., Siegel, M., Tullis, T.: Generation Y, web design, and eye tracking. Int. J. Hum Comput Stud. 68(5), 307–323 (2010)
Kous, K., Pušnik, M., Heričko, M., Polančičm G. : Usability evaluation of a library website with different end user groups. J. Librarianship Inform. Sci. 0961000618773133 (2018)
Bevan, N.: Usability is quality of use. Adv. Hum. Factors Ergon. 20(2003) 349–349 (1995)
ISO/IEC (International Organization for Standardization): Standard 9241: Ergonomic Requirements for Office Work with Visual Display Terminals (VDT)s, Part 11. Guidance on Usability (1998). https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-1:v1:en. Accessed 20 Jan 2019
Joo, S.: How are usability elements-efficiency, effectiveness, and satisfaction-correlated with each other in the context of digital libraries? Proc. Am. Soc. Inform. Sci. Technol. 47(1), 1–2 (2010)
Nielsen, J.: Usability 101: Introduction to Usability (2003)
Nielsen, J.: Usability inspection methods. In: Conference Companion on Human Factors in Computing Systems, pp. 413–414. ACM (1994)
ISO/IEC (International Organization for Standardization): Standard 9126: Software Engineering Product Quality, parts 1, 2 and 3 (2001)
Joo, S., Lin, S., Lu, K.: A usability evaluation model for academic library websites: efficiency, effectiveness and learnability. J. Libr. Inform. Stud. 9(2), 11–26 (2011)
Brinck, T., Gergle, D., Wood, S.D.: Usability for the Web: Designing Web Sites that Work. Elsevier, San Diego (2001)
Guenther, K.: Assessing Web Site Usability (2003)
Simons, T.: Speech patterns and the concept of utility in cognitive maps: the case of integrative bargaining. Acad. Manag. J. 36(1), 139–156 (1993)
Keinonen, T.: Designers, Users and Justice. Bloomsbury Publishing, London (2017)
Kaplan, R.S., Norton, D.P.: The Balanced Scorecard: Translating Strategy into Action. Harvard Business Press, Boston (1996)
Martins, I., Belo, O.: A balanced scorecard approach for evaluating the utility of a data warehousing system. In: European, Mediterranean, and Middle Eastern Conference on Information Systems, pp. 633–645. Springer, Cham (2017)
Babbie, E.: The Basics of Social Research. Wadsworth, New York (1999)
Issa, T., Isaias, P.: Sustainable Design. Springer, London (2015)
Eason, K.D.: Towards the experimental study of usability. Behav. Inform. Technol. 3(2), 133–143 (1984)
Bevan, N.: Guidelines and standards for web usability. In: Proceedings of HCI International, vol. 2005, Lawrence Erlbaum (2005)
Root, R.W., Draper, S.: Questionnaires as a software evaluation tool. In: Proceedings of the SIGCHI conference on Human Factors in Computing Systems, pp. 83–87. ACM (1983)
Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 213–218. ACM (1988)
Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Inter. 7(1), 57–78 (1995)
Brooke, J.: SUS: a “quick and dirty” usability scale.In: Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L. (eds.) Usability Evaluation in Industry. Taylor and Francis, London (1996)
Benedek, J., Miner, T.: Measuring desirability: new methods for evaluating desirability in a usability lab setting. Proc. Usability Prof. Assoc. 2003(8–12), 57 (2002)
Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: Usability Professional Association Conference, vol. 1 (2004)
Acknowledgement
This work was supported by Conselho Nacional de Pesquisa (CNPq), COMPETE: POCI-01-0145-FEDER-007043, and by FCT – Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. We also thank Instituto Federal de Educação, Ciência e Tecnologia de Goiás (IFG).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Silva, S., Belo, O. (2020). Evaluating the Utility of Human-Machine User Interfaces Using Balanced Score Cards. In: Themistocleous, M., Papadaki, M., Kamal, M.M. (eds) Information Systems. EMCIS 2020. Lecture Notes in Business Information Processing, vol 402. Springer, Cham. https://doi.org/10.1007/978-3-030-63396-7_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-63396-7_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63395-0
Online ISBN: 978-3-030-63396-7
eBook Packages: Computer ScienceComputer Science (R0)