Abstract
Several studies have reported low adherence and high resistance from clinicians to adopt digital health technologies into clinical practice, particularly the use of computer-based clinical decision support systems. Poor usability and lack of integration with the clinical workflow have been identified as primary issues. Few guidelines exist on how to analyze the collected data associated with the usability of digital health technologies. In this study, we aimed to develop a coding framework for the systematic evaluation of users’ feedback generated during focus groups and interview sessions with clinicians, underpinned by fundamental usability principles and design components. This codebook also included a coding category to capture the user’s clinical role associated with each specific piece of feedback, providing a better understanding of role-specific challenges and perspectives, as well as the level of shared understanding across the multiple clinical roles. Furthermore, a voting system was created to quantitatively inform modifications of the digital system based on usability data. As a use case, we applied this method to an electronic cognitive aid designed to improve coordination and communication in the cardiac operating room, showing that this framework is feasible and useful not only to better understand suboptimal usability aspects, but also to recommend relevant modifications in the design and development of the system from different perspectives, including clinical, technical, and usability teams. The framework described herein may be applied in other highly complex clinical settings, in which digital health systems may play an important role in improving patient care and enhancing patient safety.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Trinkley, K.E., Blakeslee, W.W., Matlock, D.D., Kao, D.P., Van Matre, A.G., Harrison, R., et al.: Clinician preferences for computerised clinical decision support for medications in primary care: a focus group study. BMJ Health Care Inform. 26, e000015 (2019)
Book, B.A.: Crossing the quality chasm: a new health system for the 21st century. BMJ 323, 1192 (2001). https://doi.org/10.1136/bmj.323.7322.1192
Hessels, A., Flynn, L., Cimiotti, J.P., Bakken, S., Gershon, R.: Impact of heath information technology on the quality of patient care. Online J. Nurs. Inform. 19 (2015). https://www.ncbi.nlm.nih.gov/pubmed/27570443
Furukawa, M.F., Eldridge, N., Wang, Y., Metersky, M.: Electronic health record adoption and rates of in-hospital adverse events. J. Patient Saf. 16, 137–142 (2020)
Ebnali, M., Shah, M., Mazloumi, A.: How mHealth apps with higher usability effects on patients with Breast Cancer? In: Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, pp. 81–84 (2019). https://doi.org/10.1177/2327857919081018
Guo, C., Ashrafian, H., Ghafur, S., Fontana, G., Gardner, C., Prime, M.: Challenges for the evaluation of digital health solutions—a call for innovative evidence generation approaches. NPJ Digital Med. 3, 1–14 (2020)
Zenati, M.A., Kennedy-Metz, L., Dias, R.D.: Cognitive engineering to improve patient safety and outcomes in cardiothoracic surgery. Semin. Thorac. Cardiovasc. Surg. 32, 1–7 (2020)
Brown, W., Yen, P.-Y., Rojas, M., Schnall, R.: Assessment of the health IT usability evaluation model (Health-ITUEM) for evaluating mobile health (mHealth) technology. J. Biomed. Inform. 46, 1080–1087 (2013). https://doi.org/10.1016/j.jbi.2013.08.001
Leech, N.L., Onwuegbuzie, A.J.: An array of qualitative data analysis tools: a call for data analysis triangulation. School Psychol. Q. 22, 557–584 (2007). https://doi.org/10.1037/1045-3830.22.4.557
Willis, G.: Pretesting of health survey questionnaires: cognitive interviewing, usability testing, and behavior coding. In: Health Survey Methods, pp. 217–242 (2014). https://doi.org/10.1002/9781118594629.ch9
Nielsen, J.: Enhancing the explanatory power of usability heuristics. In: Conference Companion on Human Factors in Computing Systems - CHI 1994 (1994). https://doi.org/10.1145/259963.260333
Mosier, J.N., Smith, S.L.: Application of guidelines for designing user interface software. Behav. Inf. Technol. 5, 39–46 (1986). https://doi.org/10.1080/01449298608914497
Kontio, J., Lehtola, L., Bragge, J.: Using the focus group method in software engineering: obtaining practitioner and user experiences. In: Proceedings of 2004 International Symposium on Empirical Software Engineering, ISESE 2004. IEEE (2004). https://doi.org/10.1109/isese.2004.1334914
Mandel, T.: The Elements of User Interface Design. Wiley, New York (1997)
Rind, A., Wang, T.D., Aigner, W., Miksch, S., Wongsuphasawat, K., Plaisant, C., et al.: Interactive information visualization to explore and query electronic health records. HCI 5, 207–298 (2013)
Avrunin, G.S., Christov, S.C., Clarke, L.A., Conboy, H.M., Osterweil, L.J., Zenati, M.A.: Process driven guidance for complex surgical procedures. AMIA Annu. Symp. Proc. 2018, 175–184 (2018)
Acknowledgment
This work was supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health Under Award Number R01HL126896 (PI: Zenati). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. No conflicts of interest were declared.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ebnali, M. et al. (2022). A Coding Framework for Usability Evaluation of Digital Health Technologies. In: Kurosu, M. (eds) Human-Computer Interaction. Theoretical Approaches and Design Methods. HCII 2022. Lecture Notes in Computer Science, vol 13302. Springer, Cham. https://doi.org/10.1007/978-3-031-05311-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-05311-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05310-8
Online ISBN: 978-3-031-05311-5
eBook Packages: Computer ScienceComputer Science (R0)