Abstract
There is growing interest in developing video-based methods to evaluate the usefulness and usability of computerised tools in healthcare. In this paper, we propose a human-computer interaction evaluation method that protects the confidentiality of patient information and reduces litigation risks for participants by embedding and analysing performance on “probes” in complex scenarios with high face validity in a simulated setting. We describe the application of the method to a series of three studies of bar code medication administration (BCMA) software, used to reduce medication errors in the Veteran’s Administration.
This is a preview of subscription content, log in via an institution to check access.
We’re sorry, something doesn't seem to be working properly.
Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.
References
Allan EL, Barker KN (1990) Fundamentals of medication error research. Am J Hosp Pharm 47(3):555–571
Ash JS, Gorman PN, Lavelle M, Payne TH, Massaro TA, Frantz GL, Lyman JA (2003) A cross-site qualitative study of physician order entry. J Am Med Inform Assoc 10(2):188–200
Cimino JJ, Patel VL, Kushniruk AW (2001) Studying the human–computer-terminology interface. J Am Med Inform Assoc 8(2):163–173
Cohen MR (ed) (1999) Medication errors: causes, prevention, and risk management. Jones and Bartlett Publishers, Boston, MA
Dean BS, Allan EL, Barber ND, Barker KN (1995) Comparison of medication errors in an American and a British hospital. Am J Health Syst Pharm 52(22):2543–2549
Ericsson K, Simon H (1993) Protocol analysis: verbal reports as data. MIT Press, Cambridge, MA
Gillie T, Broadbent DE (1989) What makes interruptions disruptive? A study of length, similarity, and complexity. Psychol Res 50:243–250
Heath C, Hindmarsh J (2002) Analysing interaction: video, ethnography, and situated conduct. In May T (ed) Qualitative research in action: an international guide to issues in practice. Sage, London pp 99–121
Hutchins E (1995) Cognition in the wild. MIT Press, Cambridge, MA
Jordan B, Henderson A (1995) Interaction analysis: foundations and practice. J Learn Sci 4(1):39–103
Landauer TK (1995) The trouble with computers: usefulness, usability, and productivity. MIT Press, Cambridge, MA
Leape LL, Brennan TA, Laird NM, Lawthers AG, Localio AR, Barnes BA, Hebert L, Newhouse JP, Weiler PC, Hiatt HH (1991) The nature of adverse events in hospitalized patients: results from the Harvard Medical Practice Study II. N Engl J Med 324(6):377–384
Lewis JR (1995) IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 7(1):57–78
Patterson ES, Cook RI, Render ML (2002) Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc 9(5):540–553
Patterson ES, Render ML, Ebright PL (2003) Repeating human performance themes in five health care adverse events. In: Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society, Baltimore, MD, 30 September–4 October 2003, pp 1418–1422
Stablein D, Welebob E, Johnson E, Metzger J, Burgess R, Classen DC (2003) Understanding hospital readiness for computerized physician order entry. Jt Comm J Qual Saf 29(7):336–344
Acknowledgments
We thank the nurses and facilitators, Kim Zipper, Linda Fischetti, Ginny Creasman and Amanda Eisenlohr for donating their valuable time and expertise. We thank Roger Chapman and the BCMA development team, in particular Chris Tucker, Russ Carlson, Connie Johnson, Eddie Urso, and David Honeycut for working with us to improve the software. Patricia Ebright helped to develop the scenarios. We thank the anonymous reviewers for their helpful suggestions for improvement. This research was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (Project no. SAF 20-049). A VA HSR & D Merit Review Entry Program Award supported Emily Patterson. The views expressed in this article are those of the authors and do not necessarily represent the view of the Department of Veterans Affairs.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Patterson, E.S., Rogers, M.L. & Render, M.L. A simulation-based embedded probe technique for human-computer interaction evaluation. Cogn Tech Work 6, 197–205 (2004). https://doi.org/10.1007/s10111-004-0161-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10111-004-0161-3