Angela Mastrianni
Aleksandra Sarcevic
Skip Abstract Section
Abstract
Digital cognitive aids have the potential to serve as clinical decision support platforms, triggering alerts about process delays and recommending interventions. In this mixed-methods study, we examined how a digital checklist for pediatric trauma resuscitation could trigger decision support alerts and recommendations. We identified two criteria that cognitive aids must satisfy to support these alerts: (1) context information must be entered in a timely, accurate, and standardized manner, and (2) task status must be accurately documented. Using co-design sessions and near-live simulations, we created two checklist features to satisfy these criteria: a form for entering the pre-hospital information and a progress slider for documenting the progression of a multi-step task. We evaluated these two features in the wild, contributing guidelines for designing these features on cognitive aids to support alerts and recommendations in time- and safety-critical scenarios.
- [1] . 2015. An electronic checklist improves transfer and retention of critical information at intraoperative handoff of care. Anesthesia & Analgesia 120, 1 (2015), 96–104.
DOI: Google Scholar
Cross Ref
- [2] , and with the HITEC Investigators. 2017. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making 17, 1 (2017), 36.
DOI: Google ScholarCross Ref - [3] . 2013. Advanced trauma life support (ATLS®): The ninth edition. The Journal of Trauma and Acute Care Surgery 74, 5 (2013), 1363–1366.
DOI: Google ScholarCross Ref - [4] . 2001. Type and Crossmatch of the Trauma Patient. Journal of Trauma and Acute Care Surgery 50, 5 (2001), 878–881.Google ScholarCross Ref
- [5] . 1999. Accumulating and coordinating: occasions for information technologies in medical work. Computer Supported Cooperative Work (CSCW) 8, 4 (1999), 373–401.
DOI: Google ScholarDigital Library
- [6] . 2019. Critter: Augmenting creative work with dynamic checklists, automated quality assurance, and contextual reviewer feedback. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–12.
DOI: Google ScholarDigital Library - [7] . 2013. Electronic prehospital records are often unavailable for emergency department medical decision making. Western Journal of Emergency Medicine 14, 5 (2013), 482–488.
DOI: Google ScholarCross Ref - [8] . 2001. TODAY'S ELECTRONIC CHECKLISTS REDUCE LIKELIHOOD OF CREW ERRORS AND HELP PREVENT MISHAPS. ICAO Journal. Retrieved July 28, 2022 from https://trid.trb.org/view/607283Google Scholar
- [9] . 2006. Representations at work: A national standard for electronic health records. In Proceedings of the 2006 20th Anniversary Conference on Computer Supported Cooperative Work.
DOI: Google ScholarDigital Library - [10] . 2019. Beepless: Using peripheral interaction in an intensive care setting. In Proceedings of the 2019 on Designing Interactive Systems Conference. 607–620.
DOI: Google ScholarDigital Library - [11] . 2019. Human-Centered tools for coping with imperfect algorithms during medical decision-making. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.
DOI: Google ScholarDigital Library - [12] . 2018. Healthcare providers’ perceptions of a situational awareness display for emergency department resuscitation: A simulation qualitative study. International Journal for Quality in Health Care 30, 1 (2018), 16–22.
DOI: Google ScholarCross Ref - [13] . 2010. Documenting transitional information in EMR. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1787–1796.
DOI: Google ScholarDigital Library - [14] . 2021. Crisis checklists in emergency medicine: another step forward for cognitive aids. BMJ Quality & Safety 30, 9 (2021), 689–693.
DOI: Google ScholarCross Ref - [15] . 2014. Reducing confusion about grounded theory and qualitative content analysis: Similarities and differences. The Qualitative Report 19, 32 (2014), 1–20.
DOI: Google ScholarCross Ref - [16] . 2016. Smart checklists to improve healthcare outcomes. In Proceedings of the International Workshop on Software Engineering in Healthcare Systems (SEHS’16). 54–57.
DOI: Google ScholarDigital Library - [17] . 2018. Vibrotactile alarm display for critical care. In Proceedings of the 7th ACM International Symposium on Pervasive Displays. 1–7.
DOI: Google ScholarDigital Library - [18] . 2019. Design and evaluation of braced touch for touchscreen input stabilisation. International Journal of Human-Computer Studies 122 (2019), 21–37.
DOI: Google ScholarCross Ref - [19] . 2015. A comparison of paper documentation to electronic documentation for trauma resuscitations at a level i pediatric trauma center. Journal of Emergency Nursing: JEN 41, 1 (2015), 52–56.
DOI: Google ScholarCross Ref - [20] . 2002. Achieving quality in clinical decision making: Cognitive strategies and detection of bias. Academic Emergency Medicine 9, 11 (2002), 1184–1204.
DOI: Google ScholarCross Ref - [21] . 2017. Intelligent dynamic clinical checklists improved checklist compliance in the intensive care unit. British Journal of Anaesthesia 119, 2 (2017), 231–238.
DOI: Google ScholarCross Ref - [22] . 2014. Factors associated with safety events in air traffic control. Ergonomics SA : Journal of the Ergonomics Society of South Africa 26, 1 (2014), 2–18.
DOI: Google ScholarCross Ref - [23] . 2014. Efficiencies Gained by Using Electronic Medical Record and Reports in Trauma Documentation. Journal of Trauma Nursing | JTN 21, 2 (2014), 68–71.
DOI: Google ScholarCross Ref - [24] . 2014. Evaluating a clinical decision support interface for end-of-life nurse care. In Proceedings of the CHI ’14 Extended Abstracts on Human Factors in Computing Systems. 1633–1638.
DOI: Google ScholarDigital Library - [25] . 2013. Effectiveness of decision support for families, clinicians, or both on HPV vaccine receipt. Pediatrics 131, 6 (2013), 1114–1124.
DOI: Google ScholarCross Ref - [26] . 2013. A review of 25 years of CSCW research in healthcare: Contributions, challenges and future agendas. Computer Supported Cooperative Work 22, 4 (2013), 609–665.
DOI: Google ScholarDigital Library - [27] . 2016. Visual TASK: A collaborative cognitive aid for acute care resuscitation. arXiv:1605.05224 [cs]. Retrieved June 24, 2021 from http://arxiv.org/abs/1605.05224Google Scholar
- [28] . 2004. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Education Today 24, 2 (2004), 105–112.
DOI: Google ScholarCross Ref - [29] . 2015. Smarter clinical checklists: how to minimize checklist fatigue and maximize clinician performance. Anesthesia & Analgesia 121, 2 (2015), 570–573.
DOI: Google ScholarCross Ref - [30] . 2014. Randomised trial comparing the recording ability of a novel, electronic emergency documentation system with the AHA paper cardiac arrest record. Emergency Medicine Journal : EMJ 31, 10 (2014), 833.
DOI: Google ScholarCross Ref - [31] . 2006. Patterns of errors contributing to trauma mortality. Annals of Surgery 244, 3 (2006), 371–380.
DOI: Google ScholarCross Ref - [32] . 2016. Real-time tablet-based resuscitation documentation by the team leader: Evaluating documentation quality and clinical performance. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 24, 1 (2016), 51.
DOI: Google ScholarCross Ref - [33] . 2019. Cognitive aids in acute care: Investigating how cognitive aids affect and support In-hospital emergency teams. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.
DOI: Google ScholarDigital Library - [34] . 2019. How participatory design works: Mechanisms and effects. In Proceedings of the 31st Australian Conference on Human-Computer-Interaction. 30–41.
DOI: Google ScholarDigital Library - [35] . 2017. Lower alert rates by clustering of related drug interaction alerts. Journal of the American Medical Informatics Association : JAMIA 24, 1 (2017), 54–59.
DOI: Google ScholarCross Ref - [36] . 2020. Leans illusion in hexapod simulator facilitates erroneous responses to artificial horizon in airline pilots. Human Factors (2020), 0018720820975248.
DOI: Google ScholarCross Ref - [37] . 2020. From paper flight strips to digital strip systems: Changes and similarities in air traffic control work practices. Proceedings of the ACM on Human-Computer Interaction 4, CSCW1 (2020), 1–21.
DOI: Google ScholarDigital Library - [38] . 2016. An eye-tracking evaluation of driver distraction and unfamiliar road signs. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. 153–160.
DOI: Google ScholarDigital Library - [39] . 2019. Temporal rhythms and patterns of electronic documentation in time-critical medical work. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
DOI: Google ScholarDigital Library - [40] . 2011. Effects of clinical decision-support systems on practitioner performance and patient outcomes: A synthesis of high-quality systematic review findings. Journal of the American Medical Informatics Association: JAMIA 18, 3 (2011), 327–334.
DOI: Google ScholarCross Ref - [41] . 2020. Remote communication in wilderness search and rescue: Implications for the design of emergency distributed-collaboration tools for network-sparse environments. Proceedings of the ACM on Human-Computer Interaction 4, GROUP (2020), 10:1–10:26.
DOI: Google ScholarDigital Library - [42] . 2020. “You Have to Piece the Puzzle Together”: Implications for designing decision support in intensive care. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. 1509–1522.
DOI: Google ScholarDigital Library - [43] . 2014. Effect of a checklist on advanced trauma life support task performance during pediatric trauma resuscitation. Academic Emergency Medicine: Official Journal of the Society for Academic Emergency Medicine 21, 10 (2014), 1129–1134.
DOI: Google ScholarCross Ref - [44] . 2020. Experience matters: Design and evaluation of an anesthesia support tool guided by user experience theory. In Proceedings of the 2020 ACM Designing Interactive Systems Conference. 1523–1535.
DOI: Google ScholarDigital Library - [45] . 2017. Checking the lists: A systematic review of electronic checklist use in health care. Journal of Biomedical Informatics 71 (2017), S6–S12.
DOI: Google ScholarDigital Library - [46] . 2021. Towards dynamic checklists: Understanding contexts of use and deriving requirements for context-driven adaptation. ACM Transactions on Computer-Human Interaction 28, 2 (2021), 1–33.
DOI: Google ScholarDigital Library - [47] . 2019. Comparing the effects of paper and digital checklists on team performance in time-critical work. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–13.
DOI: Google ScholarDigital Library - [48] . 2017. Exploring design opportunities for a context-adaptive medical checklist through technology probe approach. In Proceedings of the 2017 Conference on Designing Interactive Systems. 57–68.
DOI: Google ScholarDigital Library - [49] . 2020. Checklist design reconsidered: Understanding checklist compliance and timing of interactions. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.
DOI: Google ScholarDigital Library - [50] . 2019. Development of a checklist for the prevention of intradialytic hypotension in hemodialysis care: Design considerations based on activity theory. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–14.
DOI: Google ScholarDigital Library - [51] . 2014. Impact of a clinical decision support system for high-alert medications on the prevention of prescription errors. International Journal of Medical Informatics 83, 12 (2014), 929–940.
DOI: Google ScholarCross Ref - [52] . 2021. A human-AI collaborative approach for clinical decision making on rehabilitation assessment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.
DOI: Google ScholarDigital Library - [53] . 2007. The influence of distraction and driving context on driver response to imperfect collision warning systems. Ergonomics 50, 8 (2007), 1264–1286.
DOI: Google ScholarCross Ref - [54] . 2012. Integrating usability testing and think-aloud protocol analysis with “near-live” clinical simulations in evaluating clinical decision support. International Journal of Medical Informatics 81, 11 (2012), 761–772.
DOI: Google ScholarCross Ref - [55] . 2021. Automating clinical documentation with digital scribes: Understanding the impact on physicians. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12.
DOI: Google ScholarDigital Library - [56] . 2010. Intraosseous access. The Journal of Emergency Medicine 39, 4 (2010), 468–475.
DOI: Google ScholarCross Ref - [57] . 2012. Novel use of electronic whiteboard in the operating room increases surgical team compliance with pre-incision safety practices. Surgery 151, 5 (2012), 660–666.
DOI: Google ScholarCross Ref - [58] . 2017. Helping experts and expert teams perform under duress: an agenda for cognitive aid research. Anaesthesia 72, 3 (2017), 289–295.
DOI: Google ScholarCross Ref - [59] . 2013. The use of cognitive aids during emergencies in anesthesia: A review of the literature. Anesthesia & Analgesia 117, 5 (2013), 1162–1171.
DOI: Google ScholarCross Ref - [60] . 2020. Understanding digital checklist use through team communication. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–8.
DOI: Google ScholarDigital Library - [61] . 2021. Designing interactive alerts to improve recognition of critical events in medical emergencies. In Proceedings of the Designing Interactive Systems Conference 2021. Association for Computing Machinery, New York, NY, 864–878. Retrieved June 29, 2021 from Google ScholarDigital Library
- [62] . 2020. Reducing alert burden in electronic health records: State of the art recommendations from four health systems. Applied Clinical Informatics 11, 1 (2020), 1–12.
DOI: Google ScholarCross Ref - [63] . 2010. Computerized clinical decision support for prescribing: Provision does not guarantee uptake. Journal of the American Medical Informatics Association : JAMIA 17, 1 (2010), 25–33.
DOI: Google ScholarCross Ref - [64] . 2018. Medication-related clinical decision support alert overrides in inpatients. Journal of the American Medical Informatics Association 25, 5 (2018), 476–481.
DOI: Google ScholarCross Ref - [65] . 2021. Disentangling the clinical data chaos: User-centered interface system design for trauma centers. PLOS ONE 16, 5 (2021), e0251140.
DOI: Google ScholarCross Ref - [66] . 2015. Technological and organizational adaptation of EMR implementation in an emergency department. ACM Transactions on Computer-Human Interaction 22, 1 (2015), 1–24.
DOI: Google ScholarDigital Library - [67] . 2012. The effects of EMR deployment on doctors’ work practices: A qualitative study in the emergency department of a teaching hospital. International Journal of Medical Informatics 81, 3 (2012), 204–217.
DOI: Google ScholarCross Ref - [68] . 2014. Improving ATLS performance in simulated pediatric trauma resuscitation using a checklist. Annals of Surgery 259, 4 (2014), 807–813.
DOI: Google ScholarCross Ref - [69] . 2017. Can teamwork and situational awareness (SA) in ED resuscitations be improved with a technological cognitive aid? Design and a pilot study of a team situation display. Journal of Biomedical Informatics 76 (2017), 154–161.
DOI: Google ScholarDigital Library - [70] . 2012. Early death and late morbidity after blood transfusion of injured children: A pilot study. Journal of Pediatric Surgery 47, 8 (2012), 1587–1591.
DOI: Google ScholarCross Ref - [71] . 2020. Right information, right time, right place: Physical alignment and misalignment in healthcare practice. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, 1–12. Retrieved August 31, 2021 from Google ScholarDigital Library
- [72] . 2015. Prehospital use of blood and plasma in pediatric trauma patients. Air Medical Journal 34, 1 (2015), 40–43.
DOI: Google ScholarCross Ref - [73] . 2019. Challenges of developing a digital scribe to reduce clinical documentation burden. npj Digital Medicine 2, 1 (2019), 1–6.
DOI: Google ScholarCross Ref - [74] . 2019. Assessing workflow of emergency physicians in the use of clinical decision support. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63, 1 (2019), 772–776.
DOI: Google ScholarCross Ref - [75] . 2008. Co-creation and the new landscapes of design. CoDesign 4, 1 (2008), 5–18.
DOI: Google ScholarCross Ref - [76] . 2012. Teamwork errors in trauma resuscitation. ACM Transactions on Computer-Human Interaction 19, 2 (2012), 1–30.
DOI: Google ScholarDigital Library - [77] . 2017. Checklist as a memory externalization tool during a critical care process. AMIA Annual Symposium Proceedings. 1080–1089.Google Scholar
- [78] . 2015. Supporting human-automation interaction in the rail industry by applying lessons from aviation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 59, 1 (2015), 1661–1665.
DOI: Google ScholarCross Ref - [79] . 2016. Complex decision-making in clinical practice. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing. 993–1004.
DOI: Google ScholarDigital Library - [80] . 2021. BLEXVFR: Designing a mobile assistant for pilots. In Proceedings of the Adjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction. 1–5.
DOI: Google ScholarDigital Library - [81] . 2009. Implementation pearls from a new guidebook on improving medication use and outcomes with clinical decision support. Journal of Healthcare Information Management 23, 4 (2009), 38–45.Google Scholar
- [82] . 2008. Grand challenges in clinical decision support. Journal of Biomedical Informatics 41, 2 (2008), 387–392.
DOI: Google ScholarDigital Library - [83] . 2003. Medical staff in emergency situations: Severity of patient status predicts stress hormone reactivity and recovery. Occupational and Environmental Medicine 60, 5 (2003), 373–375.
DOI: Google ScholarCross Ref - [84] . 2009. Questioning, exploring, narrating and playing in the control room to maintain system safety. Cognition, Technology and Work 11, 4 (2009), 279–291.
DOI: Google ScholarDigital Library - [85] . 2015. Promising adoption of an electronic clinical decision support system for antenatal and intrapartum care in rural primary healthcare facilities in sub-Saharan Africa: The QUALMAT experience. International Journal of Medical Informatics 84, 9 (2015), 647–657.
DOI: Google ScholarCross Ref - [86] . 2020. An overview of clinical decision support systems: Benefits, risks, and strategies for success. npj Digital Medicine 3, 1 (2020), 1–10.
DOI: Google ScholarCross Ref - [87] . 2016. The effect of an electronic checklist on critical care provider workload, errors, and performance. Journal of Intensive Care Medicine 31, 3 (2016), 205–212.
DOI: Google ScholarCross Ref - [88] . 2019. Impact of scribes on emergency medicine doctors’ productivity and patient throughput: Multicentre randomised trial. BMJ 364 (2019), l121.
DOI: Google ScholarCross Ref - [89] . 2021. Emergency medicine electronic health record usability: where to from here? Emergency Medicine Journal 38, 6 (2021), 408–409.
DOI: Google ScholarCross Ref - [90] . 2021. “Brilliant AI Doctor” in rural clinics: Challenges in AI-Powered clinical decision support system deployment. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–18.
DOI: Google ScholarDigital Library - [91] . 2008. A four-phase model of the evolution of clinical decision support architectures. International Journal of Medical Informatics 77, 10 (2008), 641–649.
DOI: Google ScholarCross Ref - [92] . 2011. Maintaining shared mental models in anesthesia crisis care with nurse tablet input and large-screen displays. In Proceedings of the 24th Annual ACM Symposium Adjunct on User Interface Software and Technology. 71–72.
DOI: Google ScholarDigital Library - [93] . 2014. Supporting crisis response with dynamic procedure aids. In Proceedings of the 2014 Conference on Designing Interactive Systems. 315–324.
DOI: Google ScholarDigital Library - [94] . 2015. Designing real-time decision support for trauma resuscitations. Academic Emergency Medicine 22, 9 (2015), 1076–1084.
DOI: Google ScholarCross Ref - [95] . 2019. Unremarkable AI: Fitting intelligent decision support into critical, clinical decision-making processes. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11.
DOI: Google ScholarDigital Library - [96] . 2016. Investigating the heart pump implant decision process: Opportunities for decision support tools to help. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. New York, NY, 4477–4488. Retrieved January 19, 2021 from Google ScholarDigital Library
- [97] . 2021. Data work and decision making in emergency medical services: A distributed cognition perspective. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2. 356:1-356:32.
DOI: Google ScholarDigital Library - [98] . 2022. Hands-Free electronic documentation in emergency care work through smart glasses. In Proceedings of the Information for a Better World: Shaping the Global Future 314–331.
DOI: Google ScholarDigital Library - [99] . 2010. Computerization and information assembling process: Nursing work and CPOE adoption. In Proceedings of the 1st ACM International Health Informatics Symposium. 36–45.
DOI: Google ScholarDigital Library - [100] . 2011. CPOE workarounds, boundary objects, and assemblages. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 3353–3362.
DOI: Google ScholarDigital Library
Index Terms
- Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design Guidelines
Recommendations
”Pop-Up Alerts are the Bane of My Existence”: Designing Alerts for Cognitive Aids Used in Time-Critical Medical Settings
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsDecision support alerts have the potential to assist clinicians in determining appropriate interventions for critically injured patients. The design of these alerts is critical because it can impact their adoption and effectiveness. In this late-...
Designing Interactive Alerts to Improve Recognition of Critical Events in Medical Emergencies
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems ConferenceVital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-...
Ability of machine-learning based clinical decision support system to reduce alert fatigue, wrong-drug errors, and alert users about look alike, sound alike medication
Highlights- This study sought to evaluate the overall performance of a machine learning-based CDSS (MedGuard) for triggering clinically relevant alerts, acceptance rate, and to intercept inappropriate drug errors as well as LASA drug errors.
- An ...
Abstract Background and objectiveThe overall benefits of using clinical decision support systems (CDSSs) can be restrained if physicians inadvertently ignore clinically useful alerts due to “alert fatigue” caused by an excessive number of clinically ...
Comments