Skip to main content
SpringerLink
Log in

Hands-Free Electronic Documentation in Emergency Care Work Through Smart Glasses

  • Conference paper
  • First Online:
Information for a Better World: Shaping the Global Future (iConference 2022)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13193))

Included in the following conference series:

Abstract

As U.S. healthcare system moves towards digitization, Electronic Health Records (EHRs) are increasingly adopted by medical providers. However, EHR documentation is not only time-consuming but also difficult to complete in real-time, leading to delayed, missing, or erroneous data entry. This challenge is more evident in time-critical and hands-busy clinical domains, such as Emergency Medical Services (EMS). In recent years, smart glasses have gained momentum in supporting various aspects of clinical care. However, limited research has examined the potential of smart glasses in automating electronic documentation during fast-paced medical work. In this paper, we report the design, development, and preliminary evaluations of a novel system combining smart glasses and EHRs and leveraging natural language processing (NLP) techniques to enable hands-free, real-time documentation in the context of EMS care. Although optimization is needed, our system prototype represents a substantive departure from the status quo in the documentation technology for emergency care providers, and has a high potential to enable real-time documentation while accounting for care providers’ cognitive and physical constraints imposed by the time-critical medical environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.crunchfish.com/.

References

  1. Downing, N.L., Bates, D.W., Longhurst, C.A.: Physician burnout in the electronic health record era: are we ignoring the real cause? American College of Physicians (2018)

    Google Scholar 

  2. Zulman, D.M., Shah, N.H., Verghese, A.: Evolutionary pressures on the electronic health record: caring for complexity. JAMA 316(9), 923–924 (2016)

    Article  Google Scholar 

  3. Tollefsen, W.W., et al.: iRevive: a pre-hospital database system for emergency medical services. Int. J. Healthc. Technol. Manag. 6(4–6), 454–469 (2005)

    Article  Google Scholar 

  4. Hertzum, M., Manikas, M.I., á Torkilsheyggi, A.: Grappling with the future: the messiness of pilot implementation in information systems design. Health Inform. J. 25(2), 372–388 (2019)

    Google Scholar 

  5. Sarcevic, A., Burd, R.S.: Information handover in time-critical work. In: Proceedings of the ACM 2009 International Conference on Supporting Group Work, pp. 301–310 (2009)

    Google Scholar 

  6. Holzman, T.G.: Computer-human interface solutions for emergency medical care. Interactions 6(3), 13–24 (1999)

    Article  Google Scholar 

  7. Laudermilch, D.J., Schiff, M.A., Nathens, A.B., Rosengart, M.R.: Lack of emergency medical services documentation is associated with poor patient outcomes: a validation of audit filters for prehospital trauma care. J. Am. Coll. Surg. 210(2), 220–227 (2010)

    Article  Google Scholar 

  8. Zhang, Z., et al.: Data work and decision making in emergency medical services: a distributed cognition perspective. Proc. ACM Human-Comput. Interact. 5(CSCW2), 1–32 (2021)

    Article  Google Scholar 

  9. Momen, K.S.: Identifying nursing activities to estimate the risk of cross-contamination. University of Toronto, Canada (2012)

    Google Scholar 

  10. Pilerot, O., Maurin Söderholm, H.: A conceptual framework for investigating documentary practices in prehospital emergency care. Inf. Res. 24(4), colis1931 (2019)

    Google Scholar 

  11. Aldaz, G., et al.: Hands-free image capture, data tagging and transfer using Google Glass: a pilot study for improved wound care management. PloS One 10(4), e0121179 (2015)

    Google Scholar 

  12. Jonas, S., Hannig, A., Spreckelsen, C., Deserno, T.M.: Wearable technology as a booster of clinical care. In Medical Imaging 2014: PACS and Imaging Informatics: Next Generation and Innovations, pp. 90390F. International Society for Optics and Photonics, Washington, USA (2014)

    Google Scholar 

  13. Mitrasinovic, S., et al.: Clinical and surgical applications of smart glasses. Technol. Health Care 23(4), 381–401 (2015)

    Article  Google Scholar 

  14. Lee, J.-S., Tsai, C.-T., Pen, C.-H., Lu, H.-C.: A real time collaboration system for teleradiology consultation. Int. J. Med. Inform. 72(1–3), 73–79 (2003)

    Article  Google Scholar 

  15. Noorian, A.R., et al.: Use of wearable technology in remote evaluation of acute stroke patients: feasibility and reliability of a Google Glass-based device. J. Stroke Cerebrovas. Dis. 28(10), 104258 (2019)

    Google Scholar 

  16. Cicero, M.X., et al.: Do you see what I see? Insights from using google glass for disaster telemedicine triage. Prehosp. Disaster Med. 30(1), 4 (2015)

    Article  Google Scholar 

  17. Broach, J., et al.: Usability and reliability of smart glasses for secondary triage during mass casualty incidents. In Proceedings of the Annual Hawaii International Conference on System Sciences, p. 1416. IEEE, New York (2018)

    Google Scholar 

  18. Schaer, R., Melly, T., Muller, H., Widmer, A.: Using smart glasses in medical emergency situations, a qualitative pilot study. In: 2016 IEEE Wireless Health (WH), pp. 1–5. IEEE, New York, USA (2016)

    Google Scholar 

  19. Schlosser, P., Matthews, B., Salisbury, I., Sanderson, P., Hayes, S.: Head-Worn displays for emergency medical services staff: properties of prehospital work, use cases, and design considerations. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14. ACM, New York (2021)

    Google Scholar 

  20. Saarinen, K., Aho, M.: Does the implementation of a clinical information system decrease the time intensive care nurses spend on documentation of care? Acta Anaesthesiol. Scand. 49(1), 62–65 (2005)

    Article  Google Scholar 

  21. Davidson, S.J., Zwemer, F.L., Nathanson, L.A., Sable, K.N., Khan, A.N.: Where’s the beef? The promise and the reality of clinical documentation. Acad. Emerg. Med. 11(11), 1127–1134 (2004)

    Article  Google Scholar 

  22. Chen, Y.: Documenting transitional information in EMR. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1787–1796. ACM, New York (2010)

    Google Scholar 

  23. Jagannath, S., Sarcevic, A., Young, V., Myers, S.: Temporal rhythms and patterns of electronic documentation in time-critical medical work. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13. ACM, New York (2019)

    Google Scholar 

  24. Sarcevic, A., Ferraro, N.: On the use of electronic documentation systems in fast-paced, time-critical medical settings. Interact. Comput. 29(2), 203–219 (2017)

    Google Scholar 

  25. Bossen, C., Pine, K.H., Cabitza, F., Ellingsen, G., Piras, E.M.: Data work in healthcare: an Introduction. SAGE Publications Sage UK, London (2019)

    Google Scholar 

  26. Sarcevic, A.: “Who's scribing?” documenting patient encounter during trauma resuscitation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1899–1908. ACM, New York (2010)

    Google Scholar 

  27. Meystre, S., Haug, P.J.: Natural language processing to extract medical problems from electronic clinical documents: performance evaluation. J. Biomed. Inform. 39(6), 589–599 (2006)

    Article  Google Scholar 

  28. Rosales, R., Farooq, F., Krishnapuram, B., Yu, S., Fung, G.: Automated identification of medical concepts and assertions in medical text. In: AMIA Annual Symposium Proceedings, p. 682. American Medical Informatics Association, Maryland (2010)

    Google Scholar 

  29. Li, Q., Wu, Y.-F.B.: Identifying important concepts from medical documents. J. Biomed. Inform. 39(6), 668–679 (2006)

    Article  Google Scholar 

  30. Preum, S.M., Shu, S., Alemzadeh, H., Stankovic, J.A.: Emscontext: EMS protocol-driven concept extraction for cognitive assistance in emergency response. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13350–13355. AAAI Press, California (2020)

    Google Scholar 

  31. Rahman, M.A., Preum, S.M., Williams, R., Alemzadeh, H., Stankovic, J.A.: GRACE: generating summary reports automatically for cognitive assistance in emergency response. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 13356–13362. AAAI Press, California (2020)

    Google Scholar 

  32. Noma, H., Ohmura, A., Kuwahara, N., Kogure, K.: Wearable sensors for auto-event-recording on medical nursing-user study of ergonomic design. In: Eighth International Symposium on Wearable Computers, pp. 8–15. IEEE, New York (2004)

    Google Scholar 

  33. Klein, G.O., Singh, K., von Heideken, J.: Smart glasses—a new tool in medicine. Stud. Health Technol. Inform. 216, 901 (2015)

    Google Scholar 

  34. Aungst, T., Lewis, T.: Potential uses of wearable technology in medicine: lessons learnt from Google Glass. Int. J. Clin. Pract. 69(10), 1179–1183 (2015)

    Article  Google Scholar 

  35. Yu, J., Ferniany, W., Guthrie, B., Parekh, S.G., Ponce, B.: Lessons learned from google glass: telemedical spark or unfulfilled promise? Surg. Innov. 23(2), 156–165 (2016)

    Article  Google Scholar 

  36. Klinker, K., Wiesche, M., Krcmar, H.: Digital transformation in health care: augmented reality for hands-free service innovation. Inf. Syst. Front. 22, 1–13 (2019)

    Google Scholar 

  37. Holton, J.A.: The coding process and its challenges. The Sage Handbook of Grounded Theory 3, 265–289 (2007)

    Article  Google Scholar 

  38. Hartson, R., Pyla, P.S.: The UX Book: Process and Guidelines for Ensuring a Quality User Experience. Elsevier (2012)

    Google Scholar 

  39. Zhang, Z., Sarcevic, A., Bossen, C.: Constructing common information spaces across distributed emergency medical teams. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 934–947. ACM, New York (2017)

    Google Scholar 

  40. Zhang, Z., Sarcevic, A., Burd, R.S.: Supporting information use and retention of pre-hospital information during trauma resuscitation: a qualitative study of pre-hospital communications and information needs. In: AMIA Annual Symposium Proceedings, p. 1579. American Medical Informatics Association, Maryland (2013)

    Google Scholar 

  41. Aronson, A.R.: MetaMap: mapping text to the UMLS metathesaurus. Bethesda, MD: NLM, NIH, DHHS 1, 26 (2006)

    Google Scholar 

  42. Sager, N., Lyman, M., Bucknall, C., Nhan, N., Tick, L.J.: Natural language processing and the representation of clinical data. J. Am. Med. Inform. Assoc. 1(2), 142–160 (1994)

    Article  Google Scholar 

  43. Kormilitzin, A., Vaci, N., Liu, Q., Nevado-Holgado, A.: Med7: a transferable clinical natural language processing model for electronic health records. Artif. Intell. Med. 102086 (2021)

    Google Scholar 

  44. Manning, C.D., et al.: The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics, pp. 55–60. Association for Computational Linguistics, Pennsylvania (2014)

    Google Scholar 

  45. Chinchor, N., Sundheim, B.M.: MUC-5 evaluation metrics. In: Fifth Message Understanding Conference (MUC-5): Proceedings of a Conference Held in Baltimore, Maryland (1993)

    Google Scholar 

  46. Zeng, X., Li, Y., Zhai, Y., Zhang, Y.: Counterfactual generator: a weakly-supervised method for named entity recognition. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7270–7280. Association for Computational Linguistics, Pennsylvania (2020)

    Google Scholar 

  47. Thieu, T., et al.: A comprehensive study of mobility functioning information in clinical notes: entity hierarchy, corpus annotation, and sequence labeling. Int. J. Med. Inform. 147, 104351 (2021)

    Google Scholar 

  48. Sen, S., Ekbal, A., Bhattacharyya, P.: Parallel corpus filtering based on fuzzy string matching. In: Proceedings of the Fourth Conference on Machine Translation (Volume 3: Shared Task Papers, Day 2), pp. 289–293. Association for Computational Linguistics, Pennsylvania (2019)

    Google Scholar 

  49. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phys. Doklady 10(8), 707–710 (1966)

    MathSciNet  Google Scholar 

  50. Romare, C., Hass, U., Skär, L.: Healthcare professionals’ views of smart glasses in intensive care: a qualitative study. Intensive Crit. Care Nurs. 45, 66–71 (2018)

    Article  Google Scholar 

  51. Maruri, H.A.C., et al.: V-Speech: noise-robust speech capturing glasses using vibration sensors. Proc. ACM Interact. Mob. Wearable Ubiquit. Technol. 2(4), 1–23 (2018)

    Article  Google Scholar 

  52. Dawson, D.E.: National emergency medical services information system (NEMSIS). Prehosp. Emerg. Care 10(3), 314–316 (2006)

    Article  Google Scholar 

  53. Li, J., Sun, A., Han, J., Li, C.: A survey on deep learning for named entity recognition. arXiv preprint arXiv:1812.09449 (2018)

  54. Gehrmann, S., et al.: Comparing deep learning and concept extraction based methods for patient phenotyping from clinical narratives. PloS One 13(2), e0192360 (2018)

    Google Scholar 

Download references

Acknowledgement

This work was supported by National Science Foundation (NSF) Award #1948292. We would like to thank the ESO company for sharing their EHR system with us so we can create a hypothetical EHR system based on their design. We also want to thank our research participants for providing valuable feedback.

Author information

Authors and Affiliations

  1. Pace University, New York, NY, 10038, USA

    Zhan Zhang & Richard Harris

  2. Indiana University-Purdue University Indianapolis, Indianapolis, IN, 46202, USA

    Xiao Luo & Susanna George

  3. Interfaith Medical Center, New York, NY, 11213, USA

    Jack Finkelstein

Authors
  1. Zhan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Richard Harris
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Susanna George
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jack Finkelstein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhan Zhang .

Editor information

Editors and Affiliations

  1. Humboldt-Universität zu Berlin, Berlin, Germany

    Malte Smits

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, Z., Luo, X., Harris, R., George, S., Finkelstein, J. (2022). Hands-Free Electronic Documentation in Emergency Care Work Through Smart Glasses. In: Smits, M. (eds) Information for a Better World: Shaping the Global Future. iConference 2022. Lecture Notes in Computer Science(), vol 13193. Springer, Cham. https://doi.org/10.1007/978-3-030-96960-8_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-96960-8_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-96959-2

  • Online ISBN: 978-3-030-96960-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation