Abstract
During surgery, the patient’s vital signs and the field of endoscopic view are displayed on multiple screens. As a result, both surgeons’ and anesthesiologists’ visual attention (VA) is crucial. Moreover, the distribution of said VA and the acquisition of specific cues might directly impact patient outcomes.
Recent research utilizes portable, head-mounted eye-tracking devices to gather precise and comprehensive information. Nevertheless, these technologies are not feasible for enduring data acquisition in an operating room (OR) environment. This is particularly the case during medical emergencies.
This study presents an alternative methodology: a webcam-based gaze target prediction model. Such an approach may provide continuous visual behavioral data with minimal interference to the physicians’ workflow in the OR. The proposed end-to-end framework is suitable for both standard and emergency surgeries.
In the future, such a platform may serve as a crucial component of context-aware assistive technologies in the OR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chetwood, A.S.A., et al.: Collaborative eye tracking: a potential training tool in laparoscopic surgery. Surgical Endoscopy 26(7), 2003–9 (2012). https://doi.org/10.1007/s00464-011-2143-x. http://www.ncbi.nlm.nih.gov/pubmed/22258302
Chong, E., Wang, Y., Ruiz, N., Rehg, J.M.: Detecting attended visual targets in video. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5395–5405 (2020). https://doi.org/10.1109/CVPR42600.2020.00544. https://github.com/ejcgt/attention-target-detection
Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: Retinaface: single-shot multi-level face localisation in the wild. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5202–5211 (2020). https://doi.org/10.1109/CVPR42600.2020.00525
Fang, Y., et al.: Dual attention guided gaze target detection in the wild. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 11385–11394 (2021). https://doi.org/10.1109/CVPR46437.2021.01123
Feng, Z.H., Kittler, J., Awais, M., Huber, P., Wu, X.J.: Wing loss for robust facial landmark localisation with convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2235–2245 (2018)
Gil, A.M., Birdi, S., Kishibe, T., Grantcharov, T.P.: Eye tracking use in surgical research: a systematic review. J. Surg. Res. 279, 774–787 (2022). https://doi.org/10.1016/j.jss.2022.05.024. https://linkinghub.elsevier.com/retrieve/pii/S0022480422003419
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Qi, D., Tan, W., Yao, Q., Liu, J.: YOLO5Face: Why Reinventing a Face Detector (2021). https://www.github.com/deepcam-cn/yolov5-face. http://arxiv.org/abs/2105.12931
Ramasinghe, S., Athuraliya, C.D., Khan, S.H.: A context-aware capsule network for multi-label classification. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11131, pp. 546–554. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11015-4_40
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 779–788 (2016). https://doi.org/10.1109/CVPR.2016.91. http://pjreddie.com/yolo/
Roche, T.R., et al.: Anesthesia personnel’s visual attention regarding patient monitoring in simulated non-critical and critical situations, an eye-tracking study. BMC Anesthesiology 22(1) (2022). https://doi.org/10.1186/s12871-022-01705-6. https://doi.org/10.1186/s12871-022-01705-6
Schulz, C.M., et al.: Visual attention of anaesthetists during simulated critical incidents. British J. Anaesthesia 106(6), 807–813 (2011). https://doi.org/10.1093/bja/aer087. www.anvil-software.de
Szulewski, A., Egan, R., Gegenfurtner, A., Howes, D., Dashi, G., McGraw, N.C., Hall, A.K., Dagnone, D., Van Merrienboer, J.J.: A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance. Canadian J. Emergency Med. 21(1), 129–137 (2019). https://doi.org/10.1017/cem.2018.391
Tien, T., Pucher, P.H., Sodergren, M.H., Sriskandarajah, K., Yang, G.Z., Darzi, A.: Eye tracking for skills assessment and training: a systematic review. J. Surgical Res. 191(1), 169–178 (2014). https://doi.org/10.1016/j.jss.2014.04.032. https://linkinghub.elsevier.com/retrieve/pii/S0022480414004326
Tomas, H., et al.: GOO: a dataset for gaze object prediction in retail environments. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 3119–3127 (2021). https://doi.org/10.1109/CVPRW53098.2021.00349. https://github.com/upeee/GOO-GAZE2021
Wagner, M., et al.: Video-based reflection on neonatal interventions during COVID-19 using eye-tracking glasses: an observational study. Archives of disease in childhood. Fetal Neonatal Edition 107(2), 156–160 (2022). https://doi.org/10.1136/archdischild-2021-321806. https://fn.bmj.com/content/107/2/156 https://fn.bmj.com/content/107/2/156.abstract
Wang, B., Hu, T., Li, B., Chen, X., Zhang, Z.: GaTector: a unified framework for gaze object prediction. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19566–19575. IEEE, June 2022. https://doi.org/10.1109/CVPR52688.2022.01898. https://ieeexplore.ieee.org/document/9879784/
White, M.R., et al.: Getting inside the expert’s head: an analysis of physician cognitive processes during trauma resuscitations. Ann. Emerg. Med. 72(3), 289–298 (2018). https://doi.org/10.1016/j.annemergmed.2018.03.005
Xiong, Y., Zhu, K., Lin, D., Tang, X.: Recognize complex events from static images by fusing deep channels. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. vol. 07–12-June, pp. 1600–1609 (2015). https://doi.org/10.1109/CVPR.2015.7298768
Yang, S., Luo, P., Loy, C.C., Tang, X.: WIDER FACE: A face detection benchmark. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. vol. 2016-Decem, pp. 5525–5533 (2016). https://doi.org/10.1109/CVPR.2016.596, http://mmlab.ie.cuhk.edu.hk/projects/
Zhang, D., Wang, B., Wang, G., Zhang, Q., Zhang, J., Han, J., You, Z.: Onfocus detection: identifying individual-camera eye contact from unconstrained images. SCIENCE CHINA Inf. Sci. 65(6), 1–12 (2022). https://doi.org/10.1007/s11432-020-3181-9
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gershov, S., Mahameed, F., Raz, A., Laufer, S. (2024). More Than Meets the Eye: Physicians’ Visual Attention in the Operating Room. In: Wu, S., Shabestari, B., Xing, L. (eds) Applications of Medical Artificial Intelligence. AMAI 2023. Lecture Notes in Computer Science, vol 14313. Springer, Cham. https://doi.org/10.1007/978-3-031-47076-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-47076-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47075-2
Online ISBN: 978-3-031-47076-9
eBook Packages: Computer ScienceComputer Science (R0)
