Abstract
Current telemedicine systems for remote medical consultation are based on decades old video-conferencing technology. Their primary role is to deliver video and voice communication between medical providers and to transmit vital signs of the patient. This technology, however, does not provide the expert physician with the same hands-on experience as when examining a patient in person. Virtual and Augmented Reality (VR and AR) on the other hand have the capacity to enhance the experience and communication between healthcare professionals in geographically distributed locations. By transmitting RGB+D video of the patient, the expert physician can interact with this real-time 3D representation in novel ways. Furthermore, the use of AR technology at the patient side has potential to improve communication by providing clear visual instructions to the caregiver. In this paper, we propose a framework for 3D real-time communication that combines interaction via VR and AR. We demonstrate the capabilities of our framework on a prototype system consisting of a depth camera, projector and 3D display. The system is used to analyze the network performance and data transmission quality of the multimodal streaming in a remote scenario.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
RGB frames were reduced to 960 × 540 resolution.
References
Goldzweig, C.L., Towfigh, A., Maglione, M., Shekelle, P.G.: Costs and benefits of health information technology: new trends from the literature. Health Aff. (Millwood) 28, w282–w293 (2009)
Kvedar, J., Coye, M.J., Everett, W.: Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Health Aff. (Millwood) 33, 194–199 (2014)
Flodgren, G., Rachas, A., Farmer, A.J., Inzitari, M., Shepperd, S.: Interactive telemedicine: effects on professional practice and health care outcomes. The Cochrane Library (2015)
Augestad, K.M., Lindsetmo, R.O.: Overcoming distance: video-conferencing as a clinical and educational tool among surgeons. World J. Surg. 33, 1356–1365 (2009)
Markarian, G., Mihaylova, L., Tsitserov, D.V., Zvikhachevskaya, A.: Video distribution techniques over WiMAX networks for m-health applications. IEEE Trans. Inf. Technol. Biomed. 16, 24–30 (2012)
Pande, A., Mohapatra, P., Zambreno, J.: Securing multimedia content using joint compression and encryption. IEEE MultiMedia 20, 50–61 (2013)
Latifi, R., Weinstein, R., Porter, J., Ziemba, M., Judkins, D., Ridings, D., Nassi, R., Valenzuela, T., Holcomb, M., Leyva, F.: Telemedicine and telepresence for trauma and emergency care management. Scand. J. Surg. 96, 281–289 (2007)
Boniface, K.S., Shokoohi, H., Smith, E.R., Scantlebury, K.: Tele-ultrasound and paramedics: real-time remote physician guidance of the focused assessment with sonography for trauma examination. Am. J. Emerg. Med. 29, 477–481 (2011)
Söderholm, H.M., Sonnenwald, D.H., Cairns, B., Manning, J.E., Welch, G.F., Fuchs, H.: The potential impact of 3D telepresence technology on task performance in emergency trauma care. In: Proceedings of the 2007 International ACM Conference on Supporting Group Work, pp. 79–88 (2007)
Garcia, P.: Telemedicine for the battlefield: present and future technologies. In: Rosen, J., Hannaford, B., Satava, R.M. (eds.) Surgical Robotics, pp. 33–68. Springer, Heidelberg (2011)
Irizarry, D., Wadman, M.C., Bernhagen, M.A., Miljkovic, N., Boedeker, B.H.: Using the battlefield telemedicine system (BTS) to train deployed medical personnel in complicated medical tasks-a proof of concept. In: MMVR, pp. 215–217 (2012)
Ling, G.S., Rhee, P., Ecklund, J.M.: Surgical innovations arising from the Iraq and Afghanistan wars. Annu. Rev. Med. 61, 457–468 (2010)
Satava, R.M.: Virtual reality and telepresence for military medicine. Comput. Biol. Med. 25, 229–236 (1995)
Welch, G., Fuchs, H., Cairns, B., Mayer-Patel, K., Sonnenwald, D.H., Ilie, A., Noland, M., Noel, V., Yang, H.: Improving, expanding and extending 3D telepresence. In: International Workshop on Advanced Information Processing for Ubiquitous Networks, 15th International Conference on Artifical Reality and Telexistence (ICAT 2005) (2005)
Jang-Jaccard, J., Nepal, S., Celler, B., Yan, B.: WebRTC-based video conferencing service for telehealth. Computing 98, 169–193 (2016)
Protocol buffers (2016). https://developers.google.com/protocol-buffers
Maeda, K.: Performance evaluation of object serialization libraries in XML, JSON and binary formats. In: 2012 Second International Conference on Digital Information and Communication Technology and Its Applications (DICTAP), pp. 177–182. IEEE (2012)
Kinect for windows (2016). https://developer.microsoft.com/en-us/windows/kinect
Kinect projector toolkit (2016). https://github.com/genekogan/KinectProjectorToolkit
Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: 20th International Conference on Pattern Recognition (ICPR), pp. 2366–2369. IEEE (2010)
Acknowledgements
This work was partially supported by the following sources: National Science Foundation (NSF) grant #1427260, Office of Naval Research (ONR) grant #N00014-09-1-0230, and Siemens Fellowship (#20150859).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Anton, D., Kurillo, G., Yang, A.Y., Bajcsy, R. (2017). Augmented Telemedicine Platform for Real-Time Remote Medical Consultation. In: Amsaleg, L., Guðmundsson, G., Gurrin, C., Jónsson, B., Satoh, S. (eds) MultiMedia Modeling. MMM 2017. Lecture Notes in Computer Science(), vol 10132. Springer, Cham. https://doi.org/10.1007/978-3-319-51811-4_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-51811-4_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-51810-7
Online ISBN: 978-3-319-51811-4
eBook Packages: Computer ScienceComputer Science (R0)