skip to main content
10.1145/1152399.1152418acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicatConference Proceedingsconference-collections
Article

Visual support for medical communication by using projector-based augmented reality and thermal markers

Published: 05 December 2005 Publication History

Abstract

This paper presents a system that allows patients and physicians to experience better communication during medical consultations using Augmented Reality (AR) technology. The AR system can superimpose augmentations (i.e., human body components) onto the real patient's body, and such annotated information serves to form the cornerstone for collaborative work between the two actors. We focus on the advantages of projector-based technology and ARToolKit. Our technique, based on thermal markers (i.e., using human body temperature as a source of information) is used for tracking the location of pain in the patient through the projected augmentations. The second aim of using thermal markers is to protect the patient's privacy. The required calibration method between thermal-camera and projector is also presented. The system's feasibility is demonstrated through development of a complete application.

References

[1]
{{Azu97} R. T. Azuma: "A Survey of Augmented Reality." Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355--385.
[2]
{Ash03} M. Ashdown and P. Robinson: "Experiences Implementing and Using Personal Projected Displays." In IEEE International Workshop on Projector-Camera Systems, (Procams 2003).
[3]
{Ash03a} M. Ashdown and R. Sukthankar. "Robust calibration of camera-projector system for multi-planar displays." Technical Report HP Labs HPL-2003-24, January 2003.
[4]
{Baj92} M. Bajura, H. Fuchs, and R. Ohbuchi: "Merging Virtual Objects with the Real World: Seeing Ultrasound Imagery within the Patient." In Proceedings of SIGGRAPH '92 (Chicago, IL, July 26--31, 1992). In Computer Graphics 26, #2 (July 1992): 203--210.
[5]
{Bil99} M. Billinghurst, H. Kato, S. Weghorst, and T. A. Furness. "A mixed-reality 3D conferencing application." Technical report, Human Interface Technology Laboratory, University of Washington, 1999.
[6]
{Fis04} J. Fischer, M. Neff, D. Freudenstein, and D. Bartz. "Medical Augmented Reality based on Commercial Image Guided Surgery." In Eurographics Symposium on Virtual Environments (EGVE), June 2004.
[7]
{Fuc98} H. Fuchs, M. Livingston, R. Raskar, D. Colucci, K. Keller, A. State, J. Crawford, P. Rademacher, S. Drake and A. Mayer: "Augmented reality visualization for laparoscopic surgery." In Proceedings of Medical Image Computing and Computer-Assisted Intervention '98 (MICCAI '98), volume 1496 of LNCS, pages 934--943, October, 1998.
[8]
{Goe03} G. Goebbels, K. Troche, M. Braun, A. Ivanovic, A. Grab, K. von Lübtow, H. F. Zeilhofer, R. Sader, F. Thieringer, K. Albrecht, K. Praxmarer, E. Keeve, N. Hanssen, Z. Krol and F. Hasenbrink: "Development of an Augmented Reality System for intra-operative navigation in maxillo-facial surgery." In 2. Jahrestagung des Gesellschaft für Computer-und Roboterassistierte Chirurgie (CURAC), November 2003.
[9]
{Gra03} R. Grasset and J.-D. Gascuel. "Réalité Augmentée et Environnement Collaboratif: Un Tour d'Horizon." In: AFIG '03 (Actes des 16èmes journéees de l'AFIG), December 2003.
[10]
{Ina00} M. Inami, N. Kawakami, D. Sekiguchi, Y. Ya-nagida, T. Maeda and S. Tachi, Visuo-Haptic Display Using Head-Mounted Projector, Proceedings of IEEE Virtual Reality 2000, pp.233--240, 2000.
[11]
{Joh05} K. Johnsen, R. Dickerson, A. Raij, B. Lok, J. Jackson, M. Shin, J. Hernandez, A. Stevens and D. S. Lind: "Experiences in Using Immersive Virtual Characters to Educate Medical Communication Skills." In Proceedings of IEEE Virtual Reality 2005 (VR 2005).
[12]
{Kit05} I. Kitahara, M. Ito, H. Noma, K. Kogure and N. Hagita, "Thermal-ID: A Personal Identification Method Using Body Temperature." Adjunct Proc. of The 3rd International Conference on Pervasive Computing (Pervasive 2005) Advances in Pervasive Computing, pp. 69--72.
[13]
{Mal02} P. Malbezin, W. Piekarski and B. Thomas: "Measuring ARToolkit accuracy in long-distance tracking experiments." In: 1st Int'l AR Toolkit Workshop. (2002).
[14]
{QR} http://www.denso-wave.com/qrcode/index-e.html
[15]
{Ras01} R. Raskar, G. Welch, K. L. Low and D. Bandyopadhyay: "Shader Lamps: Animating Real Objects with Image Based Illumination," Eurographics Workshop on Rendering, June 2001.
[16]
{Sie04} T. Sielhorst, T. Obst, R. Burgkart, R. Riener and N. Navab. "An augmented reality delivery simulator for medical training." In International Workshop on Augmented Environments for Medical Imaging - MICCAI Satellite Workshop, 2004.
[17]
{Stat96} A. State, M. Livingston, G. Hirota, W. Garrett, M. Whitton, H. Fuchs and E. Pisano "Technologies for Augmented-Reality Systems: realizing Ultrasound-Guided Needle Biopsies." In Proceedings of SIGGRAPH '96 (New Orleans, LA, August 4--9, 1996). In Computer Graphics Proceedings, Annual Conference Series, 1996, ACM SIGGRAPH, 439--446.
[18]
{Tac03} S. Tachi, "Telexistence and Retro-reflective Projection Technology (RPT)," Proceedings of the 5th Virtual Reality International Conference (VRIC2003) pp. 69/1--69/9, Laval Virtual 2003, France, May 13--18, 2003.
[19]
{Tak02} M. Takemura and Y. Ohta, "Diminishing Head-Mounted Display for Shared Mixed Reality," IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2002), pp. 149--156. Oct. 2002.

Cited By

View all
  • (2024)HelpingHands: Developing Health Procedure Skills Through Augmented Reality-Based Remote Guidance2024 28th International Conference Information Visualisation (IV)10.1109/IV64223.2024.00065(338-343)Online publication date: 22-Jul-2024
  • (2022)In-situ monitoring of Material Extrusion processes via thermal videoimaging with application to Big Area Additive Manufacturing (BAAM)Additive Manufacturing10.1016/j.addma.2022.10299558(102995)Online publication date: Oct-2022
  • (2021)Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering LawApplied Sciences10.3390/app1110450511:10(4505)Online publication date: 14-May-2021
  • Show More Cited By
  1. Visual support for medical communication by using projector-based augmented reality and thermal markers

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      ICAT '05: Proceedings of the 2005 international conference on Augmented tele-existence
      December 2005
      291 pages
      ISBN:0473106574
      DOI:10.1145/1152399
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 05 December 2005

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. augmented reality
      2. collaborative work
      3. health science
      4. medical communication
      5. thermal marker

      Qualifiers

      • Article

      Conference

      ICAT05
      ICAT05: The International Conference on Augmented Tele-Existence
      December 5 - 8, 2005
      Christchurch, New Zealand

      Acceptance Rates

      ICAT '05 Paper Acceptance Rate 48 of 48 submissions, 100%;
      Overall Acceptance Rate 48 of 48 submissions, 100%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)6
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 19 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)HelpingHands: Developing Health Procedure Skills Through Augmented Reality-Based Remote Guidance2024 28th International Conference Information Visualisation (IV)10.1109/IV64223.2024.00065(338-343)Online publication date: 22-Jul-2024
      • (2022)In-situ monitoring of Material Extrusion processes via thermal videoimaging with application to Big Area Additive Manufacturing (BAAM)Additive Manufacturing10.1016/j.addma.2022.10299558(102995)Online publication date: Oct-2022
      • (2021)Development of a Drawing Application for Communication Support during Endoscopic Surgery and Its Evaluation Using Steering LawApplied Sciences10.3390/app1110450511:10(4505)Online publication date: 14-May-2021
      • (2017)VisMerge: Light Adaptive Vision Augmentation via Spectral and Temporal Fusion of Non-visible Light2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2017.19(22-31)Online publication date: Oct-2017
      • (2015)Mobile 3D augmented-reality system for ultrasound applications2015 IEEE International Ultrasonics Symposium (IUS)10.1109/ULTSYM.2015.0488(1-4)Online publication date: Oct-2015
      • (2015)Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared textureVirtual Reality10.1007/s10055-014-0256-y19:1(45-56)Online publication date: 1-Mar-2015
      • (2015)Patient Adapted Augmented Reality System for Real-Time Echocardiographic ApplicationsAugmented Environments for Computer-Assisted Interventions10.1007/978-3-319-24601-7_15(145-154)Online publication date: 10-Dec-2015
      • (2014)Augmented reality based tools for echocardiographic acquisitions2014 IEEE International Ultrasonics Symposium10.1109/ULTSYM.2014.0171(695-698)Online publication date: Sep-2014
      • (2014)A Personalized Learning System with an AR Augmented Reality Browser for Ecosystem FieldworkProceedings of the 2014 IEEE 28th International Conference on Advanced Information Networking and Applications10.1109/AINA.2014.16(89-97)Online publication date: 13-May-2014
      • (2012)HeartPadProceedings of the Workshop at SIGGRAPH Asia10.1145/2425296.2425326(169-176)Online publication date: 26-Nov-2012

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media