Joseph T. Samosky
ABSTRACT
BodyExplorerAR is a system designed to enhance a learner's ability to explore anatomy, physiology and clinical interventions though naturalistic interaction with an augmented reality enhanced full-body mannequin simulator. We are developing a platform that integrates projective AR and multi-modal sensor inputs. A user can use an IR pen to open, resize and move viewports providing windows into the body that can display dynamic anatomy. The user can point to an organ and display additional information such as graphs of physiological parameters or heart sounds. Custom sensing systems provide natural interactions with common medical devices such as syringes, breathing tubes and catheters. A user can open a window displaying the beating heart in situ, display an electrocardiogram (ECG), then inject drugs and see and hear changes in heart rate. Our goal is an engaging experience that empowers a learner to create customized, media-rich explorations revealing the internal consequences of external actions.
- Bandyopadhyay, D., Raskar, R. and Fuchs, H., Dynamic Shader Lamps: Painting on Movable Objects. In The Second IEEE and ACM International Symposium on Augmented Reality, (2001), 207--216. Google Scholar
Digital Library
- Bichlmeier, C., Wimme, F., Heining, S. M., et al., Contextual Anatomic Mimesis: Hybrid In-Situ Visualization Method for Improving Multi-Sensory Depth Perception in Medical Augmented Reality. In Proceedings of ISMAR 2007, 129--138. Google ScholarDigital Library
- Blackwell, M., Nikou, C., DiGioia, A. M., et al. An Image Overlay system for medical data visualization. Medical Image Analysis, 2000, 4 (1). 67--72.Google ScholarCross Ref
- Goto, T., Kondo, D., Kouno, M., et al. Development of medical educational platform. Proceedings of 10th International Conference on Virtual Systems and Multimedia (VSMM 2004), 2004. 686--694.Google Scholar
- Jalbert, G. Lay Figure. U. S. patent number 1,653,180. December 20, 1927.Google Scholar
- Kondo, D., Kijima, R. and Takahashi, Y. Dynamic anatomical model for medical education using free form projection display. Proceedings of 13th International Conference on Virtual Systems and Multimedia (VSMM 2007), 2007. 142--149.Google Scholar
- Lee, J. C. Hacking the Nintendo Wii Remote. IEEE Pervasive Computing, 2008, 7 (3). 39--45. Google ScholarDigital Library
- Lee, J. C. Low-cost Multi-Point Interactive Whiteboards Using the Wiimote. http://johnnylee.net/projects/wii Accessed July 14, 2011.Google Scholar
- Ni, T., Karlson, A. K. and Wigdor, D., AnatOnMe: Facilitating doctor-patient communication using a projection-based handheld device. In CHI 2011, (Vancouver, BC, 2011), 3333--3342. Google ScholarDigital Library
- Raskar, R., Welch, G., Cutts, M., et al. The office of the future: a unified approach to image-based modeling and spatially immersive displays Proceedings of SIGGRAPH 98, 1998, 179--188. Google ScholarDigital Library
- Raskar, R., Welch, G. and Fuchs, H., Spatially augmented reality. In First International Workshop on Augmented Reality, (1998), IEEE, 11--20. Google ScholarDigital Library
- Samosky, J. T., Baillargeon, E., Bregman, R., et al. Real-time "X-ray vision" for healthcare simulation: An interactive projective overlay system to enhance intubation training and other procedural training. Studies in Health Technology and Informatics, 2011, 163. 549--551.Google Scholar
- Schwald, B., Seibert, H. and Weller, T., A flexible tracking concept applied to medical scenarios using an AR window. In Proceedings of the International Symposium on Mixed and Augmented Reality, (ISMAR'02), (2002), 261--262. Google ScholarDigital Library
- Sielhorst, T., Obst, T., Burgkart, R., et al., An augmented reality delivery simulator for medical training. In AMI-ARCS 2004 International Workshop on Augmented Environments for Medical Imaging - MICCAI Satellite Workshop, (2004), 11--20.Google Scholar
- Wellner, P. Interacting with paper on the DigitalDesk. Commun. ACM, 1993, 36 (7). 87--96. Google ScholarDigital Library
- Wilson, A. D., Using a depth camera as a touch sensor. In ACM International COnference on Interactive Tabletops and Surfaces, (2010), 69--72. Google ScholarDigital Library
Index Terms
- BodyExplorerAR: enhancing a mannequin medical simulator with sensing and projective augmented reality for exploring dynamic anatomy and physiology
Recommendations
Tangible 3D tabletops: combining tangible tabletop interaction and 3D projection
NordiCHI '12: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through DesignIn this paper we present the tangible 3D tabletop and discuss the design potential of this novel interface. The tangible 3D tabletop combines tangible tabletop interaction with 3D projection in such a way that the tangible objects may be augmented with ...
Pressure Simulation in the Heart with Valve Interlocking and Isovolumetric Contraction
SA '20: SIGGRAPH Asia 2020 PostersWe have performed a particle based simulation in the heart with the aorta and the left ventricle models generated from real CT data. In the simulation, blood flow from the left ventricle to the aorta is visualized, and the pressure change in the aorta ...
Tangible Interaction in Learning Astronomy through Augmented Reality Book-Based Educational Tool
IVIC '09: Proceedings of the 1st International Visual Informatics Conference on Visual Informatics: Bridging Research and PracticeLive Solar System (LSS) is an Augmented Reality book-based educational tool. Augmented Reality (AR) has its own potential in the education field, because it can provide a seamless interaction between real and virtual objects. LSS applied the Tangible ...
Comments