skip to main content
10.1145/3334480.3383169acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
abstract
Public Access

ARTEMIS: Mixed-Reality Environment for Immersive Surgical Telementoring

Published: 25 April 2020 Publication History

Abstract

The golden hour following a traumatic injury holds the highest likelihood where surgical treatment may prevent mortality and morbidity. However, the increasing occurrence of large-scale disasters, overwhelm local medical systems and patients find themselves without timely access to medical expertise. To respond to this problem medical care experts are exploring telemedicine, which typically attempts to use synchronous audiovisual communication. While current telemedicine approaches have limited ability to impact the physical care required for trauma, AR technology allows medical professionals to see their patients while seeing additional digital information. In this paper we describe ARTEMIS (Augmented Reality Technology to Enable reMote Integrated Surgery), an immersive AR-VR telementoring infrastructure that allows experienced surgeons to remotely aid less experienced medical professionals in the field. ARTEMIS provides Mixed Reality immersive visual aids by tracking a patient in real-time and showing a reconstructed 3D point cloud in a VR environment; expert surgeons can interact with the 3D point cloud representation of the patient, instruct the remote novice through real-time 3D annotations projected in AR on the patient's body, using hand-maneuvers shown in AR through an avatar of the expert surgeon, and by projecting small video clips of specific procedures in the AR space for the novice to follow.

Supplemental Material

MP4 File
Supplemental video
MP4 File
Preview video

References

[1]
J. Amores, X. Benavides, and P. Maes. Showme: A remote collaboration system that supports immersive gestural communication. In Proc. CHI 2015.
[2]
K. M. Augestad and R. O. Lindsetmo. Overcoming distance: video-conferencing as a clinical and educational tool among surgeons. World journal of surgery, 33(7):1356--1365, 2009.
[3]
H. Chen, A. S. Lee, M. Swift, and J. C. Tang. 3d collaboration method over hololens and skype end points. In ImmersiveME '15, pages 27--30, 2015.
[4]
Y. Feng, J. Chhikara, J. Ramsey, and H. Mentis. Perceived usefulness and acceptance of telestration in laparoscopic training. In WISH Symposium at CHI 2017.
[5]
D. Gasques, J. G. Johnson, T. Sharkey, and N. Weibel. What you sketch is what you get: Quick and easy augmented reality prototyping with PintAR. In Extended Abstracts of CHI 2019.
[6]
W. Huang, S. Kim, M. Billinghurst, and L. Alem. Sharing hand gesture and sketch cues in remote collaboration. J VIS COMMUN IMAGE R, 58:428--438, 2019.
[7]
T. Sharkey, J. Johnson, D. Gasques, and N. Weibel. I want to be a surgeon! role playing for remote surgery in mixed reality. In WISH Symposium at CHI 2019.

Cited By

View all
  • (2024)Understanding The Needs of Mentoring in Surgery to Guide the Design of Surgical Telementoring SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374208:CSCW1(1-28)Online publication date: 26-Apr-2024
  • (2024)MR Microsurgical Suture Training System with Level-Appropriate SupportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642324(1-19)Online publication date: 11-May-2024
  • (2024)Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and InstructionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641929(1-12)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. ARTEMIS: Mixed-Reality Environment for Immersive Surgical Telementoring

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems
    April 2020
    4474 pages
    ISBN:9781450368193
    DOI:10.1145/3334480
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 April 2020

    Check for updates

    Author Tags

    1. ar
    2. artemis
    3. mixed reality
    4. surgery
    5. telemedicine
    6. vr

    Qualifiers

    • Abstract

    Funding Sources

    • US Department of Defense

    Conference

    CHI '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)246
    • Downloads (Last 6 weeks)43
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Understanding The Needs of Mentoring in Surgery to Guide the Design of Surgical Telementoring SystemsProceedings of the ACM on Human-Computer Interaction10.1145/36374208:CSCW1(1-28)Online publication date: 26-Apr-2024
    • (2024)MR Microsurgical Suture Training System with Level-Appropriate SupportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642324(1-19)Online publication date: 11-May-2024
    • (2024)Looking Together ≠ Seeing the Same Thing: Understanding Surgeons' Visual Needs During Intra-operative Coordination and InstructionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641929(1-12)Online publication date: 11-May-2024
    • (2024)HandyNotes: using the hands to create semantic representations of contextually aware real-world objects2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00049(265-275)Online publication date: 16-Mar-2024
    • (2024)Survey of Annotations in Extended Reality SystemsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.328886930:8(5074-5096)Online publication date: 1-Aug-2024
    • (2024)Holo-Stroke: Assessing for Immersive Stroke Care Through Stroke Hologram TeleportationTelemedicine and e-Health10.1089/tmj.2024.022930:10(2583-2591)Online publication date: 1-Oct-2024
    • (2024)Technologies Used for Telementoring in Open Surgery: A Scoping ReviewTelemedicine and e-Health10.1089/tmj.2023.066930:7(1810-1824)Online publication date: 1-Jul-2024
    • (2024)TeleOR: Real-Time Telemedicine System for Full-Scene Operating RoomMedical Image Computing and Computer Assisted Intervention – MICCAI 202410.1007/978-3-031-72089-5_59(628-638)Online publication date: 7-Oct-2024
    • (2023)Extended Reality Telemedicine Collaboration System Using Patient Avatar Based on 3D Body Pose EstimationSensors10.3390/s2401002724:1(27)Online publication date: 20-Dec-2023
    • (2023)Real-Time 3D Reconstruction Pipeline for Room-Scale, Immersive, Medical TeleconsultationApplied Sciences10.3390/app13181019913:18(10199)Online publication date: 11-Sep-2023
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media