skip to main content
10.1145/3132787.3139204acmconferencesArticle/Chapter ViewAbstractPublication Pagessiggraph-asiaConference Proceedingsconference-collections
abstract

Static local environment capturing and sharing for MR remote collaboration

Published: 27 November 2017 Publication History

Abstract

We present a Mixed Reality (MR) system that supports entire scene capturing of the local physical work environment for remote collaboration in a large-scale workspace. By integrating the key-frames captured with external depth sensor as one single 3D point-cloud data set, our system could reconstruct the entire local physical workspace into the VR world. In this case, the remote helper could observe the local scene independently from the local user's current head and camera position, and provide gesture guiding information even before the local user staring at the target object. We conducted a pilot study to evaluate the usability of the system by comparing it with our previous oriented view system which only sharing the current camera view together with the real-time head orientation data. Our results indicate that this entire scene capturing and sharing system could significantly increase the remote helper's spatial awareness of the local work environment, especially in a large-scale workspace, and gain an overwhelming user preference (80%) than previous system.

References

[1]
Matt Adcock, Stuart Anderson, and Bruce Thomas. 2013. RemoteFusion: Real Time Depth Camera Fusion for Remote Collaboration on Physical Tasks. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI '13). ACM, New York, NY, USA, 235--242.
[2]
Leila Alem and Jane Li. 2011. A Study of Gestures in a Video-mediated Collaborative Assembly Task. Adv. in Hum.-Comp. Int. 2011, Article 1 (Jan. 2011), 7 pages.
[3]
Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1343--1348.
[4]
Mark Billinghurst, Alaeddin Nassani, and Carolin Reichherzer. 2014. Social panoramas: using wearable computers to share experiences. In SIGGRAPH Asia 2014 Mobile Graphics and Interactive Applications. ACM, 25.
[5]
Yang Chen and Gérard Medioni. 1991. Object modeling by registration of multiple range images. In Robotics and Automation, 1991. Proceedings., 1991 IEEE International Conference on. IEEE, 2724--2729.
[6]
Susan R. Fussell, Leslie D. Setlock, and Robert E. Kraut. 2003. Effects of Head-mounted and Scene-oriented Video Systems on Remote Collaboration on Physical Tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 513--520.
[7]
Lei Gao, Huidong Bai, Gun Lee, and Mark Billinghurst. 2016. An Oriented Point-cloud View for MR Remote Collaboration. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications (SA '16). ACM, New York, NY, USA, Article 8, 4 pages.
[8]
Chad Harms and Frank Biocca. 2004. Internal consistency and reliability of the networked minds measure of social presence. (2004).
[9]
Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or handsfree?: Remote collaboration via lightweight head-mounted displays and handheld devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1825--1836.
[10]
Shunichi Kasahara, Shohei Nagai, and Jun Rekimoto. 2014. LiveSphere: immersive experience sharing with 360 degrees head-mounted cameras. In Proceedings of the adjunct publication of the 27th annual ACM symposium on User interface software and technology. ACM, 61--62.
[11]
Seungwon Kim, Gun Lee, Nobuchika Sakata, and Mark Billinghurst. 2014. Improving co-presence with augmented visual communication cues for sharing experience through video conference. In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on. IEEE, 83--92.
[12]
Joel Lanir, Ran Stone, Benjamin Cohen, and Pavel Gurevich. 2013. Ownership and Control of Point of View in Remote Assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2243--2252.
[13]
Abhishek Ranjan, Jeremy P. Birnholtz, and Ravin Balakrishnan. 2007. Dynamic Shared Visual Spaces: Experimenting with Automatic Camera Control in a Remote Repair Task. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 1177--1186.
[14]
Dongmahn Seo, Suhyun Kim, JaeWook Yoo, Hogun Park, and Heedong Ko. 2012. Immersive panorama TV service system. In Consumer Electronics (ICCE), 2012 IEEE International Conference on. IEEE, 201--202.
[15]
Rodrigo Silva, Bruno Feijó, Pablo B Gomes, Thiago Frensh, and Daniel Monteiro. 2016. Real time 360°video stitching and streaming. In ACM SIGGRAPH 2016 Posters. ACM, 70.
[16]
Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D Mobile Collaboration with Spatial Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 179--188.
[17]
Franco Tecchia, Leila Alem, and Weidong Huang. 2012. 3D Helping Hands: A Gesture Based MR System for Remote Collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (VRCAI '12). ACM, New York, NY, USA, 323--328.

Cited By

View all
  • (2024)Panoramic Ray Tracing for Interactive Mixed Reality Rendering Based on 360° RGBD VideoIEEE Computer Graphics and Applications10.1109/MCG.2023.332738344:1(62-75)Online publication date: Jan-2024
  • (2024)Efficient VR-AR communication method using virtual replicas in XR remote collaborationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103304190:COnline publication date: 1-Oct-2024
  • (2024)A novel MR remote collaborative assembly system using reconstructed attribute-enhanced product modelsThe International Journal of Advanced Manufacturing Technology10.1007/s00170-024-14721-2135:7-8(3903-3940)Online publication date: 2-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SA '17: SIGGRAPH Asia 2017 Mobile Graphics & Interactive Applications
November 2017
116 pages
ISBN:9781450354103
DOI:10.1145/3132787
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 November 2017

Check for updates

Author Tags

  1. RGB-D camera
  2. mixed reality
  3. point cloud
  4. remote collaboration
  5. tele-presence

Qualifiers

  • Abstract

Conference

SA '17
Sponsor:
SA '17: SIGGRAPH Asia 2017
November 27 - 30, 2017
Bangkok, Thailand

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)47
  • Downloads (Last 6 weeks)3
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Panoramic Ray Tracing for Interactive Mixed Reality Rendering Based on 360° RGBD VideoIEEE Computer Graphics and Applications10.1109/MCG.2023.332738344:1(62-75)Online publication date: Jan-2024
  • (2024)Efficient VR-AR communication method using virtual replicas in XR remote collaborationInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103304190:COnline publication date: 1-Oct-2024
  • (2024)A novel MR remote collaborative assembly system using reconstructed attribute-enhanced product modelsThe International Journal of Advanced Manufacturing Technology10.1007/s00170-024-14721-2135:7-8(3903-3940)Online publication date: 2-Nov-2024
  • (2024)Communication Cues for Remote GuidanceComputer‐Supported Collaboration10.1002/9781119719830.ch4(81-114)Online publication date: 24-May-2024
  • (2023)UnMapped: Leveraging Experts’ Situated Experiences to Ease Remote Guidance in Collaborative Mixed RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581444(1-20)Online publication date: 19-Apr-2023
  • (2023)Using Virtual Replicas to Improve Mixed Reality Remote CollaborationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324711329:5(2785-2795)Online publication date: May-2023
  • (2023)A User Study on the Comparison of View Interfaces for VR-AR Communication in XR Remote CollaborationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.224129440:19(5794-5809)Online publication date: 17-Aug-2023
  • (2023)A Survey of Computer-Supported Remote Collaboration on Physical ObjectsHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42286-7_19(346-368)Online publication date: 25-Aug-2023
  • (2022)A Survey on Synchronous Augmented, Virtual, andMixed Reality Remote Collaboration SystemsACM Computing Surveys10.1145/353337655:6(1-27)Online publication date: 7-Dec-2022
  • (2022)ColabAR: A Toolkit for Remote Collaboration in Tangible Augmented Reality LaboratoriesProceedings of the ACM on Human-Computer Interaction10.1145/35129286:CSCW1(1-22)Online publication date: 7-Apr-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media