skip to main content
10.1145/3586183.3606743acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article
Public Access

VRoxy: Wide-Area Collaboration From an Office Using a VR-Driven Robotic Proxy

Published: 29 October 2023 Publication History

Abstract

Recent research in robotic proxies has demonstrated that one can automatically reproduce many non-verbal cues important in co-located collaboration. However, they often require a symmetrical hardware setup in each location. We present the VRoxy system, designed to enable access to remote spaces through a robotic embodiment, using a VR headset in a much smaller space, such as a personal office. VRoxy maps small movements in VR space to larger movements in the physical space of the robot, allowing the user to navigate large physical spaces easily. Using VRoxy, the VR user can quickly explore and navigate in a low-fidelity rendering of the remote space. Upon the robot’s arrival, the system uses the feed of a 360 camera to support real-time interactions. The system also facilitates various interaction modalities by rendering the micro-mobility around shared spaces, head and facial animations, and pointing gestures on the proxy. We demonstrate how our system can accommodate mapping multiple physical locations onto a unified virtual space. In a formative study, users could complete a design decision task where they navigated and collaborated in a complex 7.5m x 5m layout using a 3m x 2m VR space.

Supplemental Material

ZIP File
Supplemental File

References

[1]
2022. Lifelike avatars for the metaverse. https://avatarsdk.com/
[2]
S. O. Adalgeirsson and C. Breazeal. 2010. MeBot: A robotic platform for socially embodied telepresence. In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 15–22. https://doi.org/10.1109/HRI.2010.5453272
[3]
Georgios Albanis, Nikolaos Zioulis, Petros Drakoulis, Vasileios Gkitsas, Vladimiros Sterzentsenko, Federico Alvarez, Dimitrios Zarpalas, and Petros Daras. 2021. Pano3D: A Holistic Benchmark and a Solid Baseline for 360° Depth Estimation. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 3722–3732. https://doi.org/10.1109/CVPRW53098.2021.00413
[4]
Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver. 2010. Google Street View: Capturing the World at Street Level. Computer 43 (2010). http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5481932&tag=1
[5]
Ignacio Avellino, Cédric Fleury, Wendy E. Mackay, and Michel Beaudouin-Lafon. 2017. CamRay: Camera Arrays Support Remote Collaboration on Wall-Sized Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 6718–6729. https://doi.org/10.1145/3025453.3025604
[6]
Mahdi Azmandian, Timofey Grechkin, Mark T Bolas, and Evan A Suma. 2015. Physical Space Requirements for Redirected Walking: How Size and Shape Affect Performance. In ICAT-EGVE. 93–100.
[7]
Beam. 2023. Suitable Technologies Inc. Retrieved Feb 6, 2023 from https://suitabletech.com
[8]
Costas Boletsis and Jarl Erik Cedergren. 2019. VR locomotion in the new era of virtual reality: an empirical comparison of prevalent techniques. Advances in Human-Computer Interaction 2019 (2019).
[9]
Bill Buxton. 2009. Mediaspace – Meaningspace – Meetingspace. Springer London, London, 217–231. https://doi.org/10.1007/978-1-84882-483-6_13
[10]
William A. S. Buxton. 1992. Telepresence: Integrating Shared Task and Person Spaces. In Proceedings of the Conference on Graphics Interface ’92 (Vancouver, British Columbia, Canada). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 123–129.
[11]
Zekun Cao, Jason Jerald, and Regis Kopper. 2018. Visually-induced motion sickness reduction via static and dynamic rest frames. In 2018 IEEE conference on virtual reality and 3D user interfaces (VR). IEEE, 105–112.
[12]
Weiya Chen, Anthony Plancoulaine, Nicolas Férey, Damien Touraine, Julien Nelson, and Patrick Bourdot. 2013. 6DoF navigation in virtual worlds: comparison of joystick-based and head-controlled paradigms. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology. 111–114.
[13]
Kayla Davis, Taylor Hayase, Irene Humer, Brandon Woodard, and Christian Eckhardt. 2022. A Quantitative Analysis of Redirected Walking in Virtual Reality Using Saccadic Eye Movements. In Advances in Visual Computing, George Bebis, Bo Li, Angela Yao, Yang Liu, Ye Duan, Manfred Lau, Rajiv Khadka, Ana Crisan, and Remco Chang (Eds.). Springer Nature Switzerland, Cham, 205–216.
[14]
Mohamed Elobaid, Yue Hu, Giulio Romualdi, Stefano Dafarra, Jan Babic, and Daniele Pucci. 2020. Telexistence and Teleoperation for Walking Humanoid Robots. In Intelligent Systems and Applications, Yaxin Bi, Rahul Bhatia, and Supriya Kapoor (Eds.). Springer International Publishing, Cham, 1106–1121.
[15]
Mehrad Faridan, Bheesha Kumari, and Ryo Suzuki. 2023. ChameleonControl: Teleoperating Real Human Surrogates through Mixed Reality Gestural Guidance for Remote Hands-on Classrooms. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (New York, NY, USA) (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3544548.3581449
[16]
Lars Fritsche, Felix Unverzag, Jan Peters, and Roberto Calandra. 2015. First-person tele-operation of a humanoid robot. In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids). 997–1002. https://doi.org/10.1109/HUMANOIDS.2015.7363475
[17]
Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. In Touch with the Remote World: Remote Collaboration with Augmented Reality Drawings and Virtual Navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (Edinburgh, Scotland) (VRST ’14). Association for Computing Machinery, New York, NY, USA, 197–205. https://doi.org/10.1145/2671015.2671016
[18]
William W. Gaver, John Bowers, Andrew Boucher, Hans Gellerson, Sarah Pennington, Albrecht Schmidt, Anthony Steed, Nicholas Villars, and Brendan Walker. 2004. The Drift Table: Designing for Ludic Engagement. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems (Vienna, Austria) (CHI EA ’04). Association for Computing Machinery, New York, NY, USA, 885–900. https://doi.org/10.1145/985921.985947
[19]
C. Gutwin and S. Greenberg. 2004. A Descriptive Framework of Workspace Awareness for Real-Time Groupware. Computer Supported Cooperative Work (CSCW) 11 (2004), 411–446.
[20]
Yasamin Heshmat, Brennan Jones, Xiaoxuan Xiong, Carman Neustaedter, Anthony Tang, Bernhard E. Riecke, and Lillian Yang. 2018. Geocaching with a Beam: Shared Outdoor Activities through a Telepresence Robot with 360 Degree Viewing. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173933
[21]
Keita Higuchi and Jun Rekimoto. 2013. Flying head: a head motion synchronization mechanism for unmanned aerial vehicle control. In CHI’13 Extended Abstracts on Human Factors in Computing Systems. 2029–2038.
[22]
Jim Hollan and Scott Stornetta. 1992. Beyond Being There. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Monterey, California, USA) (CHI ’92). Association for Computing Machinery, New York, NY, USA, 119–125. https://doi.org/10.1145/142750.142769
[23]
Xandex Inc.2023. KUBI Telepresence Robot. Retrieved Feb 6, 2023 from https://kubiconnect.com/
[24]
Victoria Interrante, Brian Ries, and Lee Anderson. 2007. Seven League Boots: A New Metaphor for Augmented Locomotion through Moderately Large Scale Immersive Virtual Environments. In 2007 IEEE Symposium on 3D User Interfaces. https://doi.org/10.1109/3DUI.2007.340791
[25]
Brennan Jones, Yaying Zhang (yaying zhang), Priscilla N. Y. Wong, and Sean Rintel. 2021. Belonging There: VROOM-ing into the Uncanny Valley of XR Telepresence. In CSCW 2021. ACM. https://www.microsoft.com/en-us/research/publication/belonging-there-vroom-ing-into-the-uncanny-valley-of-xr-telepresence/
[26]
Wendy Ju, Lawrence Neeley, Terry Winograd, and Larry Leifer. 2006. Thinking with Erasable Ink: Ad-hoc Whiteboard Use in Collaborative Design. Technical Report.
[27]
David Kirk and Danae Fraser. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks, Vol. 2. 1191–1200. https://doi.org/10.1145/1124772.1124951
[28]
Sven Kratz and Fred Rabelo Ferriera. 2016. Immersed remotely: Evaluating the use of Head Mounted Devices for remote collaboration in robotic telepresence. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). 638–645. https://doi.org/10.1109/ROMAN.2016.7745185
[29]
Changyang Li, Haikun Huang, Jyh-Ming Lien, and Lap-Fai Yu. 2021. Synthesizing Scene-Aware Virtual Reality Teleport Graphs. ACM Trans. Graph. 40, 6, Article 229 (dec 2021), 15 pages. https://doi.org/10.1145/3478513.3480478
[30]
Jiannan Li, Maurício Sousa, Chu Li, Jessie Liu, Yan Chen, Ravin Balakrishnan, and Tovi Grossman. 2022. ASTEROIDS: Exploring Swarms of Mini-Telepresence Robots for Physical Skill Demonstration. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 111, 14 pages. https://doi.org/10.1145/3491102.3501927
[31]
Yi-Jun Li, Frank Steinicke, and Miao Wang. 2022. A comprehensive review of redirected walking techniques: Taxonomy, methods, and future directions. Journal of Computer Science and Technology 37, 3 (2022), 561–583.
[32]
Paul Luff and Christian Heath. 1998. Mobility in Collaboration. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (Seattle, Washington, USA) (CSCW ’98). Association for Computing Machinery, New York, NY, USA, 305–314. https://doi.org/10.1145/289444.289505
[33]
Luos. 2023. Luos: Open-source and real-time orchestrator for cyber-physical-systems, to easily design, test and deploy embedded applications and digital twins. https://www.luos.io
[34]
Michal Luria, Samantha Reig, Xiang Zhi Tan, Aaron Steinfeld, Jodi Forlizzi, and John Zimmerman. 2019. Re-Embodiment and Co-Embodiment: Exploration of social presence for robots and conversational agents. In Proceedings of the 2019 on Designing Interactive Systems Conference. 633–644.
[35]
Daniel R. Montello. 1998. A New Framework for Understanding the Acquisition of Spatial Knowledge in Large-Scale Environments. https://api.semanticscholar.org/CorpusID:40864171
[36]
Pieter Moors, Filip Germeys, Iwona Pomianowska, and Karl Verfaillie. 2015. Perceiving where another person is looking: the integration of head and body information in estimating another personâs gaze. Frontiers in Psychology 6 (2015). https://doi.org/10.3389/fpsyg.2015.00909
[37]
Raul Mur-Artal, Jose Maria Martinez Montiel, and Juan D Tardos. 2015. ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE transactions on robotics 31, 5 (2015), 1147–1163.
[38]
Inc. Niantic. 2023. Scaniverse 3D Scanner with LiDAR for iPhone and iPad. Retrieved Feb 6, 2023 from https://scaniverse.com/
[39]
K. Otsuka. 2016. MMSpace: Kinetically-augmented telepresence for small group-to-group conversations. In 2016 IEEE Virtual Reality (VR). 19–28. https://doi.org/10.1109/VR.2016.7504684
[40]
Tomislav Pejsa, Julian Kantor, Hrvoje Benko, Eyal Ofek, and Andrew Wilson. 2016. Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW ’16). ACM, New York, NY, USA, 1716–1725. https://doi.org/10.1145/2818048.2819965
[41]
Thammathip Piumsomboon, Gun A. Lee, Jonathon D. Hart, Barrett Ens, Robert W. Lindeman, Bruce H. Thomas, and Mark Billinghurst. 2018. Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). ACM, New York, NY, USA, Article 46, 13 pages. https://doi.org/10.1145/3173574.3173620
[42]
Irene Rae, Bilge Mutlu, and Leila Takayama. 2014. Bodies in Motion: Mobility, Presence, and Task Awareness in Telepresence. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). ACM, New York, NY, USA, 2153–2162. https://doi.org/10.1145/2556288.2557047
[43]
Pranav Rane, Varun Mhatre, and Lakshmi Kurup. 2014. Study of a home robot: Jibo. International journal of engineering research and technology 3, 10 (2014), 490–493.
[44]
Bernhard E Riecke, Bobby Bodenheimer, Timothy P McNamara, Betsy Williams, Peng Peng, and Daniel Feuereissen. 2010. Do we need to walk for effective virtual reality navigation? physical rotations alone may suffice. In Spatial Cognition VII: International Conference, Spatial Cognition 2010, Mt. Hood/Portland, OR, USA, August 15-19, 2010. Proceedings 7. Springer, 234–247.
[45]
Pollen Robotics. 2023. Orbita: A 3D joint for robotic head motion realism. https://medium.com/pollen-robotics/orbita-is-turning-heads-literally-d10d378550e2
[46]
Roy A. Ruddle, Ekaterina Volkova, and Heinrich H. Bülthoff. 2011. Walking Improves Your Cognitive Map in Environments That Are Large-Scale and Large in Extent. ACM Trans. Comput.-Hum. Interact. 18, 2, Article 10 (jul 2011), 20 pages. https://doi.org/10.1145/1970378.1970384
[47]
Mose Sakashita, Tatsuya Minagawa, Amy Koike, Ippei Suzuki, Keisuke Kawahara, and Yoichi Ochiai. 2017. You as a Puppet: Evaluation of Telepresence User Interface for Puppetry. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery, New York, NY, USA, 217–228. https://doi.org/10.1145/3126594.3126608
[48]
Mose Sakashita, E. Andy Ricci, Jatin Arora, and François Guimbretière. 2022. RemoteCoDe: Robotic Embodiment for Enhancing Peripheral Awareness in Remote Collaboration Tasks. Proc. ACM Hum.-Comput. Interact. 6, CSCW1, Article 63 (apr 2022), 22 pages. https://doi.org/10.1145/3512910
[49]
Mose Sakashita, Ruidong Zhang, Xiaoyi Li, Hyunju Kim, Michael Russo, Cheng Zhang, Malte F. Jung, and François Guimbretière. 2023. ReMotion: Supporting Remote Collaboration in Open Space with Automatic Robotic Embodiment. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (New York, NY, USA) (CHI ’23). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3544548.3580699
[50]
MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa, and Masahiko Inami. 2018. Fusion: full body surrogacy for collaborative communication. In ACM SIGGRAPH 2018 Emerging Technologies. 1–2.
[51]
Ehsan Sayyad, Misha Sra, and Tobias Höllerer. 2020. Walking and teleportation in wide-area virtual reality experiences. In 2020 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 608–617.
[52]
Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D Mobile Collaboration with Spatial Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). ACM, New York, NY, USA, 179–188. https://doi.org/10.1145/2470654.2470679
[53]
Frank Steinicke, Gerd Bruder, Jason Jerald, Harald Frenz, and Markus Lappe. 2010. Estimation of Detection Thresholds for Redirected Walking Techniques. IEEE Transactions on Visualization and Computer Graphics 16, 1 (2010), 17–27. https://doi.org/10.1109/TVCG.2009.62
[54]
Susumu Tachi, Yasuyuki Inoue, and Fumihiro Kato. 2020. TELESAR VI: Telexistence Surrogate Anthropomorphic Robot VI. International Journal of Humanoid Robotics 17, 5 (Oct 2020), 2050019(1–33).
[55]
Anthony Tang, Melanie Tory, Barry Po, Petra Neumann, and Sheelagh Carpendale. 2006. Collaborative Coupling over Tabletop Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada) (CHI ’06). Association for Computing Machinery, New York, NY, USA, 1181–1190. https://doi.org/10.1145/1124772.1124950
[56]
Unity Technologies. 2023. Unity Render Streaming. https://github.com/Unity-Technologies/UnityRenderStreaming
[57]
Theophilus Teo, Louise Lawrence, Gun A. Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed Reality Remote Collaboration Combining 360 Video and 3D Reconstruction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3290605.3300431
[58]
Balasaravanan Thoravi Kumaravel, Fraser Anderson, George Fitzmaurice, Bjoern Hartmann, and Tovi Grossman. 2019. Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 161–174. https://doi.org/10.1145/3332165.3347872
[59]
Yunong Tian, Guodong Yang, Zhe Wang, Hao Wang, En Li, and Zize Liang. 2019. Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Computers and electronics in agriculture 157 (2019), 417–426.
[60]
Katherine M. Tsui and Holly A. Yanco. 2013. Design Challenges and Guidelines for Social Interaction Using Mobile Telepresence Robots. Reviews of Human Factors and Ergonomics 9, 1 (2013), 227–301. https://doi.org/10.1177/1557234X13502462 arXiv:https://doi.org/10.1177/1557234X13502462
[61]
Roel Vertegaal. 1999. The GAZE Groupware System: Mediating Joint Attention in Multiparty Communication and Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 294–301. https://doi.org/10.1145/302979.303065

Cited By

View all
  • (2024)SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642945(1-14)Online publication date: 11-May-2024

Index Terms

  1. VRoxy: Wide-Area Collaboration From an Office Using a VR-Driven Robotic Proxy

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    UIST '23: Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
    October 2023
    1825 pages
    ISBN:9798400701320
    DOI:10.1145/3586183
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 29 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    Conference

    UIST '23

    Acceptance Rates

    Overall Acceptance Rate 561 of 2,567 submissions, 22%

    Upcoming Conference

    UIST '25
    The 38th Annual ACM Symposium on User Interface Software and Technology
    September 28 - October 1, 2025
    Busan , Republic of Korea

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)397
    • Downloads (Last 6 weeks)56
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)SharedNeRF: Leveraging Photorealistic and View-dependent Rendering for Real-time and Remote CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642945(1-14)Online publication date: 11-May-2024

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media