research-article

Public Access

DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments

Authors:

Julie DorseyAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 411, Pages 1 - 12

https://doi.org/10.1145/3411764.3445552

Published: 07 May 2021 Publication History

All formats PDF

Abstract

Most augmented reality (AR) authoring tools only support the author’s current environment, but designers often need to create site-specific experiences for a different environment. We propose DistanciAR, a novel tablet-based workflow for remote AR authoring. Our baseline solution involves three steps. A remote environment is captured by a camera with LiDAR; then, the author creates an AR experience from a different location using AR interactions; finally, a remote viewer consumes the AR content on site. A formative study revealed understanding and navigating the remote space as key challenges with this solution. We improved the authoring interface by adding two novel modes: Dollhouse, which renders a bird’s-eye view, and Peek, which creates photorealistic composite images using captured images. A second study compared this improved system with the baseline, and participants reported that the new modes made it easier to understand and navigate the remote scene.

Supplementary Material

VTT File (3411764.3445552_videofigurecaptions.vtt)

Download
3.41 KB

VTT File (3411764.3445552_videopreviewcaptions.vtt)

Download
.57 KB

Supplementary Materials (3411764.3445552_supplementalmaterials.zip)

Download
94.67 MB

MP4 File (3411764.3445552_videofigure.mp4)

Supplemental video

Download
69.16 MB

MP4 File (3411764.3445552_videopreview.mp4)

Preview video

Download
10.33 MB

References

[1]

Edward H Adelson, Charles H Anderson, James R Bergen, Peter J Burt, and Joan M Ogden. 1984. Pyramid Methods in Image Processing. RCA Engineer 29, 6 (1984), 33–41.

[2]

Adobe. 2020. Aero. https://www.adobe.com/products/aero.html. Accessed Sep 12, 2020.

[3]

Apple. 2020. ARKit. https://developer.apple.com/documentation/arkit. Accessed Sep 12, 2020.

[4]

Apple. 2020. Reality Composer. https://developer.apple.com/augmented-reality/reality-composer/. Accessed Sep 12, 2020.

[5]

Apple. 2020. RealityKit. https://developer.apple.com/documentation/realitykit. Accessed Sep 12, 2020.

[6]

Apple. 2020. SceneKit. https://developer.apple.com/documentation/scenekit. Accessed Sep 12, 2020.

[7]

Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K Chilana. 2020. Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[8]

Matt Bell. 2020. Matterport. https://matterport.com/. Accessed Dec 17, 2020.

[9]

Jan Brejcha, Michal Lukác, Zhili Chen, Stephen DiVerdi, and Martin Cadík. 2018. Immersive Trip Reports. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany). Association for Computing Machinery, New York, NY, USA, 389–401.

Digital Library

[10]

Ricardo Cabello. 2020. three.js. https://threejs.org/. Accessed Sep 12, 2020.

[11]

Andy Cockburn, Amy Karlson, and Benjamin B Bederson. 2009. A Review of Overview+Detail, Zooming, and Focus+Context Interfaces. Comput. Surveys 41, 1 (2009), 1–31.

Digital Library

[12]

Ruofei Du, Eric Turner, Maksym Dzitsiuk, Luca Prasso, Ivo Duarte, Jason Dourgarian, Joao Afonso, Jose Pascoal, Josh Gladstone, Nuno Cruces, Shahram Izadi, Adarsh Kowdle, Konstantine Tsotsos, and David Kim. 2020. DepthLab: Real-Time 3D Interaction with Depth Maps for Mobile Augmented Reality. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology(Virtual Event, USA). Association for Computing Machinery, New York, NY, USA, 829–843.

Digital Library

[13]

Facebook. 2020. Spark AR Studio. https://sparkar.facebook.com/ar-studio/. Accessed Sep 12, 2020.

[14]

Anhong Guo, Ilter Canberk, Hannah Murphy, Andrés Monroy-Hernández, and Rajan Vaish. 2019. Blocks: Collaborative and Persistent Augmented Reality Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 3, Article 83 (Sept. 2019), 24 pages.

Digital Library

[15]

Sinem Guven, Steven Feiner, and Ohan Oda. 2006. Mobile Augmented Reality Interaction Techniques for Authoring Situated Media On-Site. In 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, 235–236.

[16]

André Kunert, Alexander Kulik, Stephan Beck, and Bernd Froehlich. 2014. Photoportals: Shared References in Space and Time. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA). Association for Computing Machinery, New York, NY, USA, 1388–1399.

Digital Library

[17]

Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid Augmented Reality Video Prototyping Using Sketches and Enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA). Association for Computing Machinery, New York, NY, USA, 1–13.

Digital Library

[18]

Yuwei Li, Xi Luo, Youyi Zheng, Pengfei Xu, and Hongbo Fu. 2017. SweepCanvas: Sketch-based 3D Prototyping on an RGB-D Image. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 387–399.

Digital Library

[19]

David Lindlbauer and Andy D. Wilson. 2018. Remixed Reality: Manipulating Space and Time in Augmented Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada). Association for Computing Machinery, New York, NY, USA, 1–13.

Digital Library

[20]

Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, and Jan Kautz. 2019. PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4450–4459.

[21]

Feng Liu, Yu-hen Hu, and Michael L. Gleicher. 2008. Discovering Panoramas in Web Videos. In Proceedings of the 16th ACM International Conference on Multimedia (Vancouver, British Columbia, Canada). Association for Computing Machinery, New York, NY, USA, 329–338.

Digital Library

[22]

Blair MacIntyre, Maribeth Gandy, Jay Bolter, Steven Dow, and Brendan Hannigan. 2003. DART: the Designer’s Augmented Reality Toolkit. In The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings. IEEE, 329–330.

[23]

Sergio Orts-Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, Yury Degtyarev, David Kim, Philip L. Davidson, Sameh Khamis, Mingsong Dou, Vladimir Tankovich, Charles Loop, Qin Cai, Philip A. Chou, Sarah Mennicken, Julien Valentin, Vivek Pradeep, Shenlong Wang, Sing Bing Kang, Pushmeet Kohli, Yuliya Lutchyn, Cem Keskin, and Shahram Izadi. 2016. Holoportation: Virtual 3D Teleportation in Real-Time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan). Association for Computing Machinery, New York, NY, USA, 741–754.

Digital Library

[24]

Patrick Paczkowski, Min H. Kim, Yann Morvan, Julie Dorsey, Holly Rushmeier, and Carol O’Sullivan. 2011. Insitu: Sketching Architectural Designs in Context. ACM Trans. Graph. 30, 6 (Dec. 2011), 1–10.

Digital Library

[25]

Benjamin Reynolds, Hisham Bedri, Valentin Heun, Anna Fusté, and Christian Vazquez. 2019. Remote Spatial Programming and Collaboration Using a Real-Time Volumetric Capture Space. In ACM SIGGRAPH 2019 Virtual, Augmented, and Mixed Reality (Los Angeles, California). Association for Computing Machinery, New York, NY, USA, Article 27, 1 pages.

Digital Library

[26]

Armin Ronacher. 2020. Flask. https://flask.palletsprojects.com/en/1.1.x/. Accessed Sep 12, 2020.

[27]

Aditya Sankar and Steve M. Seitz. 2017. Interactive Room Capture on 3D-Aware Mobile Devices. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada). Association for Computing Machinery, New York, NY, USA, 415–426.

Digital Library

[28]

Noah Snavely, Steven M. Seitz, and Richard Szeliski. 2006. Photo Tourism: Exploring Photo Collections in 3D. ACM Trans. Graph. 25, 3 (July 2006), 835–846.

Digital Library

[29]

Noah Snavely, Steven M Seitz, and Richard Szeliski. 2008. Modeling the World from Internet Photo Collections. International Journal of Computer Vision 80, 2 (2008), 189–210.

Digital Library

[30]

Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D Mobile Collaboration with Spatial Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France). Association for Computing Machinery, New York, NY, USA, 179–188.

Digital Library

[31]

Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA). ACM Press/Addison-Wesley Publishing Co., USA, 265–272.

Digital Library

[32]

Yuta Sugiura, Hikaru Ibayashi, Toby Chong, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Takashi Shinmura, Masaaki Mochimaru, and Takeo Igarashi. 2018. An Asymmetric Collaborative System for Architectural-Scale Space Design. In Proceedings of the 16th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (Tokyo, Japan). Association for Computing Machinery, New York, NY, USA, Article 21, 6 pages.

Digital Library

[33]

Franco Tecchia, Leila Alem, and Weidong Huang. 2012. 3D Helping Hands: A Gesture Based MR System for Remote Collaboration. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (Singapore, Singapore). Association for Computing Machinery, New York, NY, USA, 323–328.

Digital Library

[34]

Unity Technologies. 2020. Unity. https://unity.com/. Accessed Sep 12, 2020.

[35]

Balasaravanan Thoravi Kumaravel, Fraser Anderson, George Fitzmaurice, Björn Hartmann, and Tovi Grossman. 2019. Loki: Facilitating Remote Instruction of Physical Tasks Using Bi-Directional Mixed-Reality Telepresence. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA). Association for Computing Machinery, New York, NY, USA, 161–174.

Digital Library

[36]

Michael Tsang, George W. Fitzmaurice, Gordon Kurtenbach, Azam Khan, and Bill Buxton. 2002. Boom Chameleon: Simultaneous Capture of 3D Viewpoint, Voice and Gesture Annotations on a Spatially-Aware Display. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (Paris, France). Association for Computing Machinery, New York, NY, USA, 111–120.

Digital Library

[37]

Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Ke Huo, Yuanzhi Cao, and Karthik Ramani. 2020. CAPturAR: An Augmented Reality Tool for Authoring Human-Involved Context-Aware Applications. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA). Association for Computing Machinery, New York, NY, USA, 328–341.

Digital Library

[38]

Zeyu Wang, Shiyu Qiu, Qingyang Chen, Natallia Trayan, Alexander Ringlein, Julie Dorsey, and Holly Rushmeier. 2019. AniCode: Authoring Coded Artifacts for Network-Free Personalized Animations. The Visual Computer 35, 6-8 (2019), 885–897.

Digital Library

[39]

Michelle Q Wang Baldonado, Allison Woodruff, and Allan Kuchinsky. 2000. Guidelines for Using Multiple Views in Information Visualization. In Proceedings of the Working Conference on Advanced Visual Interfaces. 110–119.

Digital Library

[40]

Evan You. 2020. Vue.js. https://vuejs.org/. Accessed Sep 12, 2020.

[41]

Ya-Ting Yue, Yong-Liang Yang, Gang Ren, and Wenping Wang. 2017. SceneCtrl: Mixed Reality Enhancement via Efficient Scene Editing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada). Association for Computing Machinery, New York, NY, USA, 427–436.

Digital Library

[42]

Yinda Zhang, Neal Wadhwa, Sergio Orts-Escolano, Christian Häne, Sean Fanello, and Rahul Garg. 2020. Du2Net: Learning Depth Estimation from Dual-Cameras and Dual-Pixels. In European Conference on Computer Vision, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 582–598.

Digital Library

Cited By

Hai-Jew S(2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024
https://doi.org/10.4018/979-8-3693-1078-6.ch006
Liebers CMegarajan PAuda JStratmann TPfingsthorn MGruenefeld USchneegass S(2024)Keep the Human in the Loop: Arguments for Human Assistance in the Synthesis of Simulation Data for Robot TrainingMultimodal Technologies and Interaction10.3390/mti80300188:3(18)Online publication date: 1-Mar-2024
https://doi.org/10.3390/mti8030018
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Show More Cited By

Index Terms

DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Index terms have been assigned to the content through auto-classification.

Recommendations

ScalAR: Authoring Semantically Adaptive Augmented Reality Experiences in Virtual Reality
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Augmented Reality (AR) experiences tightly associate virtual contents with environmental entities. However, the dissimilarity of different environments limits the adaptive AR content behaviors under large-scale deployment. We propose ScalAR, an ...
Extending Virtual Reality Display Wall Environments Using Augmented Reality
SUI '19: Symposium on Spatial User Interaction

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...
Definition of a Reference Architecture for the Integration of Augmented Reality/Virtual Reality Techniques in Remote Experimentation Environments
Proceedings TEEM 2022: Tenth International Conference on Technological Ecosystems for Enhancing Multiculturality
Abstract
Remote labs are especially important for learning because they bring real results and each experience is unique. On the other hand, lack of immersion is one of the negative aspects of remote labs. This aspect can be solved through the integration ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,641
Total Downloads

Downloads (Last 12 months)381
Downloads (Last 6 weeks)80

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hai-Jew S(2024)Using Artmaking Generative AIs to Support Augmented Reality Learning Designs With Adobe Aero AppInquiries of Pedagogical Shifts and Critical Mindsets Among Educators10.4018/979-8-3693-1078-6.ch006(132-152)Online publication date: 8-Mar-2024
https://doi.org/10.4018/979-8-3693-1078-6.ch006
Liebers CMegarajan PAuda JStratmann TPfingsthorn MGruenefeld USchneegass S(2024)Keep the Human in the Loop: Arguments for Human Assistance in the Synthesis of Simulation Data for Robot TrainingMultimodal Technologies and Interaction10.3390/mti80300188:3(18)Online publication date: 1-Mar-2024
https://doi.org/10.3390/mti8030018
Liao JVan KXia ZSuzuki R(2024)RealityEffects: Augmenting 3D Volumetric Videos with Object-Centric Annotation and Dynamic Visual EffectsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661631(1248-1261)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661631
Kim MAlghofaili RLi CYu L(2024)Dragon's Path: Synthesizing User-Centered Flying Creature Animation Paths for Outdoor Augmented Reality ExperiencesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657397(1-11)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657397
Aiordachioae AVatavu R(2024)Lifelogging in Mixed RealityExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650897(1-8)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3650897
Cho HYan YTodi KParent MSmith MJonker TBenko HLindlbauer D(2024)MineXR: Mining Personalized Extended Reality InterfacesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642394(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642394
Van Brummelen JUrwin LJohnston OSayed MBrostow G(2024)Don’t Look Now: Audio/Haptic Guidance for 3D Scanning of LandmarksProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642271(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642271
Qian XWang TXu XJonker TTodi K(2024)Fast-Forward Reality: Authoring Error-Free Context-Aware Policies with Real-Time Unit Tests in Extended RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642158(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642158
Chen JZhao LLi YXie ZWilensky UHorn M(2024)“Oh My God! It’s Recreating Our Room!” Understanding Children’s Experiences with A Room-Scale Augmented Reality Authoring ToolkitProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642043(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642043
Hai-Jew S(2023)Getting Started With Augmented Reality (AR) in Inclusive Online Teaching and Learning in Higher EducationNavigating Virtual Worlds and the Metaverse for Enhanced E-Learning10.4018/979-8-3693-1034-2.ch003(20-83)Online publication date: 24-Nov-2023
https://doi.org/10.4018/979-8-3693-1034-2.ch003
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten