Weichen Bi
ABSTRACT
Mobile extended reality (XR) has developed rapidly in recent years. Compared with the app-based XR, XR in web browsers has the advantages of being lightweight and cross-platform, providing users with a pervasive experience. Therefore, many frameworks are emerging to support the development of XR in web browsers. However, little has been known about how well these frameworks perform and how complex XR apps modern web browsers can support on mobile devices. To fill the knowledge gap, in this paper, we conduct an empirical study of mobile XR in web browsers. We select seven most popular web-based XR frameworks and investigate their runtime performance, including 3D rendering, camera capturing, and real-world understanding. We find that current frameworks have the potential to further enhance their performance by increasing GPU utilization or improving computing parallelism. Besides, for 3D scenes with good rendering performance, developers can feel free to add camera capturing with little influence on performance to support augmented reality (AR) and mixed reality (MR) applications. Based on our findings, we draw several practical implications to provide better XR support in web browsers.
- 2022. A-Frame. https://github.com/aframevr/aframe.Google Scholar
- 2022. Apple App Store. https://www.apple.com/app-store/.Google Scholar
- 2022. ARCore. https://developers.google.com/ar.Google Scholar
- 2022. AR.js. https://github.com/AR-js-org/AR.js.Google Scholar
- 2022. ARKit. https://developer.apple.com/augmented-reality/.Google Scholar
- 2022. babylon.js. https://github.com/BabylonJS/Babylon.js.Google Scholar
- 2022. Chrome Dev Tool. https://developer.chrome.com/docs/devtools/.Google Scholar
- 2022. Extended Reality Market. https://www.precedenceresearch.com/extended-reality-market.Google Scholar
- 2022. FBX: Adaptable file format for 3D animation software. https://www.autodesk.com/products/fbx/overview.Google Scholar
- 2022. Firefox Dev Tool. https://firefox-dev.tools/.Google Scholar
- 2022. glTF RUNTIME 3D ASSET DELIVERY. https://www.khronos.org/gltf/.Google Scholar
- 2022. Google Play. https://play.google.com/store/apps/¿hl=en&gl=US.Google Scholar
- 2022. MindAR.js. https://github.com/hiukim/mind-ar-js.Google Scholar
- 2022. OBJ File Format. https://docs.fileformat.com/3d/obj/.Google Scholar
- 2022. Oculus Quest. https://www.oculus.com/experiences/quest/.Google Scholar
- 2022. Pico VR Headset. https://www.picoxr.com/us/G2_4K.html.Google Scholar
- 2022. Playcanvas. https://github.com/playcanvas/engine.Google Scholar
- 2022. React-three-fiber. https://github.com/pmndrs/react-three-fiber.Google Scholar
- 2022. Safari Dev Tool. https://support.apple.com/en-hk/guide/safari/sfri20948/mac.Google Scholar
- 2022. three.ar.js. https://github.com/google-ar/three.ar.js.Google Scholar
- 2022. three.js. https://github.com/mrdoob/three.js/.Google Scholar
- 2022. TraceLib: A Node.js library that provides Chrome DevTools trace models to parse arbitrary trace logs.https://github.com/saucelabs/tracelib.Google Scholar
- 2022. tracking.js. https://github.com/eduardolundgren/tracking.js/.Google Scholar
- 2022. WebXR Device API. https://www.w3.org/TR/webxr/.Google Scholar
- Christopher Andrews, Michael K Southworth, Jennifer NA Silva, and Jonathan R Silva. 2019. Extended reality in medical practice. Current treatment options in cardiovascular medicine 21, 4 (2019), 1–12.Google Scholar
- Hussein Bakri. 2019. Adaptivity of 3D web content in web-based virtual museums: a quality of service and quality of experience perspective.Google Scholar
- Tayebeh Baniasadi, Seyed Mohammad Ayyoubzadeh, and Niloofar Mohammadzadeh. 2020. Challenges and practical considerations in applying virtual reality in medical education and treatment. Oman medical journal 35, 3 (2020), e125.Google Scholar
- Eleonora Bottani and Giuseppe Vignali. 2019. Augmented reality technology in the manufacturing industry: A review of the last decade. Iise Transactions 51, 3 (2019), 284–310.Google ScholarCross Ref
- Eric Bubar, Susan Agolini, Deana Jaber, and Amanda Wright. 2021. Three Methods for Adapting Physical Games to Virtual Formats in STEM Courses – Easy (Google Suite), Medium (Web GL Games in Unity) and Hard (Virtual Reality). In HCI International 2021 - Posters, Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa (Eds.). Springer International Publishing, Cham, 141–147.Google Scholar
- Hsin-Yuan Chen, Ruey-Tzer Hsu, Ying-Chiao Chen, Wei-Chen Hsu, and Polly Huang. 2021. AR Game Traffic Characterization: A Case of Pokémon Go in a Flash Crowd Event. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (Virtual Event, Wisconsin) (MobiSys ’21). Association for Computing Machinery, New York, NY, USA, 493–494. https://doi.org/10.1145/3458864.3466914Google ScholarDigital Library
- Sofiane Chetoui, Rahul Shahi, Seif Abdelaziz, Abhinav Golas, Farrukh Hijaz, and Sherief Reda. 2022. ARBench: Augmented Reality Benchmark For Mobile Devices. In 2022 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). 242–244. https://doi.org/10.1109/ISPASS55109.2022.00035Google ScholarCross Ref
- Jaewon Choi, HyeonJung Park, Jeongyeup Paek, Rajesh Krishna Balan, and JeongGil Ko. 2019. LpGL: Low-Power Graphics Library for Mobile AR Headsets. In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 155–167. https://doi.org/10.1145/3307334.3326097Google ScholarDigital Library
- Jaewon Choi, Hyeonjung Park, Jeongyeup Paek, and JeongGil Ko. 2018. Reactive Mesh Simplification for Augmented Reality Head Mounted Displays. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (Munich, Germany) (MobiSys ’18). Association for Computing Machinery, New York, NY, USA, 527. https://doi.org/10.1145/3210240.3210820Google ScholarDigital Library
- Mariana Danielová, Pavel Janečka, Jakub Grosz, and Aleš Holý. 2019. Interactive 3D Visualizations of Laser Plasma Experiments on the Web and in VR. In EuroVis 2019 - Posters, João Madeiras Pereira and Renata Georgia Raidou (Eds.). The Eurographics Association. https://doi.org/10.2312/eurp.20191145Google ScholarCross Ref
- János Dóka, Bálint György Nagy, Muhammad Atif Ur Rehman, Dong-Hak Kim, Byung-Seo Kim, László Toka, and Balázs Sonkoly. 2020. AR over NDN: Augmented Reality Applications and the Rise of Information Centric Networking. In Proceedings of the SIGCOMM ’20 Poster and Demo Sessions (Virtual event) (SIGCOMM ’20). Association for Computing Machinery, New York, NY, USA, 44–45. https://doi.org/10.1145/3405837.3411386Google ScholarDigital Library
- Laura Freina and Michela Ott. 2015. A literature review on immersive virtual reality in education: state of the art and perspectives. In The international scientific conference elearning and software for education, Vol. 1. 10–1007.Google Scholar
- Yunha Han, Chunggi Lee, Sanghoon Kim, and Sungahn Ko. 2019. System Architecture for Progressive Augmented Reality (Poster). In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 522–523. https://doi.org/10.1145/3307334.3328605Google ScholarDigital Library
- Matteus Hemström and Anton Forsberg. 2020. A Comparison of WebVR and Native VR: Impacts on Performance and User Experience.Google Scholar
- Tanner Hobson, Jeremiah Duncan, Mohammad Raji, Aidong Lu, and Jian Huang. 2020. Alpaca: AR Graphics Extensions for Web Applications. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 174–183. https://doi.org/10.1109/VR46266.2020.00036Google ScholarCross Ref
- Chidinma U. Kalu, Stephen B. Gilbert, Jonathan W. Kelly, and Melynda Hoover. 2021. Translating Virtual Reality Research into Practice as a Way to Combat Misinformation: The DOVE Website. In HCI International 2021 - Late Breaking Posters, Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa (Eds.). Springer International Publishing, Cham, 341–348.Google Scholar
- Chidinma U. Kalu, Stephen B. Gilbert, Jonathan W. Kelly, and Melynda Hoover. 2021. Translating Virtual Reality Research into Practice as a Way to Combat Misinformation: The DOVE Website. In HCI International 2021 - Late Breaking Posters, Constantine Stephanidis, Margherita Antona, and Stavroula Ntoa (Eds.). Springer International Publishing, Cham, 341–348.Google Scholar
- Rabimba Karanjai. 2018. Optimizing Web Virtual Reality. Ph. D. Dissertation. Rice University.Google Scholar
- Sam Kavanagh, Andrew Luxton-Reilly, Burkhard Wuensche, and Beryl Plimmer. 2017. A systematic review of virtual reality in education. Themes in Science and Technology Education 10, 2 (2017), 85–119.Google Scholar
- Antonio La Salandra, Piero Fraternali, and Darian Frajberg. 2018. A Location-Based Virtual Reality Application for Mountain Peak Detection. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 1206–1214. https://doi.org/10.1145/3184558.3191559Google ScholarDigital Library
- Zeqi Lai, Y. Charlie Hu, Yong Cui, Linhui Sun, Ningwei Dai, and Hung-Sheng Lee. 2020. Furion: Engineering High-Quality Immersive Virtual Reality on Today’s Mobile Devices. IEEE Transactions on Mobile Computing 19, 7 (2020), 1586–1602. https://doi.org/10.1109/TMC.2019.2913364Google ScholarCross Ref
- Kit Yung Lam, Lik Hang Lee, and Pan Hui. 2021. A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web Browser. In Proceedings of the 29th ACM International Conference on Multimedia (Virtual Event, China) (MM ’21). Association for Computing Machinery, New York, NY, USA, 2447–2455. https://doi.org/10.1145/3474085.3475413Google ScholarDigital Library
- Tengpeng Li, Nam Son Nguyen, Xiaoqian Zhang, Teng Wang, and Bo Sheng. 2019. PROMAR: Practical Reference Object-Based Multi-User Augmented Reality (Poster). In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 531–532. https://doi.org/10.1145/3307334.3328610Google ScholarDigital Library
- Vasiliki Liagkou, Dimitrios Salmas, and Chrysostomos Stylios. 2019. Realizing virtual reality learning environment for industry 4.0. Procedia Cirp 79 (2019), 712–717.Google ScholarCross Ref
- Iulia Marneanu, Martin Ebner, and Thomas Rößler. 2014. Evaluation of Augmented Reality Frameworks for Android Development. International Journal of Interactive Mobile Technologies (iJIM) 8 (10 2014), 37–44. https://doi.org/10.3991/ijim.v8i4.3974Google ScholarCross Ref
- Rohit Mehra, Vibhu Saujanya Sharma, Vikrant Kaulgud, Sanjay Podder, and Adam P. Burden. 2020. Towards Immersive Comprehension of Software Systems Using Augmented Reality - An Empirical Evaluation. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1267–1269.Google Scholar
- Michael Oduor and Timo Perälä. 2021. Interactive Urban Play to Encourage Active Mobility: Usability Study of a Web-Based Augmented Reality Application. Frontiers in Computer Science 3 (2021). https://doi.org/10.3389/fcomp.2021.706162Google ScholarCross Ref
- Wayne Piekarski and Bruce Thomas. 2002. ARQuake: the outdoor augmented reality gaming system. Commun. ACM 45, 1 (2002), 36–38.Google ScholarDigital Library
- Siddhant Prakash, Alireza Bahremand, Linda D. Nguyen, and Robert LiKamWa. 2019. GLEAM – An Illumination Estimation Framework for Real-Time Photorealistic Augmented Reality on Mobile Devices (Demo). In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 659–660. https://doi.org/10.1145/3307334.3328570Google ScholarDigital Library
- Xiuquan Qiao, Pei Ren, Schahram Dustdar, Ling Liu, Huadong Ma, and Junliang Chen. 2019. Web AR: A Promising Future for Mobile Augmented Reality—State of the Art, Challenges, and Insights. Proc. IEEE 107, 4 (2019), 651–666. https://doi.org/10.1109/JPROC.2019.2895105Google ScholarCross Ref
- Arunkumar Ravichandran, Ish Kumar Jain, Rana Hegazy, Teng Wei, and Dinesh Bharadia. 2018. Facilitating Low Latency and Reliable VR over Heterogeneous Wireless Networks. In Proceedings of the 24th Annual International Conference on Mobile Computing and Networking (New Delhi, India) (MobiCom ’18). Association for Computing Machinery, New York, NY, USA, 723–725. https://doi.org/10.1145/3241539.3267781Google ScholarDigital Library
- Khyrina Airin Fariza Abu Samah, Mohd Nor Hajar Hasrol Jono, Al-Afiq Che Mohamad Zulkepli, Noor Afni Deraman, Shahadan Saad, Alya Geogiana Buja, and Lala Septem Riza. 2021. Immersive Virtual Reality in Preserving Historical Tourism. In 2021 IEEE 11th International Conference on System Engineering and Technology (ICSET). 234–239. https://doi.org/10.1109/ICSET53708.2021.9612577Google ScholarCross Ref
- Jiacheng Shang, Si Chen, Jie Wu, and Shu Yin. 2022. ARSpy: Breaking Location-Based Multi-Player Augmented Reality Application for User Location Tracking. IEEE Transactions on Mobile Computing 21, 2 (feb 2022), 433–447. https://doi.org/10.1109/TMC.2020.3007740Google ScholarDigital Library
- Shu Shi, Michael Hwang, Varun Gupta, and Rittwik Jana. 2019. Latency Adaptive Streaming of 8K 360 Degree Video to Mobile VR Headsets (Demo). In Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services (Seoul, Republic of Korea) (MobiSys ’19). Association for Computing Machinery, New York, NY, USA, 651–652. https://doi.org/10.1145/3307334.3328566Google ScholarDigital Library
- Hong-Han Shuai, Yueh-Hsue Li, Chun-Chieh Feng, and Wen-Chih Peng. 2018. Four-Dimensional Shopping Mall: Sequential Group Willingness Optimization under VR Environments. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 131–134. https://doi.org/10.1145/3184558.3186961Google ScholarDigital Library
- Xiang Su, Jacky Cao, and Pan Hui. 2020. 5G Edge Enhanced Mobile Augmented Reality. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (London, United Kingdom) (MobiCom ’20). Association for Computing Machinery, New York, NY, USA, Article 64, 3 pages. https://doi.org/10.1145/3372224.3417315Google ScholarDigital Library
- Shih-Wei Sun and Yi-Shan Lan. 2019. Augmented Reality Displaying Scheme in a Smart Glass Based on Relative Object Positions and Orientation Sensors. World Wide Web 22, 3 (may 2019), 1221–1239. https://doi.org/10.1007/s11280-018-0592-zGoogle ScholarDigital Library
- Kevin S Tang, Derrick L Cheng, Eric Mi, and Paul B Greenberg. 2020. Augmented reality in medical education: a systematic review. Canadian medical education journal 11, 1 (2020), e81.Google Scholar
- Thi Tran, Gilles Foucault, and Romain Pinquié. 2022. Benchmarking of 3D Modelling in Virtual Reality. Computer-Aided Design and Applications 19 (03 2022), 1184–1190. https://doi.org/10.14733/cadaps.2022.1184-1190Google ScholarCross Ref
- Chao Wang, Shuanq Lianq, and Jinyuan Jia. 2018. Immersing Web3D Furniture into Real Interior Images. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 721–722. https://doi.org/10.1109/VR.2018.8446341Google ScholarCross Ref
- Ziming Wu, Jiabin Guo, Shuangli Zhang, Chen Zhao, and Xiaojuan Ma. 2019. An AR Benchmark System for Indoor Planar Object Tracking. In 2019 IEEE International Conference on Multimedia and Expo (ICME). 302–307. https://doi.org/10.1109/ICME.2019.00060Google ScholarCross Ref
- Jingao Xu, Guoxuan Chi, Zheng Yang, Danyang Li, Qian Zhang, Qiang Ma, and Xin Miao. 2021. FollowUpAR: Enabling Follow-up Effects in Mobile AR Applications. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (Virtual Event, Wisconsin) (MobiSys ’21). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3458864.3467675Google ScholarDigital Library
- Jackie (Junrui) Yang, Tuochao Chen, Fang Qin, Monica S. Lam, and James A. Landay. 2022. HybridTrak: Adding Full-Body Tracking to VR Using an Off-the-Shelf Webcam. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 348, 13 pages. https://doi.org/10.1145/3491102.3502045Google ScholarDigital Library
- Zhijian Yang, Yu-Lin Wei, Sheng Shen, and Romit Roy Choudhury. 2020. Ear-AR: Indoor Acoustic Augmented Reality on Earphones. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (London, United Kingdom) (MobiCom ’20). Association for Computing Machinery, New York, NY, USA, Article 56, 14 pages. https://doi.org/10.1145/3372224.3419213Google ScholarDigital Library
- Juheon Yi and Youngki Lee. 2020. Heimdall: Mobile GPU Coordination Platform for Augmented Reality Applications. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (London, United Kingdom) (MobiCom ’20). Association for Computing Machinery, New York, NY, USA, Article 35, 14 pages. https://doi.org/10.1145/3372224.3419192Google ScholarDigital Library
- Yiqin Zhao and Tian Guo. 2021. Xihe: A 3D Vision-Based Lighting Estimation Framework for Mobile Augmented Reality. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (Virtual Event, Wisconsin) (MobiSys ’21). Association for Computing Machinery, New York, NY, USA, 28–40. https://doi.org/10.1145/3458864.3467886Google ScholarDigital Library
Index Terms
- Demystifying Mobile Extended Reality in Web Browsers: How Far Can We Go?
Recommendations
A2W: Context-Aware Recommendation System for Mobile Augmented Reality Web Browser
MM '21: Proceedings of the 29th ACM International Conference on MultimediaAugmented Reality (AR) offers new capabilities for blurring the boundaries between physical reality and digital media. However, the capabilities of integrating web contents and AR remain underexplored. This paper presents an AR web browser with an ...
Moving Deep Learning into Web Browser: How Far Can We Go?
WWW '19: The World Wide Web ConferenceRecently, several JavaScript-based deep learning frameworks have emerged, making it possible to perform deep learning tasks directly in browsers. However, little is known on what and how well we can do with these frameworks for deep learning in ...
Towards Augmented Reality Applications in a Mobile Web Context
NGMAST '14: Proceedings of the 2014 Eighth International Conference on Next Generation Mobile Apps, Services and TechnologiesMobile augmented reality (MAR) applications are becoming reality due to the rapid evolution of mobile devices and their capacities. In this paper, we introduce MARW, a development framework for MAR applications on a web browser. Use of the web browser ...
Comments