skip to main content
10.1145/3532213.3532304acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccaiConference Proceedingsconference-collections
research-article

Integrating A Deep Learning-based Plane Detector in Mobile AR Systems for Improvement of Plane Detection

Published: 13 July 2022 Publication History

Abstract

With the increasing interest in Augmented Reality (AR) technology and the enhancement of mobile device capability, mobile AR systems become very popular. In mobile AR systems, plane detection, which plays a major role in determining the location of virtual objects, often shows insufficient performance. We address this limitation in mobile AR systems by adopting a Machine Learning (ML)-based plane detector. We specifically develop a hybrid plane detection pipeline in which an ML-based plane detector and the point cloud information obtained by a mobile device are fused, showing its robust performance in plane detection for various environment conditions.

References

[1]
Arkit plane detection. [Online]. https://developer.apple.com/documentation/arkit/configuration_objects/understanding_world_tracking.
[2]
Arcore plane detection. [Online]. https://developers.google.com/ar/develop/fundamentals.
[3]
Chen Liu, Kihwan Kim, Jinwei Gu, Yasutaka Furukawa, and Jan Kautz. Planercnn: 3d plane detection and reconstruction from a single image. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4450–4459, 2019.
[4]
Grace Tsai, Changhai Xu, Jingen Liu, and Benjamin Kuipers. Real-time indoor scene understanding using bayesian filtering with motion cues. In 2011 International Conference on Computer Vision, pages 121–128. IEEE, 2011.
[5]
Anne-Laure Chauve, Patrick Labatut, and Jean-Philippe Pons. Robust piecewise-planar 3d reconstruction and completion from large-scale unstructured point data. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 1261–1268. IEEE, 2010.
[6]
Michael Kaess. Simultaneous localization and mapping with infinite planes. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 4605–4611. IEEE, 2015.
[7]
Jin Zhou and Baoxin Li. Homography-based ground detection for a mobile robot platform using a single camera. In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., pages 4100–4105. IEEE, 2006.
[8]
Chen Liu, Jimei Yang, Duygu Ceylan, Ersin Yumer, and Yasutaka Furukawa. Planenet: Piece-wise planar reconstruction from a single rgb image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2579–2588, 2018.
[9]
Fengting Yang and Zihan Zhou. Recovering 3d planes from a single image via convolutional neural networks. In Proceedings of the European Conference on Computer Vision (ECCV), pages 85–100, 2018.
[10]
Ying Liu, Luyao Geng, Weidong Zhang, Yanchao Gong, and Zhijie Xu. Survey of video based small target detection. Journal of Image and Graphics, 9(4), 2021.
[11]
Florian Spiess, Lucas Reinhart, Norbert Strobel, Dennis Kaiser, Samuel Kounev, and Tobias Kaupp. People detection with depth silhouettes and convolutional neural networks on a mobile robot. Journal of Image and Graphics, 9(4), 2021.
[12]
Ryo Hasegawa, Yutaro Iwamoto, and Yen-Wei Chen. Robust japanese road sign detection and recognition in complex scenes using convolutional neural networks. Journal of Image and Graphics, 8(3):59–66, 2020.
[13]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226–231, 1996.
[14]
Paweł Nowacki and Marek Woda. Capabilities of arcore and arkit platforms for ar/vr applications. In International Conference on Dependability and Complex Systems, pages 358–370. Springer, 2019.
[15]
Record and playback api. [Online]. https://developers.google.com/ar/develop/java/recording-and-playback/introduction.
[16]
Arcore sdk. [Online]. https://github.com/google-ar/arcore-android-sdk.
[17]
Planercnn github. [Online]. https://github.com/NVlabs/planercnn.

Cited By

View all
  • (2024)FetchAid: Making Parcel Lockers More Accessible to Blind and Low Vision People With Deep-learning Enhanced Touchscreen Guidance, Error-Recovery Mechanism, and AR-based Search SupportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642213(1-15)Online publication date: 11-May-2024
  • (2024)PlaneSeg: Building a Plug-In for Boosting Planar Region SegmentationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.326254435:8(11486-11500)Online publication date: Aug-2024
  • (2023)A Comparative Evaluation of Augmented Reality Frameworks: A Plane Mapping and Resource Utilisation Perspective2023 IST-Africa Conference (IST-Africa)10.23919/IST-Africa60249.2023.10187850(1-9)Online publication date: 31-May-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICCAI '22: Proceedings of the 8th International Conference on Computing and Artificial Intelligence
March 2022
809 pages
ISBN:9781450396110
DOI:10.1145/3532213
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Machine Learning
  2. Mobile AR
  3. Plane Detection

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICCAI '22

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)15
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)FetchAid: Making Parcel Lockers More Accessible to Blind and Low Vision People With Deep-learning Enhanced Touchscreen Guidance, Error-Recovery Mechanism, and AR-based Search SupportProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642213(1-15)Online publication date: 11-May-2024
  • (2024)PlaneSeg: Building a Plug-In for Boosting Planar Region SegmentationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.326254435:8(11486-11500)Online publication date: Aug-2024
  • (2023)A Comparative Evaluation of Augmented Reality Frameworks: A Plane Mapping and Resource Utilisation Perspective2023 IST-Africa Conference (IST-Africa)10.23919/IST-Africa60249.2023.10187850(1-9)Online publication date: 31-May-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media