skip to main content
10.1145/3463914.3463916acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicvarsConference Proceedingsconference-collections
research-article

Image-Based Localization for Augmented Reality application: A Review

Published: 11 December 2021 Publication History

Abstract

Augmented reality (AR) refers to seamlessly inserted virtual objects into the real world in a real-time way. The real-time requirement can be associated with pose estimation or, equivalently, camera pose localization. Herein, we provide an overview of the camera pose localization domain for AR, explain the pose estimation problem, and provide a survey of relevant image-based localization methods. We highlight the localization problem via feature extraction and matching through mapping the scene and constructing the 3D scene structure.

References

[1]
Wang, X., Ong, S. and Nee, A., 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing, 4(1), pp.1-22.
[2]
Billinghurst, M., Clark, A. and Lee, G., 2015. A Survey of Augmented Reality. Foundations and Trends®in Human–Computer Interaction, 8(2-3),pp.73-272.
[3]
Milgram, P. and Kishino, F., 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems, 77(12),pp.1321-1329.
[4]
Azuma, R., 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, 6(4), pp.355-385.
[5]
Marchand, E., Uchiyama, H. and Spindler, F., 2016. Pose Estimation for Augmented Reality: A Hands-On Survey. IEEE Transactions on Visualization and Computer Graphics, 22(12), pp.2633-2651.
[6]
Rabbi, I. and Ullah, S., 2016. A Survey on Augmented Reality Challenges and Tracking. Acta Graphica, 24(1-2), pp. 29-46.
[7]
Feng Zhou, Duh, H. and Billinghurst, M., 2008. Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR. 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.
[8]
Sicaru, I. A., Ciocianu, C. G., and Boiangiu, C. A., 2017. A SURVEY ON AUGMENTED REALITY. Journal of Information Systems & Operations Management, 11(2), pp. 263-280.
[9]
Wu,Y., Tang,F. and Li,H.,2018. Image-based camera localization: an overview. Visual Computing for Industry, Biomedicine, and Art, 1(1).
[10]
Dabove, P., Pietra V., and Lingua A. M., 2017. Positioning Techniques with Smartphone Technology: Performances and Methodologies in Outdoor and Indoor Scenarios. In Smartphones from an Applied Research Perspective pp. 163-183.
[11]
Yu, L., Ong, S. and Nee, A., 2015. A tracking solution for mobile augmented reality based on sensor-aided marker-less tracking and panoramic mapping. Multimedia Tools and Applications, 75(6), pp.3199-3220.
[12]
Reitmayr, G. and Drummond, T., 2006. Going out: robust model-based tracking for outdoor augmented reality. 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.
[13]
Meyer, J., Schlebusch, T., Fuhl, W. and Kasneci, E., 2020. A Novel Camera-Free Eye Tracking Sensor for Augmented Reality Based on Laser Scanning. IEEE Sensors Journal, 20(24), pp.15204-15212.
[14]
Chen, J., Cao, R. and Wang, Y., 2015. Sensor-Aware Recognition and Tracking for Wide-Area Augmented Reality on Mobile Phones. Sensors, 15(12), pp.31092-31107.
[15]
Feng, Y. and Golparvar-Fard, M., 2019. Image-Based Localization for Facilitating Construction Field Reporting on Mobile Devices. Advances in Informatics and Computing in Civil and Construction Engineering, pp.585-592.
[16]
Taketomi, T., Okada, K., Yamamoto, G., Miyazaki, J. and Kato, H., 2014. Camera pose estimation under dynamic intrinsic parameter change for augmented reality. Computers & Graphics, 44, pp.11-19.
[17]
Jabborov, F. and Cho, J., 2020. Image-Based Camera Localization Algorithm for Smartphone Cameras Based on Reference Objects. Wireless Personal Communications, 114(3), pp.2511-2527.
[18]
Li, J., Wang, C., Kang, X. and Zhao, Q., 2019. Camera localization for augmented reality and indoor positioning: a vision-based 3D feature database approach. International Journal of Digital Earth, 13(6), pp.727-741.
[19]
Gao, Q., Wan, T., Tang, W. and Chen, L., 2017. A Stable and Accurate Marker-Less Augmented Reality Registration Method. 2017 International Conference on Cyberworlds (CW).
[20]
Alaniz II, A. L., and Mantaring, C. M, 2008. Real-Time Camera Pose Estimation for Virtual Reality Navigation. stanford.edu.
[21]
Li, M. and Rottensteiner, F., 2019. VISION-BASED INDOOR LOCALIZATION VIA A VISUAL SLAM APPROACH. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2/W13, pp.827-833.
[22]
Taira, H., 2018. InLoc: Indoor Visual Localization with Dense Matching and View Synthesis. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.
[23]
Yeh, Y. and Lin, H., 2018. 3D Reconstruction and Visual SLAM of Indoor Scenes for Augmented Reality Application. 2018 IEEE 14th International Conference on Control and Automation (ICCA).
[24]
Durrant-Whyte, H. and Bailey, T., 2006. Simultaneous localization and mapping: part I. IEEE Robotics & Automation Magazine, 13(2), pp.99-110.
[25]
Bailey, T. and Durrant-Whyte, H., 2006. Simultaneous localization and mapping (SLAM): part II. IEEE Robotics & Automation Magazine, 13(3), pp.108-117.
[26]
Filipenko, M. and Afanasyev, I., 2018. Comparison of Various SLAM Systems for Mobile Robot in an Indoor Environment. 2018 International Conference on Intelligent Systems (IS).
[27]
Taketomi, T., Uchiyama, H. and Ikeda, S., 2017. Visual SLAM algorithms: a survey from 2010 to 2016. IPSJ Transactions on Computer Vision and Applications, 9(1).
[28]
Liu, J., Xie, Y., Gu, S. and Chen, X., 2019. A SLAM-Based Mobile Augmented Reality Tracking Registration Algorithm. International Journal of Pattern Recognition and Artificial Intelligence, 34(01), p.2054005.
[29]
Syahputra, M., Aulia, M. and Arisandy, D., 2020. Augmented Reality Technologies for Interior Design Planning using a Simultaneous Localization and Mapping Method. IOP Conference Series: Materials Science and Engineering, 851, p.012067.
[30]
Davison, 2003. Real-time simultaneous localisation and mapping with a single camera. Proceedings Ninth IEEE International Conference on Computer Vision, 2.
[31]
Klein, G. and Murray, D., 2007. Parallel Tracking and Mapping for Small AR Workspaces. 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.
[32]
Mur-Artal, R., Montiel, J. and Tardos, J., 2015. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Transactions on Robotics, 31(5), pp.1147-1163.
[33]
Engel, J., Sturm, J. and Cremers, D., 2013. Semi-dense Visual Odometry for a Monocular Camera. 2013 IEEE International Conference on Computer Vision.
[34]
Li, S., Zhang, T., Gao, X., Wang, D. and Xian, Y., 2019. Semi-direct monocular visual and visual-inertial SLAM with loop closure detection. Robotics and Autonomous Systems, 112, pp.201-210.
[35]
Li, P., Li, H. and Chen, B., 2020. A method of Monocular Visual Odometry Combining Feature points and Pixel Gradient for Dynamic Scene. 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring).
[36]
Forster, C., Pizzoli, M. and Scaramuzza, D., 2014. SVO: Fast semi-direct monocular visual odometry. 2014 IEEE International Conference on Robotics and Automation (ICRA).
[37]
Mistretta, F., Sanna, G., Stochino, F. and Vacca, G., 2019. Structure from Motion Point Clouds for Structural Monitoring. Remote Sensing, 11(16), p.1940.
[38]
Iglhaut, J., Cabo, C., Puliti, S., Piermattei, L., O'Connor, J. and Rosette, J., 2019. Structure from Motion Photogrammetry in Forestry: a Review. Current Forestry Reports, 5(3), pp.155-168.
[39]
Yuan, Y., Ding, Y., Zhao, L. and Lv, L., 2018. An Improved Method of 3D Scene Reconstruction Based on SfM. 2018 3rd International Conference on Robotics and Automation Engineering (ICRAE).
[40]
James, M., 2019. Guidelines on the use of structure-from-motion photogrammetry in geomorphic research. Earth Surface Processes and Landforms, 44(10), pp.2081-2084.
[41]
Bang, J., Lee, D., Kim, Y. and Lee, H., 2017. Camera Pose Estimation Using Optical Flow and ORB Descriptor in SLAM-Based Mobile AR Game. 2017 International Conference on Platform Technology and Service (PlatCon).
[42]
Ali, I. H., & Salman, S., 2018. A Performance Analysis of Various Feature Detectors and their Descriptors for Panorama Image Stitching. International Journal of Pure and Applied Mathematics, 119 (15), pp. 147-161.
[43]
Tareen, S. and Saleem, Z., 2018. A comparative analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK. 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET).
[44]
Yan Ke and Sukthankar, R., 2004. PCA-SIFT: a more distinctive representation for local image descriptors. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004.
[45]
Abdel-Hakim, A. and Farag, A., 2006. CSIFT: A SIFT Descriptor with Color Invariant Characteristics. 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR’06).
[46]
Mikolajczyk, K. and Schmid, C., 2005. A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), pp.1615-1630.
[47]
Bosch, A., Zisserman, A. and Munoz, X., 2008. Scene Classification Using a Hybrid Generative/Discriminative Approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(4), pp.712-727.
[48]
van de Sande, K., Gevers, T. and Snoek, C., 2010. Evaluating Color Descriptors for Object and Scene Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(9), pp.1582-1596.
[49]
AI, D., HAN, X., RUAN, X. and CHEN, Y., 2010. Color Independent Components Based SIFT Descriptors for Object/Scene Classification. IEICE Transactions on Information and Systems, E93-D(9), pp.2577-2586.
[50]
Li, Y., Liu, W., Li, X., Huang, Q. and Li, X., 2014. GA-SIFT: A new scale invariant feature transform for multispectral image using geometric algebra. Information Sciences, 281, pp.559-572.
[51]
Bay, H., Ess, A., Tuytelaars, T. and Van Gool, L., 2008. Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding, 110(3), pp.346-359.
[52]
Rosten, E., Porter, R. and Drummond, T., 2010. Faster and Better: A Machine Learning Approach to Corner Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), pp.105-119.
[53]
Calonder, M., Lepetit, V., Strecha, C. and Fua, P., 2010. BRIEF: Binary Robust Independent Elementary Features. Computer Vision – ECCV 2010, pp.778-792.
[54]
Tafti, A., 2016. A comparative study on the application of SIFT, SURF, BRIEF and ORB for 3D surface reconstruction of electron microscopy images. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 6(1), pp.17-30.
[55]
Rublee, E., Rabaud, V., Konolige, K. and Bradski, G., 2011. ORB: An efficient alternative to SIFT or SURF. 2011 International Conference on Computer Vision.
[56]
Leutenegger, S., Chli, M. and Siegwart, R., 2011. BRISK: Binary Robust invariant scalable keypoints. 2011 International Conference on Computer Vision.

Cited By

View all
  • (2024)MeshVPR: Citywide Visual Place Recognition Using 3D MeshesComputer Vision – ECCV 202410.1007/978-3-031-72904-1_19(321-339)Online publication date: 29-Sep-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICVARS '21: Proceedings of the 2021 5th International Conference on Virtual and Augmented Reality Simulations
March 2021
72 pages
ISBN:9781450389327
DOI:10.1145/3463914
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 December 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Augmented reality
  2. Camera localization Pose estimation
  3. Feature detector and descriptor
  4. Image-based localization
  5. Point cloud
  6. SLAM
  7. SfM

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICVARS 2021

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)37
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)MeshVPR: Citywide Visual Place Recognition Using 3D MeshesComputer Vision – ECCV 202410.1007/978-3-031-72904-1_19(321-339)Online publication date: 29-Sep-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media