Skip to main content
SpringerLink
Log in

A Geometry-Based Point Cloud Reduction Method for Mobile Augmented Reality System

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, a geometry-based point cloud reduction method is proposed, and a real-time mobile augmented reality system is explored for applications in urban environments. We formulate a new objective function which combines the point reconstruction errors and constraints on spatial point distribution. Based on this formulation, a mixed integer programming scheme is utilized to solve the points reduction problem. The mobile augmented reality system explored in this paper is composed of the offline and online stages. At the offline stage, we build up the localization database using structure from motion and compress the point cloud by the proposed point cloud reduction method. While at the online stage, we compute the camera pose in real time by combining an image-based localization algorithm and a continuous pose tracking algorithm. Experimental results on benchmark and real data show that compared with the existing methods, this geometry-based point cloud reduction method selects a point cloud subset which helps the image-based localization method to achieve higher success rate. Also, the experiments conducted on a mobile platform show that the reduced point cloud not only reduces the time consuming for initialization and re-initialization, but also makes the memory footprint small, resulting a scalable and real-time mobile augmented reality system.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Agarwal S, Snavely N, Simon I, Seitz S M, Szeliski R. Building Rome in a day. In Proc. the 12th IEEE International Conference on Computer Vision, September 2009, pp.72-79.

  2. Arth C, Wagner D, Klopschitz M, Irschara A, Schmalstieg D.Wide area localization on mobile phones. In Proc. the 8th IEEE International Symposium on Mixed and Augmented Reality, October 2009, pp.73-82.

  3. Arth C, Mulloni A, Schmalstieg D. Exploiting sensors on mobile phones to improve wide-area localization. In Proc. the 21st International Conference on Pattern Recognition, November 2012, pp.2152-2156.

  4. Middelberg S, Sattler T, Untzelmann O, Kobbelt L. Scalable 6-DOF localization on mobile devices. In Proc. the 13th European Conference on Computer Vision, September 2014, pp.268-283.

  5. Jeong Y, Bok Y, Kim J S, Kweon I S. Complementation of cameras and lasers for accurate 6D SLAM: From correspondences to bundle adjustment. In Proc. the 2011 IEEE International Conference on Robotics and Automation, May 2011, pp.3581-3588.

  6. Miraldo P, Araujo H. Pose estimation for non-central cameras using planes. Journal of Intelligent & Robotic Systems, 2015, 80(3/4): 595-608.

  7. Miraldo P, Araujo H, Gon¸calves N. Pose estimation for general cameras using lines. IEEE Transactions on Cybernetics, 2015, 45(10): 2156-2164.

  8. Liu H, Zhang G, Bao H. Robust keyframe-based monocular SLAM for augmented reality. In Proc. the 2016 IEEE International Symposium on Mixed and Augmented Reality, September 2016.

  9. Sattler T, Leibe B, Kobbelt L. Fast image-based localization using direct 2D-to-3D matching. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.667-674.

  10. Sattler T, Weyand T, Leibe B, Kobbelt L. Image retrieval for image-based localization revisited. In Proc. the 2012 British Machine Vision Conference, Sept. 2012.

  11. Choudhary S, Indelman V, Christensen H I, Dellaert F. Information-based reduced landmark SLAM. In Proc. the 2015 IEEE International Conference on Robotics and Automation, May 2015, pp.4620-4627.

  12. Torii A, Arandjelović R, Sivic J, Okutomi M, Pajdla T. 24/7 place recognition by view synthesis. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.1808-1817.

  13. Zeisl B, Sattler T, Pollefeys M. Camera pose voting for large-scale image-based localization. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.2704-2712.

  14. Feng Y, Fan L, Wu Y. Fast localization in large-scale environments using supervised indexing of binary features. IEEE Transactions on Image Processing, 2016, 25(1): 343-358.

  15. Campbell D, Petersson L, Kneip L, Li H. Globally-optimal inlier set maximisation for simultaneous camera pose and feature correspondence. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017.

  16. Yang J, Li H, Campbell D, Jia Y. Go-ICP: A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, November 2016, 38(11): 2241-2254.

  17. Li Y, Snavely N, Huttenlocher D P. Location recognition using prioritized feature matching. In Proc. the 11th European Conference on Computer Vision, September 2010, pp.791-804.

  18. Irschara A, Zach C, Frahm J M, Bischof H. From structurefrom-motion point clouds to fast location recognition. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp.2599-2606.

  19. Dong Z, Zhang G, Jia J, Bao H. Efficient keyframe-based real-time camera tracking. Computer Vision and Image Understanding, 2014, 118: 97-110.

  20. Cao S, Snavely N. Minimal scene descriptions from structure from motion models. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.461-468.

  21. Park H S, Wang Y, Nurvitadhi E, Hoe J C, Sheikh Y, Chen M. 3D point cloud reduction using mixed-integer quadratic programming. In Proc. the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2013, pp.229-236.

  22. Leutenegger S, Chli M, Siegwart R. BRISK: Binary robust invariant scalable keypoints. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.2548-2555.

  23. Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91-110.

  24. Rosten E, Drummond T. Machine learning for high-speed corner detection. In Proc. the 9th European Conference on Computer Vision, May 2006, pp.430-443.

  25. Gionis A, Indyk P, Motwani R. Similarity search in high dimensions via hashing. In Proc. the 25th International Conference on Very Large Data Bases, September 1999, pp.518-529.

  26. Chum O, Matas J. Matching with PROSAC — Progressive sample consensus. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Volume 1, June 2005, pp.220-226.

  27. Lepetit V, Moreno-Noguer F, Fua P. EPnP: An accurate O(n) solution to the PnP problem. International Journal of Computer Vision, 2009, 81(2): 155-166.

  28. Takacs G, Chandrasekhar V, Tsai S, Chen D, Grzeszczuk R, Girod B. Unified real-time tracking and recognition with rotation-invariant fast features. In Proc. the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2010, pp.934-941.

  29. Stewart C V. Robust parameter estimation in computer vision. SIAM Review, 1999, 41(3): 513-537.

  30. Hartley R, Zisserman A. Multiple View Geometry in Computer Vision (2nd edition). Cambridge University Press, 2004.

  31. Fischler M A, Bolles R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24(6): 381-395.

Download references

Author information

Authors and Affiliations

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China

    Hao-Ren Wang, Juan Lei, Ao Li & Yi-Hong Wu

  2. University of Chinese Academy of Sciences, Beijing, 100049, China

    Hao-Ren Wang, Juan Lei & Ao Li

Authors
  1. Hao-Ren Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi-Hong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yi-Hong Wu.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 1337 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, HR., Lei, J., Li, A. et al. A Geometry-Based Point Cloud Reduction Method for Mobile Augmented Reality System. J. Comput. Sci. Technol. 33, 1164–1177 (2018). https://doi.org/10.1007/s11390-018-1879-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-018-1879-3

Keywords

Search

Navigation