Skip to main content
SpringerLink
Log in

Real-time camera pose estimation via line tracking

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Real-time camera calibration has been intensively studied in augmented reality. However, for texture-less and texture-repeated scenes as well as poorly illuminated scenes, obtaining a stable calibration is still an open problem. In the paper, we propose a method of calibrating a live video by tracking orthogonal vanishing points. Since vanishing points cannot be obtained directly on the image, the tracking is achieved by tracking parallel lines. This is a changeling problem due to the fact that vanishing points are sensitive to image noise, camera movement, and illumination variation. We tackle the challenges by three optimization procedures and flexible process of degenerated cases. During three optimizations, several explicitly geometric constraints are incorporated, ensuring the calibration result robust to poor illumination and camera movement. A variety of challenging examples demonstrate that the proposed algorithm outperforms state-of-the-art methods for texture-less and texture-repeated scenes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bazin, J.C., Pollefeys, M.: 3-line ransac for orthogonal vanishing point detection. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4282–4287 (2012)

  2. Boulanger, K., Bouatouch, K., Pattanaik, S.: Atip: A tool for 3d navigation inside a single image with automatic camera calibration. In: TPCG, The Eurographics Association, pp. 71–79 (2006)

  3. Cantoni, V., Lombardi, L., Porta, M., Sicard, N.: Vanishing point detection: representation analysis and new approaches. In: Proceedings of the International Conference on Image Analysis and Processing, 2001, pp. 90–94 (2001)

  4. Chen, X., Jia, R., Ren, H., Zhang, Y.: A new vanishing point detection algorithm based on Hough transform. In: Third International Joint Conference on Computational Science and Optimization, pp. 440–443 (2010)

  5. Elloumi, W., Treuillet, S., Leconge, R.: Real-time estimation of camera orientation by tracking orthogonal vanishing points in videos. In: International Conference on Computer Vision Theory and Applications (2013)

  6. Guillou, E., Meneveaux, D., Maisel, E., Bouatouch, K.: Using vanishing points for camera calibration and coarse 3d reconstruction from a single image. Vis. Comput. 16(7), 396–410 (2000)

    Article  MATH  Google Scholar 

  7. He, Q., Chu, C.H.H.: An efficient vanishing point detection by clustering on the normalized unit sphere. In: International Workshop on Computer Architecture for Machine Perception and Sensing, pp. 203–207 (2007)

  8. Hornacek, M., Maierhofer, S.: Extracting vanishing points across multiple views. In: Computer Vision and Pattern Recognition, pp. 953–960 (2010)

  9. Junejo, I., Foroosh, H.: Dissecting the image of the absolute conic. In: IEEE International Conference on Video and Signal Based Surveillance, p. 77 (2006)

  10. Kroeger, T., Dai, D., Gool, L.V.: Joint vanishing point extraction and tracking. In: Computer Vision and Pattern Recognition, pp. 2449–2457 (2015)

  11. Kroeger, T., Dai, D., Timofte, R., Gool, L.V.: Discovery of sets of mutually orthogonal vanishing points in videos. In: Applications and Computer Vision Workshops, pp. 63–70 (2015)

  12. Liu, H., Zhang, G., Bao, H.: Robust keyframe-based monocular slam for augmented reality. In: IEEE International Symposium on Mixed and Augmented Reality, pp. 1–10 (2016)

  13. Liu, Y., Granier, X.: Online tracking of outdoor lighting variations for augmented reality with moving cameras. In: IEEE Educational Activities Department, pp. 573–580 (2012)

  14. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  15. Lu, X., Yao, J., Li, K., Li, L.: Cannylines: a parameter-free line segment detector. In: IEEE International Conference on Image Processing, pp. 507–511 (2015)

  16. Lutton, E., Maitre, H., Lopezkrahe, J.: Contribution to the determination of vanishing points using Hough transform. IEEE Trans. Pattern Anal. Mach. Intell. 16(4), 430–438 (1994)

    Article  Google Scholar 

  17. Mur-Artal, R., Tards, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Rob. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  18. Orghidan, R., Salvi, J., Gordan, M., Orza, B.: Camera calibration using two or three vanishing points. In: Computer Science and Information Systems, pp. 123–130 (2012)

  19. Quan, L., Mohr, R.: Determining perspective structures using hierarchical Hough transform. Pattern Recogn. Lett. 9(4), 279–286 (1989)

    Article  MATH  Google Scholar 

  20. Ran, L., Hua, Z., Liu, M., Xia, X.: Stereo cameras self-calibration based on SIFT. In: International Conference on Measuring Technology and Mechatronics Automation, pp. 352–355 (2009)

  21. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: IEEE International Conference on Computer Vision, pp. 2564–2571 (2012)

  22. Simon, G., Fitzgibbon, A.W., Zisserman, A.: Markerless tracking using planar structures in the scene. In: Proceedings of the International Symposium on Augmented Reality, pp. 120–128 (2000)

  23. Snchez, P.J., Monzn, L.N., de la Salgado, N.A.: Robust optical flow estimation. Image Process. OnLine 3, 252–270 (2013). https://doi.org/10.5201/ipol.2013.21

    Article  Google Scholar 

  24. Straub, J., Rosman, G., Freifeld, O., Leonard, J.J., Fisher, J.W.: A mixture of Manhattan frames: beyond the Manhattan world. In: Computer Vision and Pattern Recognition, pp. 3770–3777 (2014)

  25. Tardif, J.P.: Non-iterative approach for fast and accurate vanishing point detection. In: IEEE International Conference on Computer Vision, pp. 1250–1257 (2009)

  26. Tsai, F., Chang, H.: Detection of vanishing points using Hough transform for single view 3d reconstruction. In: Asian Conference on Remote Sensing, pp. 1182–1189 (2013)

  27. Xu, C., Zhang, L., Li, C., Koch, R.: Pose estimation from line correspondences: a complete analysis and a series of solutions. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1209 (2017)

    Article  Google Scholar 

  28. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

Download references

Funding

This study was funded by National Natural Science Foundation (NSFC) of China (Nos. 61572333, 61472261, 61402081).

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Chengdu, People’s Republic of China

    Yanli Liu, Xianghui Chen, Tianlun Gu & Yanci Zhang

  2. National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, People’s Republic of China

    Yanli Liu, Xianghui Chen, Tianlun Gu & Yanci Zhang

  3. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, People’s Republic of China

    Guanyu Xing

Authors
  1. Yanli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianghui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianlun Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanci Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guanyu Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guanyu Xing.

Ethics declarations

Conflict of interest

All authors declare that they have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Chen, X., Gu, T. et al. Real-time camera pose estimation via line tracking. Vis Comput 34, 899–909 (2018). https://doi.org/10.1007/s00371-018-1523-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-018-1523-9

Keywords

Search

Navigation