Skip to main content
SpringerLink
Log in

Real-time camera orientation estimation based on vanishing point tracking under Manhattan World assumption

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper proposes a real-time pipeline for estimating the camera orientation based on vanishing points for indoor navigation assistance on a Smartphone. The orientation of embedded camera relies on the ability to find a reliable triplet of orthogonal vanishing points. The proposed pipeline introduces a novel sampling strategy among finite and infinite vanishing points with a random sample consensus-based line clustering and a tracking along a video sequence to enforce the accuracy and the robustness by extracting the three most pertinent orthogonal directions while preserving a short processing time for real-time application. Experiments on real images and video sequences acquired with a Smartphone show that the proposed strategy for selecting orthogonal vanishing points is pertinent as our algorithm gives better results than the recently published RNS optimal method, in particular for the yaw angle, which is actually essential for the navigation task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Antone, M., Teller, S.: Automatic recovery of relative camera rotations for urban scene. In: Proceedings of the of the IEEE Conference on computer vision and pattern recognition (CVPR), Hilton Head Island, South Carolina, 13–15 June 2000. IEEE Computer Society (2000)

  2. Barinova, O., Lempitsky, V., Tretiakand, E., Kohli P.: Geometric image parsing in man-made environments. In: Proceedings of the 11th European Conference on computer vision (ECCV), Crete, Greece, 5–11 September 2010. Lecture Notes in Computer Science, vol. 6312. Springer, Berlin Heidelberg (2010)

  3. Barnard, S.T.: Interpreting perspective images. Artif. Intell. 21(4), 435–462 (1983). (Elsevier Science B.V)

    Article  Google Scholar 

  4. Bazin, J.C., Pollefeys, M.: 3-line RANSAC for orthogonal vanishing point detection. In: Proceedings of IEEE/RSJ International Conference on intelligent robots and systems, IROS 2012, Vilamoura, Algarve, Portugal, 7–12 October (2012)

  5. Bazin, J.C., Seo, Y., Demonceaux, C., Vasseur, P., Ikeuchi, K., Kweon, I., Pollefeys, M.: Globally optimal line clustering and vanishing points estimation in a Manhattan world. In: Proceedings of the IEEE International Conference on computer vision and pattern recognition (CVPR), Providence, Rhode Island, USA, 16–21 June 2012. IEEE Computer Society (2012)

  6. Boulanger, K., Bouatouch, K., Pattanaik, S.: ATIP: a tool for 3D navigation inside a single image with automatic camera calibration. In: Proceedings of the EG UK Conference on theory and practice of computer graphics, Middlesbrough, UK, 20–22 June (2006)

  7. Cantoni, V., Lombardi, L., Porta, M., Sicard, N.: Vanishing point detection: representation analysis and new approaches. In: Proceedings of the IEEE International Conference on image analysis and processing (ICIAP), 26–28 September 2001, Palermo, Italy 2001. IEEE Computer Society (2001)

  8. Collins, R.T., Weiss, R.S.: Vanishing point calculation as statistical inference on the unit sphere. In: Proceedings of the 3rd International Conference on computer vision (ICCV), Osaka, Japan, 4–7 December, 1990. IEEE (1990)

  9. Coughlan, J.M., Yuille, A.L.: Manhattan world: compass direction from a single image by Bayesian inference. In: Proceedings of the 7th IEEE International Conference on computer vision (ICCV), Kerkyra, Greece, 20–27 September 1999. IEEE (1999)

  10. Denis, P., Elder, J.H., Estrada, F.: Efficient edge-based methods for estimating Manhattan frames in urban imagery. In: Proceedings of the 10th European Conference on computer vision (ECCV), Marseille, France, 12–18 October 2008. Lecture Notes in Computer Science, vol. 5303. Springer, Berlin Heidelberg (2008)

  11. Elloumi, W., Treuillet, S., Leconge, R.: Tracking orthogonal vanishing points in video sequences for a reliable camera orientation in Manhattan world. In: Proceedings of the 5th International Congress on image and signal processing (CISP 2012), Chongqing, China, 16–18 October (2012)

  12. Fisher, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981). doi:10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  13. Förstner, W.: Optimal vanishing point detection and rotation estimation of single images from a Legoland scene. In: Proceedings of the ISPRS symposium commission III PCV. S, Paris, 1–3 September (2010)

  14. Hide, C., Moore, T., Botterill, T.: Low cost vision-aided IMU for pedestrian navigation. J. Glob. Pos. Syst. 10(1), 3–10 (2011). doi:10.5081/jgps.10.1.3

    Article  Google Scholar 

  15. Hwangbo, M., Kanade, T.: Visual-inertial UAV attitude estimation using urban scene regularities. In: Proceedings of the IEEE International Conference on robotics and automation (ICRA 2011), Shanghai, China, 9–13 May, 2011. IEEE (2011)

  16. Kalantari, M., Hashemi, A., Jung, F., Guédon, J.P.: A new solution to the relative orientation problem using only 3 points and the vertical direction. J. Math. Imaging Vis. Arch. 39(3), 259–268 (2011). doi: 10.1007/s10851-010-0234-2

  17. Kalantari, M., Jung, F., Guédon, J.P.: Precise, automatic and fast method for vanishing point detection. Photogramm. Rec. 24(127), 246–263 (2009)

    Article  Google Scholar 

  18. Kosecka, J., Zhang, W.: Video compass. In: Proceedings of the 7th European Conference on computer vision (ECCV), Copenhagen, Denmark, 27 May–2 June 2002. Lecture Notes in Computer Science, vol. 2353. Springer, Berlin Heidelberg (2002)

  19. Lutton, E., Maitre, H., Lopez-Krahe, J.: Contribution to the determination of vanishing points using Hough transform. IEEE Trans. Pattern Anal. Mach. Intell. 16(4), 430–438 (1994)

    Article  Google Scholar 

  20. Martins, A., Aguiar, P., Figueiredo, M.: Orientation in Manhattan world: equiprojective classes and sequential estimation. IEEE Trans. Pattern Anal. Mach. Intell. 27, 822–826 (2005)

    Article  Google Scholar 

  21. Matas, J., Galambos, C., Kittler, J.V.: Robust detection of lines using the progressive probabilistic Hough transform. J. Computer Vis. Image Underst.—Spec. Issue Robust Stat. Tech. Image Underst. 78(1), 119–137 (2000)

    Article  Google Scholar 

  22. Micusik, B., Wildenauer, H., Vincze, M.: Towards detection of orthogonal planes in monocular images of indoor environments. In: Proceedings of the IEEE International Conference on robotics and automation (ICRA 2008), Pasadena, California, USA, 19–23 May, 2008. IEEE (2008)

  23. Mingawa, A., Tagawa, N., Moriya, T., Gotoh, T.: Vanishing point and vanishing line estimation with line clustering. IEICE Trans. Inf. Syst. E83–D(7), 1574–1582 (2000)

  24. Mirzaei, F.M., Roumeliotis, S.I.: Optimal estimation of vanishing points in a Manhattan world. In: Proceedings of the 13th IEEE International Conference on computer vision (ICCV), Barcelona, Spain, 6–13 November 2011. IEEE (2011)

  25. Nieto, M., Salgado, L.: Simultaneous estimation of vanishing points and their converging lines using the EM algorithm. Pattern Recogn. Lett. 32(14), 1691–1700 (2011)

    Article  Google Scholar 

  26. Pflugfelder, R., Bischof, H.: Online auto-calibration in man-made world. In: Proceedings of the digital image computing: techniques and applications (DICTA), Cairns, Australia. 6–8 December 2005. IEEE Computer Society (2005)

  27. Rother, C.: A new approach for vanishing point detection in architectural environments. In: Proceedings of the 11th British machine vision conference (BMVC), University of Bristol, 11–14 September (2000)

  28. Saurer, O., Fraundorfer, F., Pollefeys, M.: Visual localization using global visual features and vanishing points. In: Proceedings of the CLEF (notebook papers/LABs/workshops), Padua, Italy, 20–23 September (2010)

  29. Shufelt, J.A.: Performance evaluation and analysis of vanishing point detection techniques. IEEE Trans. Pattern Anal. Mach. Intell. 21(3), 282–288 (1999)

    Article  Google Scholar 

  30. Tardif, J.-P.: Non-iterative approach for fast and accurate vanishing point detection. In: Proceedings of the 12th IEEE International Conference on computer vision (ICCV), Kyoto, Japan, 27 September–4 October 2009. IEEE (2009)

  31. Tuytelaars, T., Gool, L.J.V., Proesmans, M., Moons, T.: A cascaded Hough transform as an aid in aerial image interpretation. In: Proceedings of the 6th IEEE International Conference on computer vision (ICCV), Bombay, India, 4–7 January 1998. IEEE Computer Society (1998)

  32. Wildenauer, H., Vincze, M.: Vanishing point detection in complex man-made worlds. In: Proceedings of the International Conference on image analysis and processing (ICIAP), Modena, Palazzo Ducale, 10–13 September (2007)

Download references

Acknowledgments

This study is supported by HERON Technologies SAS and the Conseil Général du LOIRET. The authors gratefully acknowledge the contribution of Kamel Guissous and Aladine Chetouani for their help concerning Smartphone coding.

Author information

Authors and Affiliations

  1. Laboratoire PRISME, Université d’Orléans, 12 rue de Blois, 45067, Orléans Cedex, France

    Wael Elloumi, Sylvie Treuillet & Rémy Leconge

Authors
  1. Wael Elloumi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sylvie Treuillet
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rémy Leconge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wael Elloumi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (AVI 24861 kb)

Supplementary material 2 (AVI 25861 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Elloumi, W., Treuillet, S. & Leconge, R. Real-time camera orientation estimation based on vanishing point tracking under Manhattan World assumption. J Real-Time Image Proc 13, 669–684 (2017). https://doi.org/10.1007/s11554-014-0419-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0419-9

Keywords

Search

Navigation