Skip to main content
SpringerLink
Log in

An efficient and globally optimal method for camera pose estimation using line features

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

The accurate estimation of camera pose using numerous line correspondences in real time is a challenging task. This paper presents a non-iterative approach to solve the Perspective-n-Line (PnL) problem. The method can provide high speed and global optimality, as well as linear complexity. A nonlinear least squares (non-LLS) objective function is first formulated by parameterizing the rotation matrix with Cayley representation. A system of three third-order equations is then derived from its optimality conditions, and then, it is solved directly based on the Gröbner basis technique. Finally, the camera pose can be easily obtained by back-substitution. A major advantage of the proposed method lies in scalability, as the size of the elimination template used in the Gröbner basis technique is independent to the number of line correspondences. Extensive and detailed experiments on synthetic data and real images are conducted, demonstrating that the proposed method achieves an accuracy that is equivalent or superior to the leading methods, but with reduced computational requirements. The source code is available at https://github.com/dannyshin1/danny/tree/master/OPnL1.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://www.umn.edu/~joel.

  2. http://www.robots.ox.ac.uk/vgg/data/data-mview.html.

References

  1. Abdelaziz, Y.I.: Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. In: ASP Symposium on Close-Range Photogrammetry (1971)

  2. Ansar, A., Daniilidis, K.: Linear pose estimation from points or lines. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 578–589 (2003)

    Article  Google Scholar 

  3. Bonnal, C., Ruault, J.M., Desjean, M.C.: Active debris removal: recent progress and current trends. Acta Astronaut. 85, 51–60 (2013)

    Article  Google Scholar 

  4. Brezov, D.S., Mladenova, C.D., Mladenov, I.M.: New perspective on the gimbal lock problem. In: AIP Conference Proceedings 1570(1), 367–374 (2013)

  5. Bronson, R., Costa, G.B.: An Introduction to Optimization. Wiley, Hoboken (2009)

    Google Scholar 

  6. Chen, H.H.: Pose determination from line-to-plane correspondences: existence condition and closed-form solutions. IEEE Trans. Pattern Anal. Mach. Intell. 6, 530–541 (1991)

    Article  Google Scholar 

  7. Cropp, A., Palmer, P., Underwood, C.I.: Pose estimation of target satellite for proximity operations. In: 14th Annual AIAA/USU Conference on Small Satellites (2000)

  8. David, P., DeMenthon, D., Duraiswami, R., Samet, H.: Simultaneous pose and correspondence determination using line features. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 2, pp. II–II. IEEE (2003)

  9. Dhome, M., Richetin, M., Lapreste, J.T., Rives, G.: Determination of the attitude of 3d objects from a single perspective view. IEEE Trans. Pattern Anal. Mach. Intell. 11(12), 1265–1278 (1989)

    Article  Google Scholar 

  10. Dong, G., Zhu, Z.: Incremental inverse kinematics based vision servo for autonomous robotic capture of non-cooperative space debris. Adv. Space Res. 57(7), 1508–1514 (2016)

    Article  Google Scholar 

  11. Ferraz, L., Binefa, X., Moreno-Noguer, F.: Very fast solution to the PnP problem with algebraic outlier rejection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 501–508. IEEE (2014)

  12. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  13. Gander, W.: Least Squares Fit of Point Clouds. Springer, Berlin (1997)

    Book  Google Scholar 

  14. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    MATH  Google Scholar 

  15. Hesch, J.A., Roumeliotis, S.I.: A direct least-squares (DLS) method for PnP. In: International Conference on Computer Vision, pp. 383–390. IEEE (2011)

  16. Kelsey, J.M., Byrne, J., Cosgrove, M., Seereeram, S., Mehra, R.K.: Vision-based relative pose estimation for autonomous rendezvous and docking. In: 2006 IEEE Aerospace Conference, pp. 20–pp. IEEE (2006)

  17. Kneip, L., Li, H., Seo, Y.: UPnP: an optimal O(n) solution to the absolute pose problem with universal applicability. In: European Conference on Computer Vision, pp. 127–142. Springer (2014)

  18. Kubota, T., Sawai, S., Hashimoto, T., Kawaguchi, J.: Robotics and autonomous technology for asteroid sample return mission. In: ICAR’05. Proceedings., 12th International Conference on Advanced Robotics, 2005., pp. 31–38. IEEE (2005)

  19. Kukelova, Z., Bujnak, M., Pajdla, T.: Automatic generator of minimal problem solvers. In: European Conference on Computer Vision, pp. 302–315. Springer (2008)

  20. Kumar, R., Hanson, A.: Robust methods for estimating pose and a sensitivity analysis. Comput. Vis. Image Underst. 60(3), 313–342 (1994)

    Article  Google Scholar 

  21. Larsson, V., Kukelova, Z., Zheng, Y.: Making minimal solvers for absolute pose estimation compact and robust. In: International Conference on Computer Vision, pp. 2335–2343. IEEE (2017)

  22. Lepetit, V., Moreno-Noguer, F., Pascal, F.: EPnP: An accurate O(n) solution to the PnP problem. Int. J. Comput. Vision 81(2), 155–166 (2009)

    Article  Google Scholar 

  23. Li, S., Xu, C., Xie, M.: A robust O(n) solution to the perspective-n-point problem. IEEE Trans. Pattern Anal. Mach. Intell. 34(7), 1444–1450 (2012)

    Article  Google Scholar 

  24. Liu, Y., Huang, T.S., Faugeras, O.D.: Determination of camera location from 2D to 3D line and point correspondences. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 82–88. IEEE (1988)

  25. Mirzaei, F.M., Roumeliotis, S.I.: Globally optimal pose estimation from line correspondences. In: International Conference on Robotics and Automation, pp. 5581–5588. IEEE (2011)

  26. Mourikis, A.I., Trawny, N., Roumeliotis, S.I., Johnson, A.E., Ansar, A., Matthies, L.: Vision-aided inertial navigation for spacecraft entry, descent, and landing. IEEE Trans. Rob. 25(2), 264–280 (2009)

    Article  Google Scholar 

  27. Nakano, G.: Globally optimal DLS method for PnP problem with Cayley parameterization. In: British Machine Vision Conference, pp. 78.1–78.11 (2015)

  28. Opromolla, R., Fasano, G., Rufino, G., Grassi, M.: A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations. Prog. Aerosp. Sci. 93, 53–72 (2017)

    Article  Google Scholar 

  29. Přibyl, B., Zemčĺk, P., Čadĺk, M.: Camera pose estimation from lines using plücker coordinates. In: British Machine Vision Conference, pp. 1–12 (2015)

  30. Silva, M.S., Ferreira, R., Gaspar, J.: Camera calibration using a color-depth camera: points and lines based DLT including radial distortion. In: International Conference on Intelligent Robots and Systems. IEEE/RSJ (2012)

  31. Urban, S., Leitloff, J., Hinz, S.: Mlpnp -a real-time maximum likelihood solution to the perspective-n-point problem. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 3(3), 131–138 (2016)

    Article  Google Scholar 

  32. Vakhitov, A., Funke, J., Moreno-Noguer, F.: Accurate and linear time pose estimation from points and lines. In: European Conference on Computer Vision, pp. 583–599. Springer (2016)

  33. Wang, P., Xu, G., Cheng, Y., Yu, Q.: A simple, robust and fast method for the perspective-n-point problem. Pattern Recogn. Lett. 108, 31–37 (2018)

    Article  Google Scholar 

  34. Wang, P., Xu, G., Cheng, Y., Yu, Q.: Camera pose estimation from lines: a fast, robust and general method. Mach. Vis. Appl. 25(5), 603–614 (2019)

    Article  Google Scholar 

  35. Xu, C., Zhang, L., Cheng, L., Koch, R.: Pose estimation from line correspondences: a complete analysis and a series of solutions. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1209–1222 (2017)

    Article  Google Scholar 

  36. Yan, K., Zhao, R., Tian, H., Liu, E., Zhang, Z.: A high accuracy method for pose estimation based on rotation parameters. Measurement 122, 392–401 (2018)

    Article  Google Scholar 

  37. Zhang, L., Xu, C., Lee, K.M., Koch, R.: Robust and efficient pose estimation from line correspondences. In: Asian Conference on Computer Vision, pp. 217–230 (2012)

  38. Zhang, X., Zhang, Z., Li, Y., Zhu, X., Yu, Q., Ou, J.: Robust methods for estimating pose and a sensitivity analysis. Appl. Opt. 51, 936–948 (2012)

    Article  Google Scholar 

  39. Zheng, Y., Kuang, Y., Sugimoto, S., Åström, K., Okutomi, M.: Revisiting the PnP problem: A fast, general and optimal solution. In: International Conference on Computer Vision, pp. 2344–2351. IEEE (2013)

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61473148) and the China Scholarship Council (No. 201906830092).

Author information

Authors and Affiliations

  1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Qida Yu & Guili Xu

  2. College of Astronautics, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China

    Yuehua Cheng

Authors
  1. Qida Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guili Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuehua Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guili Xu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, Q., Xu, G. & Cheng, Y. An efficient and globally optimal method for camera pose estimation using line features. Machine Vision and Applications 31, 48 (2020). https://doi.org/10.1007/s00138-020-01100-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-020-01100-6

Keywords

Search

Navigation