Skip to main content
SpringerLink
Log in

Optimizing PnP-Algorithms for Limited Point Correspondences Using Spatial Constraints

  • Conference paper
  • First Online:
Advances in Visual Computing (ISVC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14362))

Included in the following conference series:

  • 354 Accesses

Abstract

Pose Estimation is an important component of many real-world computer vision systems. Most existing pose estimation algorithms need a large number of point correspondences to accurately determine the pose of an object. Since the number of point correspondences depends on the object’s appearance, lighting and other external conditions, detecting many points may not be feasible. In many real-world applications, movement of objects is limited due to gravity. Hence, detecting objects with only three degrees of freedom is usually sufficient. This allows us to improve the accuracy of pose estimation by changing the underlying equation of the perspective-n-point problem to allow only three variables instead of six. By using the improved equations, our algorithm is more robust against detection errors with limited point correspondences. In this paper, we specify two scenarios where such constraints apply. The first one is about parking a vehicle on a specific spot, while the second scenario describes a camera observing objects from a birds-eye view. In both scenarios, objects can only move in the ground plane and rotate around the vertical axis. Experiments with synthetic data and real-world photographs have shown that our algorithm outperforms state-of-the-art pose estimation algorithms. Depending on the scenario, our algorithm usually achieves 50% better accuracy, while being equally fast.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bujnak, M., Kukelova, Z., Pajdla, T.: A general solution to the P4P problem for camera with unknown focal length. In: CVPR (2008). https://doi.org/10.1109/CVPR.2008.4587793

  2. Burke, J.V., Ferris, M.C.: A gauss–newton method for convex composite optimization. Math. Program. 71, 179–194 (1995). https://doi.org/10.1007/BF01585997

    Article  MathSciNet  MATH  Google Scholar 

  3. Chen, J., Zhang, L., Liu, Y., Xu, C.: Survey on 6D pose estimation of rigid object. In: CCC (2020). https://doi.org/10.23919/CCC50068.2020.9189304

  4. Collins, T., Bartoli, A.: Infinitesimal plane-based pose estimation. Int. J. Comput. Vis. 109(3), 252–286 (2014). https://doi.org/10.1007/s11263-014-0725-5

    Article  MathSciNet  MATH  Google Scholar 

  5. Dhall, A., Dai, D., Van Gool, L.: Real-time 3D traffic cone detection for autonomous driving. In: IV (2019). https://doi.org/10.1109/IVS.2019.8814089

  6. Einsiedler, J., Becker, D., Radusch, I.: External visual positioning system for enclosed carparks. In: WPNC (2014). https://doi.org/10.1109/WPNC.2014.6843287

  7. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 381–395 (1981). https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  8. Fragoso, V., DeGol, J., Hua, G.: gDLS*: generalized pose-and-scale estimation given scale and gravity priors. In: CVPR (2020). https://doi.org/10.1109/CVPR42600.2020.00228

  9. Gao, X.S., Hou, X.R., Tang, J., Cheng, H.F.: Complete solution classification for the perspective-three-point problem. In: TPAMI (2003)

    Google Scholar 

  10. Garro, V., Crosilla, F., Fusiello, A.: Solving the PnP problem with anisotropic orthogonal procrustes analysis. In: 3DIMPVT (2012). https://doi.org/10.1109/3DIMPVT.2012.40

  11. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32, 1231–1237 (2013). https://doi.org/10.1177/0278364913491297

    Article  Google Scholar 

  12. Grafarend, E.W., Shan, J.: Closed-form solution of P4P or the three-dimensional resection problem in terms of Möbius barycentric coordinates. J. Geodesy 71, 217–231 (1997). https://doi.org/10.1007/s001900050089

    Article  MATH  Google Scholar 

  13. Grunert, J.A.: Das Pothenot’sche Problem, in erweiterter Gestalt; nebst Bemerkungen über seine Anwendung in der Geodäsie. Archiv der Mathematik und Physik (1841)

    Google Scholar 

  14. Gu, R., Wang, G., Hwang, J.N.: Efficient multi-person hierarchical 3D pose estimation for autonomous driving. In: MIPR (2019). https://doi.org/10.1109/MIPR.2019.00036

  15. Hagelskjær, F., Savarimuthu, T.R., Krüger, N., Buch, A.G.: Using spatial constraints for fast set-up of precise pose estimation in an industrial setting. In: CASE (2019). https://doi.org/10.1109/COASE.2019.8842876

  16. Hajder, L., Barath, D.: Least-squares optimal relative planar motion for vehicle-mounted cameras. In: ICRA (2020). https://doi.org/10.1109/ICRA40945.2020.9196755

  17. Hesch, J.A., Roumeliotis, S.I.: A direct least-squares (DLS) method for PnP. In: ICCV (2011). https://doi.org/10.1109/ICCV.2011.6126266

  18. IAM, Universität Duisburg-Essen: Taxiladekonzept für Elektrotaxis im öffentlichen Raum. talako.uni-due.de (2022). Accessed 14 Jan 2022

    Google Scholar 

  19. Jiao, Y., et al.: Robust localization for planar moving robot in changing environment: a perspective on density of correspondence and depth. In: ICRA (2021). https://doi.org/10.1109/ICRA48506.2021.9561539

  20. Kim, I.S., Jung, T.W., Jung, K.D.: Augmented reality service based on object pose prediction using PnP algorithm. IJACT 9, 295–301 (2021)

    Google Scholar 

  21. Kim, S.T., Fan, M., Jung, S.W., Ko, S.J.: External vehicle positioning system using multiple fish-eye surveillance cameras for indoor parking lots. IEEE Syst. J. 15, 5107–5118 (2021). https://doi.org/10.1109/JSYST.2020.3019296

    Article  Google Scholar 

  22. Kneip, L., Li, H., Seo, Y.: UPnP: an optimal O(n) solution to the absolute pose problem with universal applicability. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 127–142. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_9

    Chapter  Google Scholar 

  23. Kneip, L., Scaramuzza, D., Siegwart, R.: A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation. In: CVPR (2011). https://doi.org/10.1109/CVPR.2011.5995464

  24. Lee, S., Moon, Y.K.: Camera pose estimation using voxel-based features for autonomous vehicle localization tracking. In: ITC-CSCC (2022). https://doi.org/10.1109/ITC-CSCC55581.2022.9895071

  25. Lee, T.E., et al.: Camera-to-robot pose estimation from a single image. In: ICRA (2020). https://doi.org/10.1109/ICRA40945.2020.9196596

  26. Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate o(n) solution to the PnP problem. IJCV 81, 155–166 (2009). https://doi.org/10.1007/s11263-008-0152-6

    Article  Google Scholar 

  27. Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Q. Appl. Math. 2, 164–168 (1944)

    Article  MathSciNet  MATH  Google Scholar 

  28. Li, C., Wang, X.: On convergence of the gauss-newton method for convex composite optimization. Math. Program. 91, 349–356 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lin, Y., Tremblay, J., Tyree, S., Vela, P.A., Birchfield, S.: Multi-view fusion for multi-level robotic scene understanding. In: IROS (2021). https://doi.org/10.1109/IROS51168.2021.9635994

  30. Lu, X.X.: A review of solutions for perspective-n-point problem in camera pose estimation. In: Journal of Physics: Conference Series (2018). https://doi.org/10.1088/1742-6596/1087/5/052009

  31. Marchand, E., Uchiyama, H., Spindler, F.: Pose estimation for augmented reality: a hands-on survey. TVCG 22, 2633–2651 (2016). https://doi.org/10.1109/TVCG.2015.2513408

    Article  Google Scholar 

  32. Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  33. Martull, S., Peris, M., Fukui, K.: Realistic CG stereo image dataset with ground truth disparity maps. Technical report of IEICE, PRMU (2012)

    Google Scholar 

  34. Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 308–313 (1965). https://doi.org/10.1093/comjnl/7.4.308

    Article  MathSciNet  MATH  Google Scholar 

  35. Ortín, D., Montiel, J.M.M.: Indoor robot motion based on monocular images. Robotica 19, 331–342 (2001). https://doi.org/10.1017/S0263574700003143

    Article  Google Scholar 

  36. Pan, S., Wang, X.: A survey on perspective-n-point problem. In: CCC (2021). https://doi.org/10.23919/CCC52363.2021.9549863

  37. Parameshwara, C.M., Hari, G., Fermüller, C., Sanket, N.J., Aloimonos, Y.: DiffPoseNet: direct differentiable camera pose estimation. In: CVPR (2022). https://doi.org/10.1109/CVPR52688.2022.00672

  38. Persson, M., Nordberg, K.: Lambda twist: an accurate fast robust perspective three point (P3P) solver. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 334–349. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_20

    Chapter  Google Scholar 

  39. Pošík, P., Huyer, W.: Restarted local search algorithms for continuous black box optimization. Evol. Comput. 20, 575–607 (2012). https://doi.org/10.1162/EVCO_a_00087

    Article  Google Scholar 

  40. Qingxuan, J., Ping, Z., Hanxu, S.: The study of positioning with high-precision by single camera based on p3p algorithm. In: ICII (2006). https://doi.org/10.1109/INDIN.2006.275618

  41. Roch, P., Shahbaz Nejad, B., Handte, M., Marrón, P.J.: Car pose estimation through wheel detection. In: Bebis, G., et al. (eds.) ISVC 2021. LNCS, vol. 13017, pp. 265–277. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90439-5_21

    Chapter  Google Scholar 

  42. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: An efficient alternative to sift or surf. In: ICCV (2011). https://doi.org/10.1109/ICCV.2011.6126544

  43. Scaramuzza, D.: 1-point-RANSAC structure from motion for vehicle-mounted cameras by exploiting non-holonomic constraints. IJCV 95, 74–85 (2011). https://doi.org/10.1007/s11263-011-0441-3

    Article  Google Scholar 

  44. Schweighofer, G., Pinz, A.: Globally optimal O(n) solution to the PnP problem for general camera models. In: BMVC (2008)

    Google Scholar 

  45. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: IROS (2012). https://doi.org/10.1109/IROS.2012.6385773

  46. Sweeney, C., Flynn, J., Nuernberger, B., Turk, M., Höllerer, T.: Efficient computation of absolute pose for gravity-aware augmented reality. In: ISMAR (2015). https://doi.org/10.1109/ISMAR.2015.20

  47. Terzakis, G., Lourakis, M.: A consistently fast and globally optimal solution to the perspective-n-point problem. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 478–494. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_28

    Chapter  Google Scholar 

  48. Tremblay, J., To, T., Sundaralingam, B., Xiang, Y., Fox, D., Birchfield, S.: Deep object pose estimation for semantic robotic grasping of household objects. CoRR (2018). https://doi.org/10.48550/arXiv.1809.10790

  49. Urban, S., Leitloff, J., Hinz, S.: MLPnP - a real-time maximum likelihood solution to the perspective-n-point problem. ISPRS (2016). https://doi.org/10.5194/isprs-annals-iii-3-131-2016

  50. Velichkovsky, B.M., Kotov, A., Arinkin, N., Zaidelman, L., Zinina, A., Kivva, K.: From social gaze to indirect speech constructions: how to induce the impression that your companion robot is a conscious creature. Appl. Sci. 11, 10255 (2021). https://doi.org/10.3390/app112110255

    Article  Google Scholar 

  51. Wang, Z., Yang, X.: V-head: face detection and alignment for facial augmented reality applications. In: Amsaleg, L., Guðmundsson, G.Þ, Gurrin, C., Jónsson, B.Þ, Satoh, S. (eds.) MMM 2017. LNCS, vol. 10133, pp. 450–454. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-51814-5_40

    Chapter  Google Scholar 

  52. Zhang, B., Zhang, Q., Wang, Y., Tian, Z.: The method of solving the non-coplanar perspective-four-point (P4P) problem. In: CCC (2014). https://doi.org/10.1109/ChiCC.2014.6896771

  53. Zhou, G., Wang, H., Chen, J., Huang, D.: PR-GCN: a deep graph convolutional network with point refinement for 6D pose estimation. In: ICCV (2021). https://doi.org/10.1109/ICCV48922.2021.00279

Download references

Acknowledgments

This research is funded by the Bundesministerium für Wirtschaft und Energie as part of the TALAKO project (“Taxiladekonzept für Elektrotaxis im öffentlichen Raum” tr. “Taxi Charging Concept for Public Spaces”) [18] (grant number 01MZ19002A).

Author information

Authors and Affiliations

  1. University of Duisburg-Essen, Essen, Germany

    Peter Roch, Bijan Shahbaz Nejad, Marcus Handte & Pedro José Marrón

Authors
  1. Peter Roch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bijan Shahbaz Nejad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcus Handte
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Pedro José Marrón
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter Roch .

Editor information

Editors and Affiliations

  1. University of Nevada Reno, Reno, NV, USA

    George Bebis

  2. Google Research, Mountain View, CA, USA

    Golnaz Ghiasi

  3. New York University, New York, USA

    Yi Fang

  4. Ben-Gurion University, Be'er Sheva, Israel

    Andrei Sharf

  5. Microsoft Research, Beijing, China

    Yue Dong

  6. The University of Oklahoma, Norman, OK, USA

    Chris Weaver

  7. University of Maryland, Collage Park, MD, USA

    Zhicheng Leo

  8. University of Central Florida, Orlando, FL, USA

    Joseph J. LaViola Jr.

  9. InnerOptic Technology, Hillsborough, NC, USA

    Luv Kohli

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Roch, P., Shahbaz Nejad, B., Handte, M., Marrón, P.J. (2023). Optimizing PnP-Algorithms for Limited Point Correspondences Using Spatial Constraints. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2023. Lecture Notes in Computer Science, vol 14362. Springer, Cham. https://doi.org/10.1007/978-3-031-47966-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-47966-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-47965-6

  • Online ISBN: 978-3-031-47966-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation