Skip to main content
SpringerLink
Log in

SECPNet—secondary encoding network for estimating camera parameters

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Camera parameter estimation can be used in visual odometry, robot vision, SLAM, 3D reconstruction and other directions. It is also the main research content of computer vision. Based on the deep learning strategy, we propose a secondary encoding network for camera parameters (SECPNet), which can predict the camera parameters and recover the camera pose according to a single RGB image. Based on the three-dimensional dataset ShapeNet40 (Chang et al. in An information-rich 3D model repository, 2015. arXiv:1512.03012), we build a varifocal multi-viewpoint image dataset for camera parameter estimation. Experimental results show that our method has state-of-the-art performance in camera parameter estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Abdel-Aziz, Y.I., Karara, H.M., Hauck, M.: Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. Photogramm. Eng. Remote Sens. 81(2), 103–107 (2015)

    Article  Google Scholar 

  2. Ardakani, H.K., Mousavinia, A., Safaei, F.: Four points: one-pass geometrical camera calibration algorithm. Visual Comput. 36(2), 413–424 (2020)

    Article  Google Scholar 

  3. Bock, B., Allen, M.: Blender, June 8 2010. US Patent App. 29/350,912

  4. Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6d object pose estimation using 3d object coordinates. In: European Conference on Computer Vision, pp. 536–551. Springer (2014)

  5. Brahmbhatt, S., Gu, J., Kim, K., Hays, J., Kautz, J.: Geometry-aware learning of maps for camera localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2616–2625 (2018)

  6. Cai, B., Wang, Y., Jiajun, W., Wang, M., Li, F., Ma, M., Chen, X., Wang, K.: An effective method for camera calibration in defocus scene with circular gratings. Opt. Lasers Eng. 114, 44–49 (2019)

    Article  Google Scholar 

  7. Chang, A.X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al.: Shapenet: an information-rich 3d model repository (2015). arXiv:1512.03012

  8. Charco, J.L., Vintimilla, B.X, Sappa, A.D: Deep learning based camera pose estimation in multi-view environment. In: 2018 14th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), pp. 224–228. IEEE (2018)

  9. Chen, B., Pan, B.: Camera calibration using synthetic random speckle pattern and digital image correlation. Opt. Lasers Eng. 126, 105919 (2020)

    Article  Google Scholar 

  10. De Ma, S.: A self-calibration technique for active vision systems. IEEE Trans. Robot. Autom. 12(1), 114–120 (1996)

    Article  Google Scholar 

  11. Do, T.-T., Cai, M., Pham, T., Reid, I.: Deep-6dpose: recovering 6d object pose from a single rgb image (2018). arXiv:1802.10367

  12. Duong, N.-D., Kacete, A., Sodalie, C., Richard, P.-Y., Royan, J.: xyznet: towards machine learning camera relocalization by using a scene coordinate prediction network. In: 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 258–263. IEEE (2018)

  13. Fang, Q., Zhao, K., Tang, D., Zhou, Z., Zhou, Y., Hu, T., Zhou, H.: Euler angles based loss function for camera localization with deep learning. In: 2018 IEEE 8th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 61–66. IEEE (2018)

  14. Faugeras, O.D., Luong, Q.T., Maybank, S.J.: Camera self-calibration: Theory and experiments. In: Lecture Notes in Computer Science, vol. 588 (1998)

  15. Frosio, I., Turrini, C., Alzati, A.: Camera re-calibration after zooming based on sets of conics. Visual Comput. 32(5), 663–674 (2016)

    Article  Google Scholar 

  16. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. J. Mach. Learn. Res. 9, 249–256 (2010)

    Google Scholar 

  17. Grabner, A., Roth, P.M , Lepetit, V.: Gp2c: geometric projection parameter consensus for joint 3d pose and focal length estimation in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2222–2231 (2019)

  18. Guan, W., Li, W., Xi, J.: Improved particle swarm optimization algorithm based nonlinear calibration of camera. In: 2017 29th Chinese Control And Decision Conference (CCDC), pp. 5217–5221. IEEE (2017)

  19. Guo, F., He, Y., Guan, L.: Rgb-d camera pose estimation using deep neural network. In: 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP), pp. 408–412. IEEE (2017)

  20. Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. In: Asian Conference on Computer Vision, pp. 548–562. Springer (2012)

  21. Kehl, W., Manhardt, F., Tombari, F., Ilic, S., Navab, N.: Ssd-6d: making rgb-based 3d detection and 6d pose estimation great again. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1521–1529 (2017)

  22. Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5974–5983 (2017)

  23. Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

  24. Li, S., Harada, T., Zou, W.: Estimating relative pose between nonoverlapping cameras by four laser pointers based on general camera model. In: 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 168–172. IEEE (2017)

  25. Liu, Y., Chen, X., Tianlun, G., Zhang, Y., Xing, G.: Real-time camera pose estimation via line tracking. Visual Comput. 34(6–8), 899–909 (2018)

    Article  Google Scholar 

  26. Lu, L., Li, H.: Study of camera calibration algorithm based on spatial perpendicular intersect. In: 2010 2nd International Conference on Signal Processing Systems, vol. 3, pp. V3–V125. IEEE (2010)

  27. Matei, B.C., Meer, P.: Estimation of nonlinear errors-in-variables models for computer vision applications. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1537–1552 (2006)

    Article  Google Scholar 

  28. Mottaghi, R., Xiang, Y., Savarese, S.: A coarse-to-fine model for 3d pose estimation and sub-category recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 418–426 (2015)

  29. Mousavian, A., Anguelov, D., Flynn, J., Kosecka, J.: 3d bounding box estimation using deep learning and geometry. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7074–7082 (2017)

  30. Naseer, T., Burgard, W.: Deep regression for monocular camera-based 6-dof global localization in outdoor environments. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1525–1530. IEEE (2017)

  31. Peng, S., Liu, Y., Huang, Q., Zhou, X., Bao, H.: Pvnet: pixel-wise voting network for 6dof pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4561–4570 (2019)

  32. Sahin, C., Garcia-Hernando, G., Sock, J. Kim, T.-K.: Instance- and Category-Level 6D Object Pose Estimation, pp. 243–265 (2019)

  33. Sattler, T., Maddern, W., Toft, C., Torii, A., Hammarstrand, L., Stenborg, E., Safari, D., Okutomi, M., Pollefeys, M., Sivic, J., et al.: Benchmarking 6dof outdoor visual localization in changing conditions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8601–8610 (2018)

  34. Sattler, T., Sweeney, C., Pollefeys, M.: On sampling focal length values to solve the absolute pose problem. In: European Conference on Computer Vision, pp. 828–843. Springer (2014)

  35. Sattler, T., Zhou, Q., Pollefeys, M., Leal-Taixe, L.: Understanding the limitations of cnn-based absolute camera pose regression. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3302–3312 (2019)

  36. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556

  37. Sun, J., Wang, P., Qin, Z., Qiao, H.: Overview of camera calibration for computer vision. In: Proceeding of the 11th World Congress on Intelligent Control and Automation, pp. 86–92. IEEE (2014)

  38. Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J.B., Freeman, W.T.: Pix3d: dataset and methods for single-image 3d shape modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2974–2983 (2018)

  39. Sundermeyer, M., Marton, Z.-C., Durner, M., Brucker, M., Triebel, R.: Implicit 3d orientation learning for 6d object detection from rgb images. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 699–715 (2018)

  40. Tekin, B., Sinha, S.N., Fua, P.: Real-time seamless single shot 6d object pose prediction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 292–301 (2018)

  41. Triggs, B.: Autocalibration and the absolute quadric. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 609–614. IEEE (1997)

  42. Wang, C., Martín-Martín, R., Xu, D., Lv, J., Lu, C., Fei-Fei, L., Savarese, S., Zhu, Y.: 6-pack: category-level 6d pose tracker with anchor-based keypoints (2019). arXiv:1910.10750

  43. Workman, S., Greenwell, C., Zhai, M., Baltenberger, R., Jacobs, N.: Deepfocal: a method for direct focal length estimation. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 1369–1373. IEEE (2015)

  44. Wu, C.: P3. 5p: pose estimation with unknown focal length. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2440–2448 (2015)

  45. Xiang, Y., Schmidt, T., Narayanan, V., Fox, D.: Posecnn: a convolutional neural network for 6d object pose estimation in cluttered scenes (2017). arXiv:1711.00199

  46. Xu, G.Y., Chen, L.P., Gao, F.: Study on binocular stereo camera calibration method. In: 2011 International Conference on Image Analysis and Signal Processing, pp. 133–137. IEEE (2011)

  47. Zhang, Z.: Flexible camera calibration by viewing a plane from unknown orientations. In:Proceedings of ICCV (1999)

  48. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22, 1330–1334 (2000)

    Article  Google Scholar 

  49. Zhang, Z., Zhao, R., Liu, E., Yan, K., Ma, Y.: A single-image linear calibration method for camera. Measurement 130, 298–305 (2018)

    Article  Google Scholar 

  50. Zheng, Y., Kneip, L.: A direct least-squares solution to the pnp problem with unknown focal length. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1790–1798 (2016)

  51. Zheng, Y., Sugimoto, S., Sato, I., Okutomi, M.: A general and simple method for camera pose and focal length determination. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 430–437 (2014)

  52. Zhu, Z., Wang, X., Liu, Q., Zhang, F.: Camera calibration method based on optimal polarization angle. Opt. Lasers Eng. 112, 128–135 (2019)

    Article  Google Scholar 

Download references

Funding

This study was funded by Postgraduate Research & Practice Innovation Program of Jiangsu Province Grant Number SJCX20 _ 0775.

Author information

Authors and Affiliations

  1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, 214122, China

    Defeng Liu & Lifang Chen

Authors
  1. Defeng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lifang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lifang Chen.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, D., Chen, L. SECPNet—secondary encoding network for estimating camera parameters. Vis Comput 38, 1689–1702 (2022). https://doi.org/10.1007/s00371-021-02098-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02098-2

Keywords

Search

Navigation