Skip to main content
SpringerLink
Log in
Book cover

Computer Graphics International Conference

CGI 2022: Advances in Computer Graphics pp 117–128Cite as

An Efficient Dense Depth Map Estimation Algorithm Using Direct Stereo Matching for Ultra-Wide-Angle Images

  • Conference paper
  • First Online:

  • 1120 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13443))

Abstract

We present an efficient dense depth map estimation algorithm using patch-based direct stereo matching for ultra-wide-angle images. Our algorithm takes account of the fact that the neighboring pixels inside a local patch are likely to lie on the same plane. Our algorithm propagates the “good” initial guesses to the neighboring pixels by spatial propagation, followed by a random refinement process. Therefore, this allows finding precise depth value for each point in an infinite space using a random search strategy. Our algorithm can be used to perform 3D reconstruction using the dense depth maps directly generated from ultra-wide-angle images, especially from stereo camera pairs.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://www.cvg.ethz.ch/research/planeSweepLib/.

  2. 2.

    https://github.com/menandro/vfs.

References

  1. Tao, Y., Shen, Y., Sheng, B., Li, P., Lau, R.W.H.: Video decolorization using visual proximity coherence optimization. IEEE Trans. Cybern. 48(5), 1406–1419 (2018)

    Article  Google Scholar 

  2. Zhang, B., Sheng, B., Li, P., Lee, T.Y.: Depth of field rendering using multilayer-neighborhood optimization. IEEE Trans. Vis. Comput. Graph. 26(8), 2546–2559 (2020)

    Article  Google Scholar 

  3. Guo, H., Sheng, B., Li, P., Chen, C.L.P.: Multiview high dynamic range image synthesis using fuzzy broad learning system. IEEE Trans. Cybern. 51(5), 2735–2747 (2021)

    Article  Google Scholar 

  4. Courbon, J., Mezouar, Y., Eckt, L., Martinet, P.: A generic fisheye camera model for robotic applications. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1683–1688 (2007)

    Google Scholar 

  5. Gu, Z., Liu, H., Zhang, G.: Real-time indoor localization of service robots using fisheye camera and laser pointers. In: IEEE International Conference on Robotics and Biomimetics, pp. 1410–1414 (2014)

    Google Scholar 

  6. Zhang, X., Zhao, Y., Mitchell, N., Li, W.: A new 360 camera design for multi format VR experiences. In: IEEE Conference on Virtual Reality and 3D User Interfaces, pp. 1273–1274 (2019)

    Google Scholar 

  7. Häne, C., et al.: 3D visual perception for self-driving cars using a multi-camera system: calibration, mapping, localization, and obstacle detection. Image Vis. Comput. 68, 14–27 (2017)

    Article  Google Scholar 

  8. Kumar, V.R., Klingner, M., Yogamani, S., Milz, S., Fingscheidt, T., Mader, P.: SynDistNet: self-supervised monocular fisheye camera distance estimation synergized with semantic segmentation for autonomous driving. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 61–71 (2021)

    Google Scholar 

  9. Lee, S., Kim, G.J., Choi, S.: Real-time depth-of-field rendering using anisotropically filtered mipmap interpolation. IEEE Trans. Vis. Comput. Graph. 15(3), 453–464 (2009)

    Article  Google Scholar 

  10. Wen, Y., et al.: Structure-aware motion deblurring using multi-adversarial optimized CycleGAN. IEEE Trans. Image Process. 30, 6142–6155 (2021)

    Article  Google Scholar 

  11. Liu, S., Guo, P., Feng, L., Yang, A.: Accurate and robust monocular SLAM with omnidirectional cameras. Sensors 19(20), 4494 (2019)

    Article  Google Scholar 

  12. Wang, Y., et al.: CubemapSLAM: a piecewise-pinhole monocular fisheye SLAM system. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11366, pp. 34–49. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20876-9_3

    Chapter  Google Scholar 

  13. Shah, S., Aggarwal, J.: Depth estimation using stereo fish-eye lenses. In: IEEE International Conference on Image Processing, vol. 2, pp. 740–744 (1994)

    Google Scholar 

  14. Li, S.: Real-time spherical stereo. In: IEEE International Conference on Pattern Recognition, vol. 3, pp. 1046–1049 (2006)

    Google Scholar 

  15. Li, J., Xu, W., Zhang, Z., Zhang, M., Wang, Z.: Fisheye image rectification for efficient large-scale stereo. In: International Conference on Systems and Informatics, pp. 881–885 (2016)

    Google Scholar 

  16. Häne, C., Heng, L., Lee, G.H., Sizov, A., Pollefeys, M.: Real-time direct dense matching on fisheye images using plane-sweeping stereo. In: IEEE International Conference on 3D Vision, p. 57–64 (2014)

    Google Scholar 

  17. Roxas, M., Oishi, T.: Variational fisheye stereo. IEEE Robot. Autom. Lett. 5(2), 1303–1310 (2020)

    Article  Google Scholar 

  18. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 24 (2009)

    Article  Google Scholar 

  19. Sheng, B., Li, P., Gao, C., Ma, K.L.: Deep neural representation guided face sketch synthesis. IEEE Trans. Vis. Comput. Graph. 25(12), 3216–3230 (2019)

    Article  Google Scholar 

  20. Usenko, V., Demmel, N., Cremers, D.: The double sphere camera model. In: International Conference on 3D Vision, pp. 552–560 (2018)

    Google Scholar 

  21. Bleyer, M., Rhemann, C., Rother, C.: PatchMatch stereo-stereo matching with slanted support windows. In: British Machine Vision Conference, vol. 11, pp. 1–11 (2011)

    Google Scholar 

  22. Shen, S.: Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes. IEEE Trans. Image Process. 22(5), 1901–1914 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2004)

    Google Scholar 

  24. Schubert, D., Goll, T., Demmel, N., Usenko, V., Stueckler, J., Cremers, D.: The TUM VI benchmark for evaluating visual-inertial odometry. In: International Conference on Intelligent Robots and Systems, pp. 1680–1687 (2018)

    Google Scholar 

  25. Barnes, D., Gadd, M., Murcutt, P., Newman, P., Posner, I.: The Oxford radar robotcar dataset: a radar extension to the Oxford robotcar dataset. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 6433–6438 (2020)

    Google Scholar 

  26. Maddern, W., Pascoe, G., Gadd, M., Barnes, D., Yeomans, B., Newman, P.: Real-time kinematic ground truth for the Oxford robotcar dataset. arXiv preprint arXiv: 2002.10152 (2020)

  27. Maddern, W., Pascoe, G., Linegar, C., Newman, P.: 1 year, 1000km: the Oxford RobotCar dataset. The Int. J. Robot. Res. 36(1), 3–15 (2017)

    Article  Google Scholar 

  28. Furgale, P., Rehder, J., Siegwart, R.: Unified temporal and spatial calibration for multi-sensor systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1280–1286 (2013)

    Google Scholar 

  29. Kannala, J., Brandt, S.S.: A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Trans. Pattern Anal. Mach. Intell. 28(8), 1335–1340 (2006)

    Article  Google Scholar 

  30. Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

Download references

Acknowledgement

This work is supported by the National Key R &D Program of China under grant 2021ZD0114501.

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Trustworthy Computing, Engineering Research Center of Software/Hardware Co-design Tech and Application, East China Normal University, Shanghai, China

    Xiuxiu Gui & Xinyu Zhang

Authors
  1. Xiuxiu Gui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xinyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinyu Zhang .

Editor information

Editors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

  2. Bournemouth University, Poole, UK

    Jian Zhang

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. University of Crete, Heraklion, Greece

    George Papagiannakis

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

  7. University of Calgary, Calgary, AB, Canada

    Marina Gavrilova

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gui, X., Zhang, X. (2022). An Efficient Dense Depth Map Estimation Algorithm Using Direct Stereo Matching for Ultra-Wide-Angle Images. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2022. Lecture Notes in Computer Science, vol 13443. Springer, Cham. https://doi.org/10.1007/978-3-031-23473-6_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23473-6_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23472-9

  • Online ISBN: 978-3-031-23473-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation