Skip to main content
Springer Nature Link
Log in

Parallel refinement of slanted 3D reconstruction using dense stereo induced from symmetry

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Traditional dense stereo estimation algorithms measure photo-similarity to calculate the disparity between image pairs. SymStereo is a new framework of matching cost functions that measure symmetry to evaluate the possibility of two pixels being a match. This article proposes a fully functional real-time parallel 3D reconstruction pipeline that uses dense stereo-based photo-symmetry. The logN variant of SymStereo achieves superior results for images with slanted surfaces, when compared with other algorithms (Antunes and Barreto in Int J Comput Vis 1–22, 2014). This is of particular interest for areas of computer vision such as the processing of datasets for urban scene reconstruction and also for tracking in robotics or intelligent autonomous vehicles. The output results obtained are analyzed by tuning distinct matching cost, aggregation and refinement parameters, targeting the most suitable combinations for slant dominated images. Also, the parallel approach for the aforementioned pipeline consists of a hybrid dual GPU system capable of calculating from 2 up to 132 volumes per second for high- and low-resolution images, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Antunes, M., Barreto, J.P.: SymStereo: stereo matching using induced symmetry. Int. J. Comput. Vis, 1–22 (2014)

  2. Birchfield, S., Tomasi, C.: A pixel dissimilarity measure that is insensitive to image sampling. IEEE Trans. Pattern Anal. Mach. Intell. 20, 401–406 (1998)

    Article  Google Scholar 

  3. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Eklundh, J.O. (ed) Computer Vision ECCV ’94, volume 801 of Lecture Notes in Computer Science, pp. 151–158. Springer, Berlin Heidelberg (1994)

  4. NVIDIA Corporation. CUDA Zone [Online]. https://developer.nvidia.com/cuda-zone (2014)

  5. AMD. OpenCL Zone [Online]. https://developer.amd.com/tools-and-sdks/opencl-zone/ (2014)

  6. Hill, K., Craciun, S., George, A., Lam, H.: Comparative analysis of opencl vs. hdl with image-processing kernels on stratix-v fpga. In: 2015 IEEE 26th International Conference on Application-specific Systems, Architectures and Processors (ASAP), pp. 189–193 (2015)

  7. Rodriguez-Donate, C., Botella, G., Garcia, C., Cabal-Yepez, E., Prieto-Matias, M.: Early experiences with opencl on fpgas: convolution case study. In: 2015 IEEE 23rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM), pp. 235–235 (2015)

  8. Maria, J., Amaro, J., Falcao, G., Alexandre, L.A.: Stacked autoencoders using low-power accelerated architectures for object recognition in autonomous systems. Neural Process. Lett., 1–14 (2015)

  9. Wang, B., Alvarez-Mesa, M., Chi, C.C., Juurlink, B.: Parallel h.264/avc motion compensation for gpus using opencl. IEEE Trans. Circuits Syst. Video Technol. 25(3), 525–531 (2015)

    Article  Google Scholar 

  10. Falcao, G., Silva, V., Sousa, L., Andrade, J.: Portable LDPC decoding on multicores using OpenCL [Applications Corner]. IEEE Sig. Process. Mag. 29(4), 81–109 (2012)

    Article  Google Scholar 

  11. Zhang, Z., Shan, Y.: A progressive scheme for stereo matching. In: Pollefeys, M., Van G., Luc, Z., Andrew, F. (eds) Andrew 3D Structure from Images SMILE 2000. Lecture Notes in Computer Science, vol. 2018, pp. 68–85. Springer, Berlin Heidelberg (2001)

  12. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1–3), 7–42 (2002)

    Article  MATH  Google Scholar 

  13. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-View stereo reconstruction algorithms. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 519–528 (2006)

  14. Sun, J., Zheng, N.-N., Shum, H.-Y.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)

    Article  MATH  Google Scholar 

  15. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  16. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: 8th IEEE International Conference on Computer Vision, 2001, ICCV 2001, Proceedings, vol. 2, pp. 508–515 (2001)

  17. Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. In: 6th International Conference on Computer Vision, 1998, pp. 1073–1080 (1998)

  18. Anandan, P.: A computational framework and an algorithm for the measurement of visual motion. Int. J. Comput. Vis. 2(3), 283–310 (1989)

    Article  Google Scholar 

  19. Matthies, L.H., Szeliski, R., Kanade, T.: Kalman filter-based algorithms for estimating depth from image sequences. Int. J. Comput. Vis. 3(3), 209–236 (1989)

    Article  Google Scholar 

  20. Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans. Pattern Anal. Mach. Intell. 16(9), 920–932 (1994)

    Article  Google Scholar 

  21. Yoon, K.-J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell. 28(4), 295–308 (2012)

    Google Scholar 

  22. Tang, H., Zhu, Z.: Content-based 3-D mosaics for representing videos of dynamic urban scenes. IEEE Trans. Circuits Syst. Video Technol. 22(2), 295–308 (2012)

    Article  Google Scholar 

  23. Zhu, Z.Y., Zhang, S., Chan, S.C., Shum, H.Y.: Object-based rendering and 3D reconstruction using a moveable image-based rendering system. In: 2011 7th International Workshop on Multidimensional (nD) Systems (nDs), pp. 1–4 (2011)

  24. Ferrari Pinto, R., Conceicao, A.G.S., Farias, P.C.M.A, Santos, E.T.F.: A cost effective open-source three-dimensional reconstruction system and trajectory analysis for mobile robots. In: 5th ISSNIP-IEEE Biosignals and Biorobotics Conference (2014): Biosignals and Robotics for Better and Safer Living (BRC), pp. 1–5 (2014)

  25. Zia, A., Liang, J., Zhou, J., Gao, Y.: 3d reconstruction from hyperspectral images. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 318–325 (2015)

  26. Yamao, S., Miura, M., Sakai, S., Ito, K., Aoki, T.: A sequential online 3d reconstruction system using dense stereo matching. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 341–348, (2015)

  27. Ge, K., Hu, H., Feng, J., Zhou, J.: Depth estimation using a sliding camera. IEEE Trans. Image Process. 25(2), 726–739 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  28. Kowalczuk, J., Psota, E.T., Perez, L.C.: Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23(1), 94–104 (2013)

    Article  Google Scholar 

  29. Lee, S.H., Sharma, S.: Real-time disparity estimation algorithm for stereo camera systems. IEEE Trans. Consum. Electron. 57(3), 1018–1026 (2011)

    Article  Google Scholar 

  30. Zhang, K., Lu, J., Yang, Q., Lafruit, G., Lauwereins, R., Van Gool, L.: Real-time and accurate stereo: a scalable approach with bitwise fast voting on CUDA. IEEE Trans. Circuits Syst. Video Technol. 21(7), 867–878 (2011)

    Article  Google Scholar 

  31. Zhang, J., Nezan, J.-F., Pelcat, M., Cousin, J.-G.: Real-time gpu-based local stereo matching method. In:2013 Conference on Design and Architectures for Signal and Image Processing (DASIP), pp. 209–214 (2013)

  32. Yang, A., Li, X., Jia, S., Qin, B.: Monocular three dimensional dense surface reconstruction by optical flow feedback. In: 2015 IEEE International Conference on Information and Automation, pp. 504–509 (2015)

  33. Long, Q., Xie, Q., Mita, S., Ishimaru, K., Shirai, N.: A real-time dense stereo matching method for critical environment sensing in autonomous driving. In: 2014 IEEE 17th International Conference on Intelligent Transportation Systems (ITSC), pp. 853–860 (2014)

  34. Mota, V., Falcao, G., Antunes, M., Barreto, J., Nunes, U.: Using the gpu for fast symmetry-based dense stereo matching in high resolution images. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 7520–7524 (2014)

  35. Ralha, R., Falcao, G., Andrade, J., Antunes, M., Barreto, J.P., Nunes, U.: Distributed dense stereo matching for 3d reconstruction using parallel-based processing advantages. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1126–1130 (2015)

  36. Szeliski, R., Scharstein, D.: Sampling the disparity space image. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 419–425 (2004)

    Article  Google Scholar 

  37. Collins, R.T.: A space-sweep approach to true multi-image matching. In: Proceedings of the 1996 Conference on Computer Vision and Pattern Recognition (CVPR ’96), CVPR ’96, pp. 358, Washington, DC, USA. IEEE Computer Society (1996)

  38. Kovesi, P.: Symmetry and asymmetry from local phase. In: 10th Australian Joint Conference on Artificial Intelligence (1997)

  39. Kovesi, P: Image features from phase congruency. Technical report, Videre: Journal of Computer Vision Research (1995)

  40. NVIDIA Corporation. NVIDIAs Next Generation CUDA Compute Architecture: Kepler GK110 (2012)

  41. NVIDIA Corporation. cuFFT [Online]. https://developer.nvidia.com/cuFFT, 2014

  42. Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. (IJRR) (2013)

  43. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI Vision Benchmark Suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)

  44. Richardt, C., Orr, D.A.H., Davies, I.P., Criminisi, A., Dodgson, N.A.: Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid. In: European Conference on Computer Vision (ECCV). Springer Verlag (2010)

  45. Geiger, A., Roser, M., Urtasun, R.: Efficient large-scale stereo matching. In: Kimmel, R., Klette, R., Sugimoto, A. (eds) Computer Vision ACCV 2010. Lecture Notes in Computer Science, vol. 6492, pp. 25–38. Springer, Berlin Heidelberg (2011)

Download references

Acknowledgments

This work was supported by the Portuguese Foundation for Science and Technology (FCT), with FEDER/COMPETE program funding, under Grants AMS-HMI12: RECI/EEI-AUT/0181/2012, UID/EEA/50008/2013 and also by a Google Faculty Research Award from Google Inc. This research was also carried out at the Multimedia Signal Processing Lab, Instituto de Telecomunicações, an NVIDIA GPU Research Center from the University of Coimbra, Portugal.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Instituto de Telecomunicações, University of Coimbra, Coimbra, Portugal

    Ricardo Ralha, Gabriel Falcao & Joao Amaro

  2. Department of Electrical and Computer Engineering, Institute of Systems and Robotics, University of Coimbra, Coimbra, Portugal

    Vasco Mota, Joao Barreto & Urbano Nunes

  3. Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg, Luxembourg

    Michel Antunes

Authors
  1. Ricardo Ralha
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Gabriel Falcao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Joao Amaro
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Vasco Mota
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Michel Antunes
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Joao Barreto
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Urbano Nunes
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Gabriel Falcao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ralha, R., Falcao, G., Amaro, J. et al. Parallel refinement of slanted 3D reconstruction using dense stereo induced from symmetry. J Real-Time Image Proc 16, 1037–1055 (2019). https://doi.org/10.1007/s11554-016-0592-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-016-0592-0

Keywords