Skip to main content
SpringerLink
Log in

Ensemble learning with advanced fast image filtering features for semi-global matching

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

For the past several years, a variety of algorithms have been focused on how to exploit two-dimensional scanline optimization to augment one-dimensional ones for semi-global matching. Different from the former contributions, an ensemble learning with advanced fast image filtering features for semi-global matching is proposed in this paper. Firstly, fewer categories of features (confidence measures) are extracted through various advanced fast image filters on the original scale of 8 directions’ semi-global matching disparity maps. Then, all the features are weaved together and divided into positive and negative samples for ensemble learning after comparing with ground truth. After that, the initial disparity map is obtained by leveraging the confidence probability of ensemble learning prediction. Finally, an efficient two-step single view disparity refinement strategy is employed, which no longer requires the right view’s disparity map for attaining the final refined results. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed algorithm outperforms other four most recent stereo matching algorithms. In addition, the presented algorithm shows relative high implementation efficiency compared with others.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)

    Article  Google Scholar 

  2. Gong, M., Yang, R., Wang, L., et al.: A performance study on different cost aggregation approaches used in real-time stereo matching. Int. J. Comput. Vis. 75, 283–296 (2007)

    Article  Google Scholar 

  3. Tombari, F., Mattoccia, S., Stefano, L., et al.: Classification and evaluation of cost aggregation methods for stereo correspondence. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 1–8 (2008)

  4. Yoon, K.J., Kweon, I.S.: Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. 28, 650–656 (2006)

    Article  Google Scholar 

  5. Hosni, A., Bleyer, M., Gelautz, M., et al.: Local stereo matching using geodesic support weights. In: IEEE international conference on image processing (ICIP), pp. 2093–2096 (2009)

  6. Zhang, K., Lu, J., Lafruit, G.: Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circuits Syst. Video Technol. 19, 1073–1079 (2009)

    Article  Google Scholar 

  7. Min, D., Lu, J., Do, M.N.: A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy? In: IEEE International Conference on Computer Vision (ICCV), pp. 1567–1574 (2011)

  8. Min, D., Lu, J., Do, M.N.: Joint histogram based cost aggregation for stereo matching. In: IEEE Transactions on Pattern Analysis (TPAMI), vol. 35, pp. 2539–2545 (2013)

  9. Hu, W., Zhang, K., Sun, L., et al.: Virtual support window for adaptive-weight stereo matching. In: IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2011)

  10. Zhang, K., Li, J., Li, Y., et al.: Binary stereo matching. In: IEEE International Conference on Pattern Recognition (ICPR), pp. 356–359 (2012)

  11. Christo, R., Hosni, A., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. In: IEEE conference on computer vision and pattern recognition (CVPR), pp. 3017–3024 (2011)

  12. Christo, R., Hosni, A., Bleyer, M., et al.: Fast cost-volume filtering for visual correspondence and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 32, 504–511 (2013)

    Google Scholar 

  13. Helala, M., Qureshi, F.: Accelerating cost volume filtering using salient subvolumes and robust occlusion handling. In: Asian Conference on Computer Vision (ACCV), pp. 316–331. Springer (2014)

  14. Li, X., Liu, J., Chen, G., et al.: Efficient methods using slanted support windows for slanted surfaces. IET Comput. Vis. 10, 384–391 (2016)

    Article  Google Scholar 

  15. Bleyer, M., Breiteneder, C.: Advanced Topics in Computer Vision, pp. 143–179. Springer, Heidelberg (2013)

    Book  Google Scholar 

  16. Barnes, C., Shechtman, E., Finkelstein, A., et al.: PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 1–10 (2009)

    Article  Google Scholar 

  17. Bleyer, M., Rhemann, C., Rother, C.: PatchMatch stereo-stereo matching with slanted support windows. In: British Machine Vision Conference (BMVC), pp. 1–11 (2011)

  18. Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. IEEE Trans. Circuits Syst. Video Technol. 27, 965–976 (2017)

    Article  Google Scholar 

  19. Yao, P., Zhang, H., Xue, Y., et al.: SPMVP: spatial PatchMatch stereo with virtual pixel aggregation. In: International Conference on Neural Information Processing (ICONIP), Part III, pp. 527-542. Springer (2017)

  20. Yang, Q.: A non-local cost aggregation method for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1402-1409 (2012)

  21. Yang, Q.: Stereo matching using tree filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 37, 834–846 (2015)

    Article  Google Scholar 

  22. Mei, X., Sun, X., Dong, W., et al.: Segment-tree based cost aggregation for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 313–320 (2013)

  23. Zhang, K., Fang, Y., Min, D., et al.: Cross-scale cost aggregation for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 407–414 (2014)

  24. Yang, Q.: Local smoothness enforced cost volume regularization for fast stereo correspondence. IEEE Signal Process. Lett. 22, 1429–1433 (2015)

    Article  Google Scholar 

  25. Yao, P., Zhang, H., Xue Y., et al.: Iterative color-depth MST cost aggregation for stereo matching. In: IEEE International Conference on Multimedia & Expo (ICME), pp. 1–6 (2016)

  26. Yao, P., Zhang, H., Xue, Y., et al.: Segment-tree based cost aggregation for stereo matching with enhanced segmentation advantage. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2027–2031 (2017)

  27. Yang, Q., Wang, L., Yang, R., et al.: Real-time global stereo matching using hierarchical belief propagation. In: British machine vision conference (BMVC), pp. 1–10 (2006)

  28. Yang, Q., Wang, L., Yang, R., et al.: Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Trans. Pattern Anal. Mach. Intell. 31, 492–504 (2008)

    Article  Google Scholar 

  29. Yang, Q., Wang, L., Ahuja, N.: A constant space belief propagation algorithm for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1458–1465 (2010)

  30. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: IEEE International Conference on Computer Vision (ICCV), pp. 508–515 (2001)

  31. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1222–1239 (2001)

    Article  Google Scholar 

  32. Taniai, T., Matsushita, Y., Naemura, T.: Graph cut based continuous stereo matching using locally shared labels. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1613–1620 (2014)

  33. Taniai, T., Matsushita, Y., Sato, Y., et al.: Continuous 3D label stereo matching using local expansion moves. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2725–2739 (2018)

    Article  Google Scholar 

  34. Besse, F., Rother, C., Fitzgibbon, A., et al.: PMBP: PatchMatch belief propagation for correspondence field estimation. In: British Machine Vision Conference (BMVC), pp. 1–11 (2012)

  35. Heise, P., Klose, S., Jensen, B., et al.: PM-Huber: PatchMatch with Huber regularization for stereo matching. In: IEEE International Conference on Computer Vision (ICCV), pp. 2360–2367 (2013)

  36. Zhang, C., Li, Z., Cai, R., et al.: As-rigid-as-possible stereo under second order smoothness priors. In: European Conference on Computer Vision (ECCV), pp. 112–126. Springer (2014)

  37. Lu, J., Yang, H., Min, D., et al.: PatchMatch filter: efficient edge aware filtering meets randomized search for fast correspondence field estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1854–1861 (2013)

  38. Lu, J., Li, Y., Yang, H., et al.: PatchMatch filter: edge-aware filtering meets randomized search for visual correspondence. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1866–1879 (2017)

    Article  Google Scholar 

  39. Li, Y., Min, D., Brown, M.S., et al.: SPM-BP: sped-up PatchMatch belief propagation for continuous MRFs. In: IEEE International Conference on Computer Vision (ICCV), pp. 4006–4014 (2015)

  40. Zhang, C., Li, Z., Cheng, Y., et al.: MeshStereo: a global stereo model with mesh alignment regularization for view interpolation. In: IEEE International Conference on Computer Vision (ICCV), pp. 2057–2065 (2015)

  41. Mozerov, M.G., Weijer, J.: Accurate stereo matching by two-step energy minimization. IEEE Trans. Image Process. 24, 1153–1163 (2015)

    Article  MathSciNet  Google Scholar 

  42. Yao, P., Zhang, H., Xue Y., et al.: AGO: accelerating global optimization for accurate stereo matching. In: International Conference on Multimedia Modeling (MMM), pp. 67–80. Springer (2018)

  43. Yao, P., Zhang, H., Xue, Y., et al.: MSCS: MeshStereo with cross-scale cost filtering for fast stereo matching. IET Comput. Vis. 12, 908–918 (2018)

    Article  Google Scholar 

  44. Hirschmuller, H.: Accurate and efficient stereo processing by semi-global matching and mutual information. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 807–814 (2005)

  45. Hirschmuller, H.: Stereo processing by semi-global matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30, 328–341 (2008)

    Article  Google Scholar 

  46. Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1592–1599 (2015)

  47. Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res. 17, 1–32 (2016)

    MATH  Google Scholar 

  48. Kendall, A., Martirosyan, H., Dasgupta, S., et al.: End-to-end learning of geometry and context for deep stereo regression. In: IEEE International Conference on Computer Vision (ICCV), pp. 66–75 (2017)

  49. Knobelreiter, P., Reinbacher, C., Shekhovtsov, A., et al.: End-to-end training of hybrid CNN-CRF models for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1456–1465 (2017)

  50. LeGendre, C., Batsos, K., Mordohai, P.: High-resolution stereo matching based on sampled photo-consistency computation. In: British Machine Vision Conference (BMVC), pp. 1–13 (2017)

  51. Hu, X., Mordohai, P.: A quantitative evaluation of confidence measures for stereo vision. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2121–2133 (2012)

    Article  Google Scholar 

  52. Haeusler, R., Nair, R., Kondermann., D.: Ensemble learning for confidence measures in stereo vision. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 305–312 (2013)

  53. Spyropoulos, A., Komodakis, N., Mordohai, P.: Learning to detect ground control points for improving the accuracy of stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1621–1628 (2014)

  54. Spyropoulos, A., Komodakis, N., Mordohai, P.: Correctness prediction, accuracy improvement and generalization of stereo matching using supervised learning. Int. J. Comput. Vis. 118, 300–318 (2016)

    Article  MathSciNet  Google Scholar 

  55. Park, M.G., Yoon, K.J.: Leveraging stereo matching with learning-based confidence measures. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 101–109 (2015)

  56. Poggi, M., Mattoccia S.: Learning a general-purpose confidence measure based on O(1) features and a smarter aggregation strategy for semi global matching. In: IEEE International Conference on 3D Vision (3DV), pp. 509–518 (2016)

  57. Batsos, K., Cai, C., Mordohai, P.: CBMV: a coalesced bidirectional matching volume for disparity estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2060–2069 (2018)

  58. Drory, A., Haubold, C., Avidan, S., et al.: Semi-global matching: a principled derivation in terms of message passing. In: German Conference of Pattern Recognition (GCPR), pp. 43–53. Springer (2014)

  59. Facciolo, G., Franchis, C., Meinhardt, E.: MGM: a significantly more global matching for stereo vision. In: British Machine Vision Conference (BMVC), pp. 1–12 (2015)

  60. Yao, P., Zhang, H., Xue, Y., et al.: As-global-as-possible stereo matching with adaptive smoothness prior. IET Image Process. 13, 98–107 (2019)

    Article  Google Scholar 

  61. Scharstein, D., Szeliski, R.: Learning conditional random fields for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  62. Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2007)

  63. Scharstein, D., Hirschmuller, H., Kitajima, Y., et al.: High-resolution stereo datasets with subpixel-accurate ground truth. In: German Conference on Pattern Recognition (GCPR), pp. 31–42. Springer (2014)

  64. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: European Conference on Computer Vision (ECCV), pp. 151–158. Springer (1994)

  65. Zhang, Q., Xu, L., Jia, J.: 100+ times faster weighted median filter (WMF). In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2830–2837 (2014)

  66. Yan, T., Gan, Y., Xia, Z., et al.: Segment-based disparity refinement with occlusion handling for stereo matching. IEEE Trans. Image Process. 28, 3885–3897 (2019)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work was jointly supported by the National Natural Science Foundation of China under Grants Nos. 61732015, 61932018 and 61472349, and the National Key Research & Development Program of China under Grant No. 2017YFB0202203.

Author information

Authors and Affiliations

  1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, China

    Peng Yao & Jieqing Feng

Authors
  1. Peng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jieqing Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jieqing Feng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yao, P., Feng, J. Ensemble learning with advanced fast image filtering features for semi-global matching. Machine Vision and Applications 32, 83 (2021). https://doi.org/10.1007/s00138-021-01211-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01211-8

Keywords

Search

Navigation