Skip to main content
SpringerLink
Log in

A constraints-based approach using ranking-gradient-similarity multi-block matching algorithm

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Many works have been done in stereo vision due to its critical importance to the different uses and various areas of computer vision. Recently, the popularity of research on the local block matching algorithm is difficult to compete with the convolutional neural network (CNN) approach, whereby the latter method can achieve higher accuracy at the expense of time consumption. However, the simplicity of the local block matching algorithm is easier to be implemented in real applications at a lower cost. Therefore, this paper presents a new study that improved computation and cost aggregation using a progressive approach. A new algorithm to aid in searching locally similar blocks called the ranking-gradient-similarity multi-block strategy (RGSMB) has been presented, improving processing time and accuracy. A new cost computation has been introduced in the proposed algorithm, which fully utilises the information from the limited local window region by combining the cost from three different constraints, including a Ranking constraint, a Gradient constraint, and a Similarity constraint. The proposed algorithm can accomplish remarkable accuracy by combining with multi-block matching (MBM), which is developed to diminish the conventional cost aggregation's constrained shape limitation. We evaluate the RGSMB algorithm's performance and other CNN methods for comparison. KITTI 2012 dataset of 194 and 195 image pairs and the KITTI 2015 dataset of 200 image pairs were used for training and testing. The results indicate that the proposed algorithm is better than the existing approaches, with a higher matching rate using the KITTI benchmark dataset. Compared to existing local block matching algorithms, the average errors of the RGS are less than the CT approach by nearly 9%, which is the 2nd ranked in this analysis. Also, the RGSMB has average errors of 21% less than the \(S_{7}\) + correlation when comparing with existing CNN methods. It was also observed that the RGSMB could estimate the disparity value more accurately in the depth discontinuities and homogenous regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The authors confirm that the data supporting the findings of this study are available within the article. The dataset(s) supporting the conclusions of this article is (are) available in the KITTI Data Set repository http://www.cvlibs.net/datasets/kitti/index.php.

References

  1. Adhyapak S et al (2007) Stereo matching via selective multiple windows. J Electron Imaging 16(1):013012

    Article  Google Scholar 

  2. Bahnemiri SG et al (2017) Environment mapping with stereo vision and belief propagation algorithm. In: 2017 IEEE 4th international conference on knowledge-based engineering and innovation (KBEI). IEEE

  3. Banz C et al (2010) Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: 2010 international conference on embedded computer systems (SAMOS). IEEE

  4. Binaghi E et al (2004) Neural adaptive stereo matching. Pattern Recogn Lett 25(15):1743–1758

    Article  Google Scholar 

  5. Bobick AF et al (1999) Large occlusion stereo. Int J Comput Vision 33(3):181–200

    Article  Google Scholar 

  6. Brandao P et al (2019) Widening siamese architectures for stereo matching. Pattern Recogn Lett 120:75–81

    Article  Google Scholar 

  7. Brown MZ et al (2003) Advances in computational stereo. IEEE Trans Pattern Anal Mach Intell 25(8):993–1008

    Article  Google Scholar 

  8. Cafiso S et al (2017) In-vehicle stereo vision system for identification of traffic conflicts between bus and pedestrian. J Traffic Transp Eng (Engl Ed) 4(1):3–13

    Google Scholar 

  9. Chang et al (2017) Robust stereo matching with trinary cross color census and triple image-based refinements. EURASIP J Adv Signal Proc 2017(1):27

    Article  Google Scholar 

  10. Cheng F et al (2018) Learning to refine depth for robust stereo estimation. Pattern Recogn 74:122–133

    Article  Google Scholar 

  11. Einecke N et al (2015) A multi-block-matching approach for stereo. In: Intelligent vehicles symposium (IV). IEEE

  12. Fan Y-C et al (2012) Disparity measurement using dynamic programming. In: 2012 IEEE international instrumentation and measurement technology conference (I2MTC). IEEE

  13. Fan R et al (2019) Real-time dense stereo embedded in a uav for road inspection. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  14. Fife WS et al (2013) Improved census transforms for resource-optimised stereo vision. IEEE Trans Circuits Syst Video Technol 23(1):60–73

    Article  Google Scholar 

  15. Georgoulas C et al (2010) FPGA based disparity map computation with vergence control. Microprocess Microsyst 34(7–8):259–273

    Article  Google Scholar 

  16. Gupta RK et al (2013) Window-based approach for fast stereo correspondence. IET Comput Vis 7(2):123–134

    Article  Google Scholar 

  17. Hamzah RA et al (2018) Matching cost computation based on sum of absolute RGB differences. In: 2018 IEEE symposium on computer applications & industrial electronics (ISCAIE). IEEE

  18. Hamzah RA et al (2016) Literature survey on stereo vision disparity map algorithms. J Sens. https://doi.org/10.1155/2016/8742920

    Article  Google Scholar 

  19. Hirschmüller H et al (2002) Real-time correlation-based stereo vision with reduced border errors. Int J Comput Vis 47(1–3):229–246

    Article  MATH  Google Scholar 

  20. Ji S et al (2020) Quaternary census transform based on the human visual system for stereo matching. IEEE Access 8:116501–116514

    Article  Google Scholar 

  21. Kim M et al (2019) A low-cost stereo-fisheye camera sensor for daylighting and glare control. In: Journal of physics: conference series, vol 1343, no 1. IOP Publishing, p 012157

  22. Klaus A et al (2006) Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: 18th international conference on pattern recognition, 2006. ICPR 2006. IEEE

  23. Kok KY et al (2022) A constraints based approach using ranking-gradient-similarity multi-block matching algorithm for computer vision. Available at SSRN 4093613

  24. Lee J et al (2016) Improved census transform for noise robust stereo matching. Opt Eng 55(6):063107

    Article  Google Scholar 

  25. Luo W et al (2016) Efficient deep learning for stereo matching. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  26. Ma X et al (2022) Adaptive deconvolution-based stereo matching net for local stereo matching. Appl Sci 12(4):2086

    Article  Google Scholar 

  27. Miron A et al (2014) A robust cost function for stereo matching of road scenes. Pattern Recogn Lett 38:70–77

    Article  Google Scholar 

  28. Park IK (2018) Deep self-guided cost aggregation for stereo matching. Pattern Recogn Lett 112:168–175

    Article  Google Scholar 

  29. Satoh S (2011) Simple low-dimensional features approximating NCC-based image matching. Pattern Recogn Lett 32(14):1902–1911

    Article  Google Scholar 

  30. Song K et al (2016) Noise robust image matching using adjacent evaluation census transform and wavelet edge joint bilateral filter in stereo vision. J Vis Commun Image Represent 38:487–503

    Article  Google Scholar 

  31. Wang Y et al (2019) An improved two-point calibration method for stereo vision with rotating cameras in large FOV. J Mod Opt 66(10):1106–1115

    Article  Google Scholar 

  32. Yang Y et al (2018) An efficient stereo matching algorithm based on four-moded census transform for high-resolution images. 3D Res 9(3):33

    Article  Google Scholar 

  33. Yoon K-J et al (2005) Locally adaptive support-weight approach for visual correspondence search. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005. IEEE

  34. Zabih R et al (1994) Non-parametric local transforms for computing visual correspondence. In: European conference on computer vision. Springer, Berlin

  35. Zbontar J et al (2016) Stereo matching by training a convolutional neural network to compare image patches. J Mach Learn Res 17(1–32):2

    MathSciNet  MATH  Google Scholar 

  36. Zhang B et al (2021) Local stereo matching: an adaptive weighted guided image filtering-based approach. Int J Pattern Recognit Artif Intell 35(03):2154010

    Article  Google Scholar 

  37. Zhang C et al (2019) Visual navigation based on stereo camera for water conservancy UAVs. In: 2019 9th international conference on information science and technology (ICIST), p 158–183

  38. Zhang S et al (2018) Tractor path tracking control based on binocular vision. Inf Process Agric 5(4):422–432

    Google Scholar 

  39. Zhuo BY et al (2021) Stereo matching approach using zooming images. Int Inov Comput Inf Control 17(2):473–482

    Google Scholar 

Download references

Funding

This publication was supported by Malaysia Ministry of Education (MOE)—FRGS/1/2019/TK08/USM/02/1 Grant.

Author information

Authors and Affiliations

  1. School of Aerospace Engineering, Universiti Sains Malaysia, Engineering Campus, Nibong Tebal, Pulau Pinang, Malaysia

    Kai Yit Kok & Parvathy Rajendran

  2. Faculty of Engineering and Computing, First City University College, Bandar Utama, Petaling Jaya, Selangor, Malaysia

    Parvathy Rajendran

Authors
  1. Kai Yit Kok
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Parvathy Rajendran
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KKY and PR helped in conceptualization, methodology, validation, formal analysis, investigation, data curation, writing—review and editing, visualisation; KKY contributed to software, resources, writing—original draft preparation; PR was involved in supervision, project administration, and funding acquisition. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Parvathy Rajendran.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kok, K.Y., Rajendran, P. A constraints-based approach using ranking-gradient-similarity multi-block matching algorithm. Neural Comput & Applic 35, 15615–15627 (2023). https://doi.org/10.1007/s00521-023-08574-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08574-1

Keywords

Search

Navigation