Skip to main content

Advertisement

SpringerLink
Log in

Research and implementation of adaptive stereo matching algorithm based on ZYNQ

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Stereo matching is an important method in computer vision for simulating human binocular vision to acquire spatial distance information. Implementing high-precision and real-time stereo-matching algorithms on hardware platforms with limited resources remains a significant challenge. Although the semi-global stereo-matching algorithm strikes a good balance between obtaining accuracy in the disparity map and computational complexity, it uses a fixed window for matching, resulting in lower matching accuracy in image regions with depth discontinuities and weak textures. To address the shortcomings of existing semi-global stereo-matching algorithms, an adaptive window semi-global stereo-matching algorithm is proposed, along with post-processing disparity optimization through left–right consistency check and median filtering. On test images provided by the Middlebury dataset, the average matching accuracy improved by 5.07% compared to traditional-matching algorithms. This algorithm is implemented on a Zynq UltraScale + chip, utilising 42,072 LUTs, 66,532 registers, and 101 BRAMs for the entire stereo-matching architecture. For images with a resolution of 1280 × 720 and 64 disparity levels, the final-processing speed can reach 54.24 fps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Perri, S., et al.: Stereo vision architecture for heterogeneous systems-on-chip. J. Real-Time Image Proc. 17(2), 393–415 (2018)

    Article  MathSciNet  Google Scholar 

  2. Lu, K., et al. Binocular stereo vision based on OpenCV; proceedings of the Smart and Sustainable City (ICSSC 2011), IET International Conference on, F, 2011 [C].

  3. Brown, M.Z., Burschka, D., Hager, G.D.: Advances in computational stereo. IEEE Trans. Pattern Anal. Mach. Intell. 25(8), 993–1008 (2003)

    Article  Google Scholar 

  4. Kriegman, D.J., Triendl, E., Binford, T.O.: Stereo vision and navigation in buildings for mobile robots. IEEE Trans. Robot. Autom. 5(6), 792–803 (1989)

    Article  Google Scholar 

  5. Dong, P., et al.: A 4.29 nJ/pixel stereo depth coprocessor with pixel level pipeline and region optimized semi-global matching for IoT application. IEEE Trans. Circuits Syst. I Regul. Pap. 69(1), 334–346 (2022)

    Article  Google Scholar 

  6. Bertozzi, M., et al.: Stereo vision-based vehicle detection. In: Proceedings of the Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat No 00TH8511). IEEE (2000)

  7. Mckinnon, D., Smith, R.N., Upcroft, B.: A semi-local method for iterative depth-map refinement. In: Proceedings of the 2012 IEEE International Conference on Robotics and Automation. IEEE (2012)

  8. De Silva, D., et al.: Improved depth map filtering for 3D-TV systems. In: Proceedings of the 2011 IEEE International Conference on Consumer Electronics (ICCE). IEEE (2011)

  9. Arief Setyawan, R., et al.: Implementation of stereo vision semi-global block matching methods for distance measurement. Indones. J. Electr. Eng. Comput. Sci. 12(2), 585–591 (2018)

    Google Scholar 

  10. Luo, G., Zhu, Y.: Hole filling for view synthesis using depth guided global optimization. IEEE Access 6, 32874–32889 (2018)

    Article  Google Scholar 

  11. Diebel, J., Thrun, S.: An application of markov random fields to range sensing. In: Advances in Neural Information Processing Systems, vol 18 (2005)

  12. Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2007)

    Article  Google Scholar 

  13. Zabih, R., Woodfill, J.: Non-parametric local transforms for computing visual correspondence. In: Proceedings of the Computer Vision—ECCV'94: Third European Conference on Computer Vision Stockholm, Sweden, May 2–6 1994 Proceedings, Volume II 3. Springer (1994)

  14. Li, J.: On fractional differential operators for image edge detection. Comput. Appl. Softw. 32(12), 5 (2015)

    Google Scholar 

  15. Shan, Y., et al.: Hardware acceleration for an accurate stereo vision system using mini-census adaptive support region. ACM Trans. Embed. Comput. Syst. 13(4s), 1–24 (2014)

    Article  Google Scholar 

  16. Ttofis, C., Kyrkou, C., Theocharides, T.: A low-cost real-time embedded stereo vision system for accurate disparity estimation based on guided image filtering. IEEE Trans. Comput. 65(9), 2678–2693 (2016)

    Article  MathSciNet  Google Scholar 

  17. Gehrig, S.K., Eberli, F., Meyer, T. A real-time low-power stereo vision engine using semi-global matching. In: Proceedings of the International Conference on Computer Vision Systems. Springer (2009)

  18. Wang, J., et al.: Low-resource hardware architecture for semi-global stereo matching. In: Proceedings of the 2019 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE (2019)

  19. Li, Y., et al.: High throughput hardware architecture for accurate semi-global matching. Integration 65, 417–427 (2019)

    Article  Google Scholar 

  20. Chai, Y., Cao, X.: Stereo matching algorithm based on joint matching cost and adaptive window. In: Proceedings of the 2018 IEEE 3rd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). IEEE (2018)

  21. Razak, S., Othman, M.A., Kadmin, A.F.: The effect of adaptive weighted bilateral filter on stereo matching algorithm. IJEAT 8(3), 2249–8958 (2019)

    Google Scholar 

  22. Ke, Z., Jiangbo, L., Lafruit, G.: Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circuits Syst. Video Technol. 19(7), 1073–1079 (2009)

    Article  Google Scholar 

  23. Li, J., et al.: Bayesian stereo matching method based on edge constraints. Int. J. Adv. Comput .Technol. 4(22), 36–47 (2012)

    Google Scholar 

  24. Kowalczuk, J., Psota, E.T., Perez, L.C.: Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE Trans. Circuits Syst. Video Technol. 23(1), 94–104 (2013)

    Article  Google Scholar 

  25. Wang, Y., et al.: Improvement of AD-census algorithm based on stereo vision. Sensors (Basel) 22(18), 6933 (2022)

    Article  Google Scholar 

  26. Peng, Z., Wu, L., Xiao, B.: High-speed dense matching algorithm for high-resolution aerial image based on CPU-FPGA. Vis. Comput. 39, 5263–5278 (2023)

    Article  Google Scholar 

  27. Wang, W., et al.: Real-time high-quality stereo vision system in FPGA. IEEE Trans. Circuits Syst. Video Technol. 25(10), 1696–1708 (2015)

    Article  MathSciNet  Google Scholar 

  28. Ma, Y., et al.: Five-direction occlusion filling with five layer parallel two-stage pipeline for stereo matching with sub-pixel disparity map estimation. Sensors (Basel) 22(22), 8605 (2022)

    Article  Google Scholar 

  29. Lu, Z., et al.: A resource-efficient pipelined architecture for real-time semi-global stereo matching. IEEE Trans. Circuits Syst. Video Technol. 32(2), 660–673 (2022)

    Article  Google Scholar 

  30. Sabihuddin, S., Islam, J., Maclean, W.J.: Dynamic programming approach to high frame-rate stereo correspondence: a pipelined architecture implemented on a field programmable gate array. In: Proceedings of the 2008 Canadian Conference on Electrical and Computer Engineering. IEEE (2008)

  31. Jin, M., Maruyama, T.: A real-time stereo vision system using a tree-structured dynamic programming on FPGA. In: Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays, 2012

  32. Banz, C., et al.: Real-time stereo vision system using semi-global matching disparity estimation: architecture and FPGA-implementation. In: Proceedings of the 2010 International Conference on Embedded Computer Systems: Architectures, Modeling and Simulation. IEEE (2010)

  33. Zhao, G., Zhang, L., Wu, F.: Application of improved median filtering algorithm to image de-noising. J. Appl. Opt. 32(4), 5 (2011)

    Google Scholar 

  34. Han, J., Wu, Z., Li, L.: FPGA implementation for binocular stereo matching algorithm. J. Harbin Univ. Sci. Technol. 21(004), 25–29 (2016)

    Google Scholar 

  35. Jia, Y., et al.: A miniature stereo vision machine (MSVM-III) for dense disparity mapping. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004 ICPR 2004. IEEE (2004)

  36. Perri, S., et al.: Design of real-time FPGA-based embedded system for stereo vision. In: Proceedings of the 2018 IEEE international symposium on circuits and systems (ISCAS). IEEE (2018)

  37. Pan, Y., et al.: A hardware/software co-design approach for real-time binocular stereo vision based on ZYNQ (short paper). In: Proceedings of the Collaborative Computing: Networking, Applications and Worksharing: 14th EAI International Conference, CollaborateCom 2018, Shanghai, China, December 1–3, 2018, Proceedings 14. Springer (2019)

  38. Wu, G., Yang, J., Yang, H.: Real-time low-power binocular stereo vision based on FPGA. J. Real-Time Image. Proc. 19, 29–39 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

This work was financially supported by the Science and Technology Program of Guangxi, China (No. 2018AD19184), the Project of Guangxi Education Department of China (No. 2018KY0258) and the Project of the Guilin University of Technology (No. GLUTQD2017003).

Author information

Authors and Affiliations

  1. Key Laboratory of Advanced Manufacturing and Automation Technology (Guilin University of Technology), Education Department of Guangxi Zhuang Autonomous Region, Guilin, 541006, China

    Yong Liang, Daoqian Lin, Zetao Chen, Yan Zhi, Junwen Tan & Zhenhao Yang

  2. College of Mechanical and Control Engineering, Guilin University of Technology, Guilin, 541006, China

    Yong Liang, Daoqian Lin, Zetao Chen, Yan Zhi, Junwen Tan & Jie Li

Authors
  1. Yong Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daoqian Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zetao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yan Zhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junwen Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhenhao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jie Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Author Contributions: Conceptualization, D.L.; data curation, Y.L. and Z.C.; simulation, Z.Y. and J.L.; writing—original draft preparation, Z.C. and J.T.; writing—review and editing, Z.Y. and J.L.; supervision, D.L. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Daoqian Lin.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, Y., Lin, D., Chen, Z. et al. Research and implementation of adaptive stereo matching algorithm based on ZYNQ. J Real-Time Image Proc 21, 46 (2024). https://doi.org/10.1007/s11554-024-01428-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01428-6

Keywords

Search

Navigation