skip to main content
10.1145/3109984.3109992acmconferencesArticle/Chapter ViewAbstractPublication PagessbcciConference Proceedingsconference-collections
research-article

Hardware module for low-resource and real-time stereo vision engine using semi-global matching approach

Published:28 August 2017Publication History

ABSTRACT

Stereo matching systems that generate dense, accurate, robust and real-time disparity maps are quite attractive for a variety of applications. Most of the existing stereo matching systems that fulfill to all of these requirements adopt the Semi-Global Matching (SGM) technique. This work proposes a scalable architecture based on a systolic array, fully pipeline. The design builds on a combination of multi-level parallelisms such as image line processing (two-dimensional processing) and disparity. The implementation of the SGM technique combined with the gradiente filter sobel as a pre-processing step and absolute differences as a local similarity method is a very robust approach to both high-quality images from the latest version of the Middlebury image database (22.7% of bad pixels) and low-quality images from our stereo camera system. This whole system was implemented in the FPGA platform Cyclone IV generating images of disparity in HD resolution (1024×768 pixel), with the range of 128 levels of disparities values and using four directions of paths for the SGM method. The proposed approach provided an operating frequency of 100Mhz, delivering images at a rate of 127 frames per second, using 70% of its resource in logical elements for processing and 63% of internal memory for intermediate data storage. So the proposed architecture fills the demands of real world applications regarding frame rate, depth resolution, low resource usage and accuracy.

References

  1. Christian Banz, Sebastian Hesselbarth, Holger Flatt, Holger Blume, and Peter Pirsch. 2010. Real-time stereo vision system using semi-global matching disparity estimation: Architecture and FPGA-implementation. In Embedded Computer Systems (SAMOS), 2010 International Conference on. IEEE, 93--101.Google ScholarGoogle ScholarCross RefCross Ref
  2. Jonathan T Barron and Ben Poole. 2016. The fast bilateral solver. In European Conference on Computer Vision. Springer, 617--632.Google ScholarGoogle ScholarCross RefCross Ref
  3. Stefan K Gehrig, Felix Eberli, and Thomas Meyer. 2009. A real-time low-power stereo vision engine using semi-global matching. In International Conference on Computer Vision Systems. Springer, 134--143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Stefan K Gehrig and Clemens Rabe. 2010. Real-time semi-global matching on the CPU. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 85--92.Google ScholarGoogle ScholarCross RefCross Ref
  5. Richard Hartley and Andrew Zisserman. 2003. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press, New York, NY, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Heiko Hirschmuller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence 30, 2 (2008), 328--341. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Heiko Hirschmuller and Daniel Scharstein. 2009. Evaluation of stereo matching costs on images with radiometric differences. IEEE transactions on pattern analysis and machine intelligence 31, 9 (2009), 1582--1599. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Dominik Honegger, Helen Oleynikova, and Marc Pollefeys. 2014. Real-time and low latency embedded computer vision hardware based on a combination of fpga and mobile cpu. In Intelligent Robots and Systems (IROS 2014), 2014IEEE/RSJ International Conference on. IEEE, 4930--4935.Google ScholarGoogle ScholarCross RefCross Ref
  9. Kyung-Rae Kim and Chang-Su Kim. 2016. Adaptive smoothness constraints for efficient stereo matching using texture and edge information. In Image Processing (ICIP), 2016 IEEE International Conference on. IEEE, 3429--3433.Google ScholarGoogle ScholarCross RefCross Ref
  10. Jedrzej Kowalczuk, Eric T Psota, and Lance C Perez. 2013. Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE transactions on circuits and systems for video technology 23, 1 (2013), 94--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. David Kriesel. 2007. A brief Introduction on Neural Networks. (2007).Google ScholarGoogle Scholar
  12. Xing Mei, Xun Sun, Mingcai Zhou, Shaohui Jiao, Haitao Wang, and Xiaopeng Zhang. 2011. On building an accurate stereo matching system on graphics hardware. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. IEEE, 467--474.Google ScholarGoogle ScholarCross RefCross Ref
  13. Haesol Park and Kyoung Mu Lee. 2016. Look Wider to Match Image Patches with Convolutional Neural Networks. IEEE Signal Processing Letters (2016).Google ScholarGoogle Scholar
  14. Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Nešić, Xi Wang, and Porter Westling. 2014. High-resolution stereo datasets with subpixel-accurate ground truth. In German Conference on Pattern Recognition. Springer, 31--42.Google ScholarGoogle ScholarCross RefCross Ref
  15. Daniel Scharstein and Richard Szeliski. 2002. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. International Journal of Computer Vision 47, 1 (2002), 7--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Robert Spangenberg, Tobias Langner, and Raúl Rojas. 2013. Weighted semi-global matching and center-symmetric census transform for robust driver assistance. In International Conference on Computer Analysis of Images and Patterns. Springer, 34--41.Google ScholarGoogle ScholarCross RefCross Ref
  17. Pascal Steingrube, Stefan K Gehrig, and Uwe Franke. 2009. Performance evaluation of stereo algorithms for automotive applications. In International Conference on Computer Vision Systems. Springer, 285--294. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. C Stentoumis, L Grammatikopoulos, I Kalisperakis, and G Karras. 2014. On accurate dense stereo-matching using a local adaptive multi-cost approach. ISPRS Journal of Photogrammetry and Remote Sensing 91 (2014), 29--49.Google ScholarGoogle Scholar
  19. W. Wang, J. Yan, N. Xu, Y. Wang, and F. H. Hsu. 2013. Real-time high-quality stereo vision system in FPGA. In 2013 International Conference on Field-Programmable Technology (FPT). 358--361.Google ScholarGoogle Scholar
  20. Chi Zhang, Zhiwei Li, Yanhua Cheng, Rui Cai, Hongyang Chao, and Yong Rui. 2015. Meshstereo: A global stereo model with mesh alignment regularization for view interpolation. In Proceedings of the IEEE International Conference on Computer Vision. 2057--2065. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Hardware module for low-resource and real-time stereo vision engine using semi-global matching approach

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in
          • Published in

            cover image ACM Conferences
            SBCCI '17: Proceedings of the 30th Symposium on Integrated Circuits and Systems Design: Chip on the Sands
            August 2017
            238 pages
            ISBN:9781450351065
            DOI:10.1145/3109984

            Copyright © 2017 ACM

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 28 August 2017

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article

            Acceptance Rates

            Overall Acceptance Rate133of347submissions,38%

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader