ABSTRACT
Stereo matching systems that generate dense, accurate, robust and real-time disparity maps are quite attractive for a variety of applications. Most of the existing stereo matching systems that fulfill to all of these requirements adopt the Semi-Global Matching (SGM) technique. This work proposes a scalable architecture based on a systolic array, fully pipeline. The design builds on a combination of multi-level parallelisms such as image line processing (two-dimensional processing) and disparity. The implementation of the SGM technique combined with the gradiente filter sobel as a pre-processing step and absolute differences as a local similarity method is a very robust approach to both high-quality images from the latest version of the Middlebury image database (22.7% of bad pixels) and low-quality images from our stereo camera system. This whole system was implemented in the FPGA platform Cyclone IV generating images of disparity in HD resolution (1024×768 pixel), with the range of 128 levels of disparities values and using four directions of paths for the SGM method. The proposed approach provided an operating frequency of 100Mhz, delivering images at a rate of 127 frames per second, using 70% of its resource in logical elements for processing and 63% of internal memory for intermediate data storage. So the proposed architecture fills the demands of real world applications regarding frame rate, depth resolution, low resource usage and accuracy.
- Christian Banz, Sebastian Hesselbarth, Holger Flatt, Holger Blume, and Peter Pirsch. 2010. Real-time stereo vision system using semi-global matching disparity estimation: Architecture and FPGA-implementation. In Embedded Computer Systems (SAMOS), 2010 International Conference on. IEEE, 93--101.Google ScholarCross Ref
- Jonathan T Barron and Ben Poole. 2016. The fast bilateral solver. In European Conference on Computer Vision. Springer, 617--632.Google ScholarCross Ref
- Stefan K Gehrig, Felix Eberli, and Thomas Meyer. 2009. A real-time low-power stereo vision engine using semi-global matching. In International Conference on Computer Vision Systems. Springer, 134--143. Google ScholarDigital Library
- Stefan K Gehrig and Clemens Rabe. 2010. Real-time semi-global matching on the CPU. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 85--92.Google ScholarCross Ref
- Richard Hartley and Andrew Zisserman. 2003. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press, New York, NY, USA. Google ScholarDigital Library
- Heiko Hirschmuller. 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence 30, 2 (2008), 328--341. Google ScholarDigital Library
- Heiko Hirschmuller and Daniel Scharstein. 2009. Evaluation of stereo matching costs on images with radiometric differences. IEEE transactions on pattern analysis and machine intelligence 31, 9 (2009), 1582--1599. Google ScholarDigital Library
- Dominik Honegger, Helen Oleynikova, and Marc Pollefeys. 2014. Real-time and low latency embedded computer vision hardware based on a combination of fpga and mobile cpu. In Intelligent Robots and Systems (IROS 2014), 2014IEEE/RSJ International Conference on. IEEE, 4930--4935.Google ScholarCross Ref
- Kyung-Rae Kim and Chang-Su Kim. 2016. Adaptive smoothness constraints for efficient stereo matching using texture and edge information. In Image Processing (ICIP), 2016 IEEE International Conference on. IEEE, 3429--3433.Google ScholarCross Ref
- Jedrzej Kowalczuk, Eric T Psota, and Lance C Perez. 2013. Real-time stereo matching on CUDA using an iterative refinement method for adaptive support-weight correspondences. IEEE transactions on circuits and systems for video technology 23, 1 (2013), 94--104. Google ScholarDigital Library
- David Kriesel. 2007. A brief Introduction on Neural Networks. (2007).Google Scholar
- Xing Mei, Xun Sun, Mingcai Zhou, Shaohui Jiao, Haitao Wang, and Xiaopeng Zhang. 2011. On building an accurate stereo matching system on graphics hardware. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on. IEEE, 467--474.Google ScholarCross Ref
- Haesol Park and Kyoung Mu Lee. 2016. Look Wider to Match Image Patches with Convolutional Neural Networks. IEEE Signal Processing Letters (2016).Google Scholar
- Daniel Scharstein, Heiko Hirschmüller, York Kitajima, Greg Krathwohl, Nera Nešić, Xi Wang, and Porter Westling. 2014. High-resolution stereo datasets with subpixel-accurate ground truth. In German Conference on Pattern Recognition. Springer, 31--42.Google ScholarCross Ref
- Daniel Scharstein and Richard Szeliski. 2002. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms. International Journal of Computer Vision 47, 1 (2002), 7--42. Google ScholarDigital Library
- Robert Spangenberg, Tobias Langner, and Raúl Rojas. 2013. Weighted semi-global matching and center-symmetric census transform for robust driver assistance. In International Conference on Computer Analysis of Images and Patterns. Springer, 34--41.Google ScholarCross Ref
- Pascal Steingrube, Stefan K Gehrig, and Uwe Franke. 2009. Performance evaluation of stereo algorithms for automotive applications. In International Conference on Computer Vision Systems. Springer, 285--294. Google ScholarDigital Library
- C Stentoumis, L Grammatikopoulos, I Kalisperakis, and G Karras. 2014. On accurate dense stereo-matching using a local adaptive multi-cost approach. ISPRS Journal of Photogrammetry and Remote Sensing 91 (2014), 29--49.Google Scholar
- W. Wang, J. Yan, N. Xu, Y. Wang, and F. H. Hsu. 2013. Real-time high-quality stereo vision system in FPGA. In 2013 International Conference on Field-Programmable Technology (FPT). 358--361.Google Scholar
- Chi Zhang, Zhiwei Li, Yanhua Cheng, Rui Cai, Hongyang Chao, and Yong Rui. 2015. Meshstereo: A global stereo model with mesh alignment regularization for view interpolation. In Proceedings of the IEEE International Conference on Computer Vision. 2057--2065. Google ScholarDigital Library
Index Terms
Hardware module for low-resource and real-time stereo vision engine using semi-global matching approach
Recommendations
An FPGA-based real-time occlusion robust stereo vision system using semi-global matching
AbstractStereo matching approaches are an appealing choice for acquiring depth information in a number of video processing applications. It is desirable that these solutions generate dense, robust disparity maps in real time. However, occlusion regions ...
Low-cost FPGA stereo vision system for real time disparity maps calculation
Several applications demand efficient hardware implementations of stereo vision systems in order to furnish real time three-dimensional measurements. This paper proposes a complete fast low-cost stereo vision system that performs stereo image ...
A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching
ICVS '09: Proceedings of the 7th International Conference on Computer Vision Systems: Computer Vision SystemsMany real-time stereo vision systems are available on low-power platforms. They all either use a local correlation-like stereo engine or perform dynamic programming variants on a scan-line. However, when looking at high-performance global stereo methods ...
Comments