Abstract
Stereo vision is a low cost and passive mechanism to perceive the environment for robotic applications. The huge compute requirements of stereo vision algorithms have been a major challenge for their usage in real world applications on small robots. Standard stereo depth estimation algorithms Sum of Absolute Differences (SAD), census transform and an advanced algorithm Semi-Global Matching (SGM) are discussed in this work. This paper presents novel real time implementation of these three stereo vision algorithms on two different compute platforms i) Intel AVX (Advanced Vector Extension) and ii) Nvidia Jetson GPU (Graphical Processing Unit). The Intel CPU implementation of stereo algorithms is optimized by using OpenMP (Open Multi-Processing) for multi-threading, AVX registers for vectorization and several other novel ideas for real time processing. Nvidia Jetson implementation is efficiently designed for maximum speed-up on a low end GPU such as Jetson TK1. Post processing steps such as local extrema detection, left-right consistency and median filter are used to improve the final disparity image. We have achieved speedup of the order of 30x when compared with naïve CPU implementation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source slam system for monocular, stereo, and RGB-D cameras. IEEE Trans. Rob. 33(5), 1255–1262 (2017)
Broggi, A., Caraffi, C., Fedriga, R.I., Grisleri, P.: Obstacle detection with stereo vision for off-road vehicle navigation. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005)-Workshops (2005)
Anderson, C.S., Zhang, J., Cornea, M.: Enhanced vector math support on the Intel® AVX-512 architecture. In: 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH) (2018)
Fusiello, A., Trucco, E., Verri, A.: A compact algorithm for rectification of stereo pairs. Mach. Vis. Appl. 12(1), 16–22 (2000). https://doi.org/10.1007/s001380050120
Mattoccia, S., Tombari, F., Di Stefano, L.: Fast full-search equivalent template matching by enhanced bounded correlation. IEEE Trans. Image Process. 17(4), 528–538 (2008)
Banks, J., Bennamoun, M., Corke, P.: Non-parametric techniques for fast and robust stereo matching. In: TENCON 1997 Brisbane-Australia, Proceedings of IEEE TENCON 1997, IEEE Region 10 Annual Conference, Speech and Image Technologies for Computing and Telecommunications (Cat. No. 97CH36162) (1997)
Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: theory and experiment. IEEE Trans. Pattern Anal. Mach. Intell. 16(9), 920–932 (1994)
Fusiello, A., Roberto, V., Trucco, E.: Efficient stereo with multiple windowing. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1997)
Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: Sandini, G. (ed.) ECCV 1992. LNCS, vol. 588, pp. 237–252. Springer, Heidelberg (1992). https://doi.org/10.1007/3-540-55426-2_27
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2007)
Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 (2001)
Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: FlowNet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Zhong, Y., Dai, Y., Li, H.: Self-supervised learning for stereo matching with self-improving ability. arXiv preprint arXiv:1709.00930 (2017)
Fan, R., Dahnoun, N.: Real-time implementation of stereo vision based on optimised normalised cross-correlation and propagated search range on a GPU. In: 2017 IEEE International Conference on Imaging Systems and Techniques (IST) (2017)
Honegger, D., Oleynikova, H., Pollefeys, M.: Real-time and low latency embedded computer vision hardware based on a combination of FPGA and mobile CPU. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (2014)
Hernandez-Juarez, D., Chacón, A., Espinosa, A., Vázquez, D., Moure, J.C., López, A.M.: Embedded real-time stereo estimation via semi-global matching on the GPU. Procedia Comput. Sci. 80, 143–153 (2016)
Cui, H., Dahnoun, N.: Real-time stereo vision implementation on Nvidia Jetson TX2. In: 2019 8th Mediterranean Conference on Embedded Computing (MECO). IEEE (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Syed, I.A., Datar, M., Patkar, S. (2021). Accelerated Stereo Vision Using Nvidia Jetson and Intel AVX. In: Singh, S.K., Roy, P., Raman, B., Nagabhushan, P. (eds) Computer Vision and Image Processing. CVIP 2020. Communications in Computer and Information Science, vol 1377. Springer, Singapore. https://doi.org/10.1007/978-981-16-1092-9_12
Download citation
DOI: https://doi.org/10.1007/978-981-16-1092-9_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-1091-2
Online ISBN: 978-981-16-1092-9
eBook Packages: Computer ScienceComputer Science (R0)