Abstract
Deep-learning-based stereo matching methods have achieved significant improvement over traditional methods and obtained great successes in recent years. However, how to trade off accuracy and speed and predict accurate disparity in real time has been a long-standing problem in the stereo matching area. We present an end-to-end light-weight convolutional neural network (CNN) to quickly estimate accurate disparity maps. Our proposed model is based on AnyNet, a real-time network which generates disparity in stages to achieve anytime prediction. Hourglass architecture with dilated convolutional layers is exploited to extract richer features of input stereo images. We also introduce residual connections in 2D CNN and 3D CNN to avoid information loss. Besides, we adopt a color guidance refinement to improve disparity performance. Depthwise separable convolution is used to replace standard convolution in color guidance refinement to sharply decrease the number of parameters and computational complexity. We refer to our proposed model as Light-Weight Stereo Network (LWSN). LWSN is trained and evaluated on three well-known stereo datasets. Experiments indicate that our model is effective and efficient.
This work was supported by National Key Research and Development Plan under Grant 2017YFB1301101, and in part by the National Natural Science Foundation of China through grant 61673317, 61673313 and 62076193.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Janai, J., et al.: Computer vision for autonomous vehicles: problems, datasets and state-of-the-art arXiv:1704.05519 (2017)
Wang, Y., et al.: Anytime stereo image depth estimation on mobile devices. In: International Conference on Robotics and Automation (ICRA), pp. 5893–5900 (2019)
Scharstein, D., Szeliski, R., Zabih, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. In: Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision, pp. 131–140 (2001)
Heiko, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)
Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. In: Sixth International Conference on Computer Vision, pp. 1073–1080 (1998)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, pp. 377–384 (1999)
Felzenszwalb, P.F., Huttenlocher, D.R.: Efficient belief propagation for early vision. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2004)
Hirschmuller, H.: Stereo processing by Semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)
Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. arXiv:1510.05970 (2016)
Luo, W., et al.: Efficient deep learning for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2016, pp. 5695–5703 (2016)
Mayer, N., et al.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2016, pp. 4040–4048 (2016)
Kendall, A., et al.: End-to-end learning of geometry and context for deep stereo regression. In: IEEE International Conference on Computer Vision (ICCV), vol. 2017, pp. 66–75 (2017)
Chang, J., Chen, Y.: Pyramid stereo matching network. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, vol. 2018, pp. 5410–5418 (2018)
Yang, G., Zhao, H., Shi, J., Deng, Z., Jia, J.: SegStereo: exploiting semantic information for disparity estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 660–676. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_39
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. CoRR:1511.07122 (2016)
He, K., et al.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2016, pp. 770–778 (2016)
Park, K., Kim, S., Sohn, K.: High-Precision Depth Estimation Using Uncalibrated LiDAR and Stereo Fusion. IEEE Trans. Intell. Transp. Syst. 21(1), 321–335 (2020)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2017, pp. 1800–1807 (2017)
Menze, M., Andreas, G.: Object scene flow for autonomous vehicles. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2015, pp. 3061–3070 (2015)
Geiger, A., et al.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2012, pp. 3354–3361 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, J., Duan, Z., Mei, K., Zhou, H., Tong, C. (2021). A Light-Weight Stereo Matching Network with Color Guidance Refinement. In: Sun, F., Liu, H., Fang, B. (eds) Cognitive Systems and Signal Processing. ICCSIP 2020. Communications in Computer and Information Science, vol 1397. Springer, Singapore. https://doi.org/10.1007/978-981-16-2336-3_46
Download citation
DOI: https://doi.org/10.1007/978-981-16-2336-3_46
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-2335-6
Online ISBN: 978-981-16-2336-3
eBook Packages: Computer ScienceComputer Science (R0)