Abstract
Disparity estimation is a long-standing task in computer vision and multiple approaches have been proposed to solve this problem. A recent work based on convolutional neural networks, which uses a correlation layer to perform the matching process, has achieved state-of-the-art results for the disparity estimation task. This correlation layer employs a single kernel unit which is not suitable for low texture content and repeated patterns. In this paper we tackle this problem by using a multi-scale correlation layer with several correlation kernels and different scales. The major target is to integrate the information of the local matching process by combining the benefits of using both a small correlating scale for fine details and bigger scales for larger areas. Furthermore, we investigate the training approach using horizontally elongated patches that fits the disparity estimation task. The results obtained demonstrate the benefits of the proposed approach on both synthetic and real images.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)
De-Maeztu, L., Mattoccia, S., Villanueva, A., Cabeza, R.: Linear stereo matching. In: Proceedings of the 2011 International Conference on Computer Vision, Barcelona, pp. 1708–1715 (2011)
Sun, J., Zheng, N.-N., Shum, H.-Y.: Stereo matching using belief propagation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, TPAMI, vol. 25, no. 7, pp. 787–800 (2003)
Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Patt. Anal. Mach. Intell. 30(2), 328–341 (2008)
Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., Navab, N.: Deeper depth prediction with fully convolutional residual networks. CoRR abs/1606.00373 (2016)
Mayer, N., Ilg, E., Hausser, P., Fischer, P., Cremers, D., Dosovitskiy, A., Brox, T.: A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 4040–4048 (2016)
Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4353–4361 (2015)
Zbontar, J., LeCun, Y.: Computing the stereo matching cost with a convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1592–1599 (2014)
Luo, W., Schwing, A.G., Urtasun, R.: Efficient deep learning for stereo matching. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 5695–5703 (2016)
Zbontar, J., LeCun, Y.: Stereo matching by training a convolutional neural network to compare image patches. J. Mach. Learn. Res. 17, 4 (2016)
Seki, A., Pollefeys, M.: Patch based confidence prediction for dense disparity map. In: Proceedings of the British Machine Vision Conference, BMVC, p. 23 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Jammal, S., Tillo, T., Xiao, J. (2017). Disparity Estimation Using Convolutional Neural Networks with Multi-scale Correlation. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10636. Springer, Cham. https://doi.org/10.1007/978-3-319-70090-8_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-70090-8_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70089-2
Online ISBN: 978-3-319-70090-8
eBook Packages: Computer ScienceComputer Science (R0)