Abstract
The preservation of object shape and uniform-area flatness in depth maps estimated by local stereo marching approaches is critical and still challenging. Toward the achievement of such preservation in the resulted depth map, we promote the stereo-matching algorithm by exploring and creating the causal relations while realizing the algorithmic construction in this paper. By spatial attentive causality, the decision of selected anchor pixel precedes for imposing its disparity as the range constraint of disparity searching on the other pixels in every horizontal line. By contextual attentive causality, the proportional factor of local-to-global gradient density can be applied to select the size of adaptive support weight (ASW) window. For feature-causality creation, the major cost term is delivered as the model parameter to compute other cost terms for forging the rational tie-in links between cost terms. During the comparison of ASW matching costs, the possible candidates are simultaneously confirmed for substituting the suspicious primary disparity in the subsequent content-aware refinement. For inducing the spatial attentive causality, the super-pixels are commended to enhance the cost weighting and the hole filling which is an effective dual inpainting routine. By realization and deployment of the causalities, the depth maps resulted from the proposed adaptive-supporting method will be apt to acquire the preservation of object shape and the suppression of over-shooting error in flat regions. The experimental results demonstrate that the proposed method based on the causality findings can obtain low bad-pixel rate and high visualization quality, both.




Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Ssstrunk, S. (2010). SLIC Superpixel. École Polytechnique Fédérale de Lausanne Technical report, 2010.
Aggarwal, A., & Kumar, M. (2021). Image surface texture analysis and classification using deep learning. Multimedia Tools and Applications Multimedia Tools Applications, 80, 1289–1309.
Chang, T. A., Lu, X., & Yang, J. F. (2017). Robust stereo marching with trinary cross color census and triple image-based refinements. EURASIP Journal on Advances in Signal Processing. https://doi.org/10.1186/s13634-017-0462-3
Chen, H., Huang, J., Remil, O., Xie, H., Qin, J., Guo, Y., Wei, M., & Wang, J. (2019). Structure-guided shape-preserving mesh texture smoothing via joint low-rank matrix recovery. Computer-Aided Design, 115, 122–134.
Chen, W., Zhang, M.-J., & Xiong, Z.-H. (2011). Fast semi-global stereo matching via extracting backup candidates from region boundaries. IET Computer Vision, 5(2), 143–150.
Criminisi, A., Perez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212.
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2015). Vision meets Robotics: The KITTI Dataset. Online datasets: http://www.cvlibs.net/datasets/kitti/raw_data.php.
Hamzah, R. A., Ibrahim, H., & Hassan, A. H. A. (2017). Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation. Journal of Visual Communication and Image Representation, 42, 145–160.
Kuo, P. C., Lo, K. L., Tseng, H. K., Lee, K. T., Liu, B. D., & Yan, J. F. (2018). Stereoview to Multiview conversion architecture for auto-stereoscopic 3D displays. IEEE Transactions on Circuits and Systems for Video Technology, 28(11), 3274–3287.
Lee, Z., Juang, J., & Nguyen, T. Q. (2013). Local disparity estimation with three-moded cross census and advanced support weight. IEEE Transactions on Multimedia, 15(8), 1855–1864.
Liu, J., Zhou, Z., Xu, W., & Hu, J. (2019). Adaptive support-weight stereo-matching approach with two disparity refinement steps. IETE Journal of Research, 65(3), 310–319.
Mozerov, M. G., & Weijer, J. (2015). Accurate stereo matching by two-step energy minimization. IEEE Transactions on Image Processing, 24(3), 1153–1163.
Ok, S. H., Shim, J. H., & Moon, B. (2018). Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors. The Journal of Supercomputing, 74(12), 6665–6690.
Scharstein, D, Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nesic, N., Wang, X., & Westling, P. (2014). High-accuracy stereo depth maps using structured light. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Middlebury.
Scharstein, D, & Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 195–202. Middlebury Online datasets: http://vision.middleburry.edu/stereo/data/.
Veksler, O. (2006). Reducing search space for stereo correspondence with graph cuts. In Proceedings of the British Mach. Vis. Conf., (pp. 73.1–73.10).
Wang, Y., Tung, C., & Chung, P. (2013). Efficient disparity estimation using hierarchical bilateral disparity structure based graph cut algorithm with a foreground boundary refinement mechanism. IEEE Transactions on Circuits and Systems for Video Technology, 23(5), 784–801.
Yang, Q. (2012). A non-local cost aggregation method for stereo matching. In 2012 IEEE Conference on Computer Vision and Pattern Recognition.
Yang, Q. (2015). Stereo matching using tree filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(4), 834–846.
Yang, Q., Wang, L., Yang, R., Stewénius, H., & Nistér, D. (2009). Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(3), 492–504.
Yoon, K. J., & Kweon, I. S. (2006). Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 650–656.
Zbontar, J., & LeCun, Y. (2015). Computing the stereo matching cost with a convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Zhang, K., Lu, J., & Lafruit, G. (2009). Cross-based local stereo matching using orthogonal integral images. IEEE Transactions on Circuits and Systems for Video Technology, 19(7), 1073–1079.
Zhu, S., & Li, Z. (2015). Local stereo matching using combined matching cost and adaptive cost aggregation. IEEE Transactions on Internet and Information Systems, 9(1), 224–240.
Zhu, S., Wang, Z., Zhang, X., & Li, Y. (2016). Edge-preserving guided filtering based cost aggregation for stereo matching. Journal of Visual Communication and Image Representation, 39, 107–119.
Acknowledgements
This investigation is supported by Ministry of Science and Technology 107-2221-E-415-016-MY2, Taiwan, ROC. Moreover, it needs to thank the initial contribution in simulations from Mr. Yu-Ching Chen, who had achieved the M.S degree of Computer Science and Information Engineering at 2018.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chan, DY., Chiu, TY. & Wu, XW. A causality-attentive stereo matching method for shape-preserved depth map. Multidim Syst Sign Process 33, 1203–1219 (2022). https://doi.org/10.1007/s11045-022-00838-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11045-022-00838-8