Skip to main content
Log in

A causality-attentive stereo matching method for shape-preserved depth map

  • Published:
Multidimensional Systems and Signal Processing Aims and scope Submit manuscript

Abstract

The preservation of object shape and uniform-area flatness in depth maps estimated by local stereo marching approaches is critical and still challenging. Toward the achievement of such preservation in the resulted depth map, we promote the stereo-matching algorithm by exploring and creating the causal relations while realizing the algorithmic construction in this paper. By spatial attentive causality, the decision of selected anchor pixel precedes for imposing its disparity as the range constraint of disparity searching on the other pixels in every horizontal line. By contextual attentive causality, the proportional factor of local-to-global gradient density can be applied to select the size of adaptive support weight (ASW) window. For feature-causality creation, the major cost term is delivered as the model parameter to compute other cost terms for forging the rational tie-in links between cost terms. During the comparison of ASW matching costs, the possible candidates are simultaneously confirmed for substituting the suspicious primary disparity in the subsequent content-aware refinement. For inducing the spatial attentive causality, the super-pixels are commended to enhance the cost weighting and the hole filling which is an effective dual inpainting routine. By realization and deployment of the causalities, the depth maps resulted from the proposed adaptive-supporting method will be apt to acquire the preservation of object shape and the suppression of over-shooting error in flat regions. The experimental results demonstrate that the proposed method based on the causality findings can obtain low bad-pixel rate and high visualization quality, both.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., & Ssstrunk, S. (2010). SLIC Superpixel. École Polytechnique Fédérale de Lausanne Technical report, 2010.

  • Aggarwal, A., & Kumar, M. (2021). Image surface texture analysis and classification using deep learning. Multimedia Tools and Applications Multimedia Tools Applications, 80, 1289–1309.

    Article  Google Scholar 

  • Chang, T. A., Lu, X., & Yang, J. F. (2017). Robust stereo marching with trinary cross color census and triple image-based refinements. EURASIP Journal on Advances in Signal Processing. https://doi.org/10.1186/s13634-017-0462-3

    Article  Google Scholar 

  • Chen, H., Huang, J., Remil, O., Xie, H., Qin, J., Guo, Y., Wei, M., & Wang, J. (2019). Structure-guided shape-preserving mesh texture smoothing via joint low-rank matrix recovery. Computer-Aided Design, 115, 122–134.

    Article  Google Scholar 

  • Chen, W., Zhang, M.-J., & Xiong, Z.-H. (2011). Fast semi-global stereo matching via extracting backup candidates from region boundaries. IET Computer Vision, 5(2), 143–150.

    Article  Google Scholar 

  • Criminisi, A., Perez, P., & Toyama, K. (2004). Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing, 13(9), 1200–1212.

    Article  Google Scholar 

  • Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2015). Vision meets Robotics: The KITTI Dataset. Online datasets: http://www.cvlibs.net/datasets/kitti/raw_data.php.

  • Hamzah, R. A., Ibrahim, H., & Hassan, A. H. A. (2017). Stereo matching algorithm based on per pixel difference adjustment, iterative guided filter and graph segmentation. Journal of Visual Communication and Image Representation, 42, 145–160.

    Article  Google Scholar 

  • Kuo, P. C., Lo, K. L., Tseng, H. K., Lee, K. T., Liu, B. D., & Yan, J. F. (2018). Stereoview to Multiview conversion architecture for auto-stereoscopic 3D displays. IEEE Transactions on Circuits and Systems for Video Technology, 28(11), 3274–3287.

    Article  Google Scholar 

  • Lee, Z., Juang, J., & Nguyen, T. Q. (2013). Local disparity estimation with three-moded cross census and advanced support weight. IEEE Transactions on Multimedia, 15(8), 1855–1864.

    Article  Google Scholar 

  • Liu, J., Zhou, Z., Xu, W., & Hu, J. (2019). Adaptive support-weight stereo-matching approach with two disparity refinement steps. IETE Journal of Research, 65(3), 310–319.

    Article  Google Scholar 

  • Mozerov, M. G., & Weijer, J. (2015). Accurate stereo matching by two-step energy minimization. IEEE Transactions on Image Processing, 24(3), 1153–1163.

    Article  MathSciNet  MATH  Google Scholar 

  • Ok, S. H., Shim, J. H., & Moon, B. (2018). Modified adaptive support weight and disparity search range estimation schemes for stereo matching processors. The Journal of Supercomputing, 74(12), 6665–6690.

    Article  Google Scholar 

  • Scharstein, D, Hirschmüller, H., Kitajima, Y., Krathwohl, G., Nesic, N., Wang, X., & Westling, P. (2014). High-accuracy stereo depth maps using structured light. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Middlebury.

  • Scharstein, D, & Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 195–202. Middlebury Online datasets: http://vision.middleburry.edu/stereo/data/.

  • Veksler, O. (2006). Reducing search space for stereo correspondence with graph cuts. In Proceedings of the British Mach. Vis. Conf., (pp. 73.1–73.10).

  • Wang, Y., Tung, C., & Chung, P. (2013). Efficient disparity estimation using hierarchical bilateral disparity structure based graph cut algorithm with a foreground boundary refinement mechanism. IEEE Transactions on Circuits and Systems for Video Technology, 23(5), 784–801.

    Article  Google Scholar 

  • Yang, Q. (2012). A non-local cost aggregation method for stereo matching. In 2012 IEEE Conference on Computer Vision and Pattern Recognition.

  • Yang, Q. (2015). Stereo matching using tree filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(4), 834–846.

    Article  Google Scholar 

  • Yang, Q., Wang, L., Yang, R., Stewénius, H., & Nistér, D. (2009). Stereo matching with color-weighted correlation, hierarchical belief propagation, and occlusion handling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(3), 492–504.

    Article  Google Scholar 

  • Yoon, K. J., & Kweon, I. S. (2006). Adaptive support-weight approach for correspondence search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4), 650–656.

    Article  Google Scholar 

  • Zbontar, J., & LeCun, Y. (2015). Computing the stereo matching cost with a convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

  • Zhang, K., Lu, J., & Lafruit, G. (2009). Cross-based local stereo matching using orthogonal integral images. IEEE Transactions on Circuits and Systems for Video Technology, 19(7), 1073–1079.

    Article  Google Scholar 

  • Zhu, S., & Li, Z. (2015). Local stereo matching using combined matching cost and adaptive cost aggregation. IEEE Transactions on Internet and Information Systems, 9(1), 224–240.

    Google Scholar 

  • Zhu, S., Wang, Z., Zhang, X., & Li, Y. (2016). Edge-preserving guided filtering based cost aggregation for stereo matching. Journal of Visual Communication and Image Representation, 39, 107–119.

    Article  Google Scholar 

Download references

Acknowledgements

This investigation is supported by Ministry of Science and Technology 107-2221-E-415-016-MY2, Taiwan, ROC. Moreover, it needs to thank the initial contribution in simulations from Mr. Yu-Ching Chen, who had achieved the M.S degree of Computer Science and Information Engineering at 2018.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Din-Yuen Chan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chan, DY., Chiu, TY. & Wu, XW. A causality-attentive stereo matching method for shape-preserved depth map. Multidim Syst Sign Process 33, 1203–1219 (2022). https://doi.org/10.1007/s11045-022-00838-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11045-022-00838-8

Keywords

Navigation