Abstract
This paper deals with a novel stereo algorithm that can generate accurate dense disparity maps in real time. The algorithm employs an effective cross-based variable support aggregation strategy within a scanline optimization framework. Rather than matching intensities directly, the use of adaptive support aggregation allows for precisely handling the weak textured regions as well as depth discontinuities. To improve the disparity results with global reasoning, we reformulate the energy function on a tree structure over the whole 2D image area, as opposed to dynamic programming of individual scanlines. By applying both intra- and inter-scanline optimizations, the algorithm reduces the typical ’streaking’ artifact while maintaining high computational efficiency. The experimental results are evaluated on the Middlebury stereo dataset, showing that our approach is among the best for all real-time approaches. We implement the algorithm on a commodity graphics card with CUDA architecture, running at about 35 fames/s for a typical stereo pair with a resolution of 384×288 and 16 disparity levels.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bobick, A.F., Intille, S.S., 1999. Large occlusion stereo. Int. J. Comput. Vis., 33(3):181–200. [doi:10.1023/A:1008150329 890]
Criminisi, A., Blake, A., Rother, C., Shotton, J., Torr, P.H.S., 2007. Efficient dense stereo with occlusions for new view-synthesis by four-state dynamic programming. Int. J. Comput. Vis., 71(1):89–110. [doi:10.1007/s11263-006-8525-1]
Crow, F.C., 1984. Summed-Area Tables for Texture Mapping. Proc. 11th Annual Conf. on Computer Graphics and Interactive Techniques, p.207–212. [doi:10.1145/800031.808600]
Darabiha, A., Rose, J., MacLean, W.J., 2003. Video-Rate Stereo Depth Measurement on Programmable Hardware. IEEE Conf. on Computer Vision and Pattern Recognition, p.203–210. [doi:10.1109/CVPR.2003.1211355]
Gong, M., Yang, Y., 2005. Near Real-Time Reliable Stereo Matching Using Programmable Graphics Hardware. IEEE Conf. on Computer Vision and Pattern Recognition, p.924–931. [doi:10.1109/CVPR.2005.246]
Gong, M., Yang, Y., 2007. Real-time stereo matching using orthogonal reliability-based dynamic programming. IEEE Trans. Image Process., 16(3):879–884. [doi:10.1109/TIP. 2006.891344]
Gupta, R.K., Cho, S.Y., 2010. Real-Time Stereo Matching Using Adaptive Binary Window. Proc. 3DPVT, Session 2, p.1–8.
Hartley, R.I., Zisserman, A., 2004. Multiple View Geometry. Cambridge University Press, Cambridge, UK.
Hirschmüller, H., Innocent, P., Garibaldi, J., 2002. Real-time correlation-based stereo vision with reduced border errors. Int. J. Comput. Vis., 47(1–3):229–246. [doi:10.1023/A: 1014554110407]
ISO/IEC, 2008. HHI Test Material for 3D Video. M15413.
Kim, J.C., Lee, K.M., Choi, B.T., Lee, S.U., 2005. A Dense Stereo Matching Using Two-Pass Dynamic Programming with Generalized Ground Control Points. IEEE Conf. on Computer Vision and Pattern Recognition, p.1075–1082. [doi:10.1109/CVPR.2005.22]
Konolige, K., 1997. Small Vision System: Hardware and Implementation. Proc. Int. Symp. on Robotics Research, p.111–116.
Kosov, S., Thormählen, T., Seidel, H., 2009. Accurate real-time disparity estimation with variational methods. LNCS, 5875:796–807. [doi:10.1007/978-3-642-10331-5_74]
Lei, C., Selzer, J., Yang, Y., 2006. Region-Tree Based Stereo Using Dynamic Programming Optimization. IEEE Conf. on Computer Vision and Pattern Recognition, p.2378–2385. [doi:10.1109/CVPR.2006.251]
Mattoccia, S., Tombari, F., di Stefano, L., 2007. Stereo vision enabling precise border localization within a scanline optimization framework. LNCS, 4844:517–527. [doi:10. 1007/978-3-540-76390-1_51]
McDonnell, M.J., 1981. Box-filtering techniques. Comput. Graph. Image Process., 17(1):65–70. [doi:10.1016/S0146-664X(81)80009-3]
Moallemi, C.C., van Roy, B., 2010. Convergence of min-sum message-passing for convex optimization. IEEE Trans. Inform. Theory, 56(4):2041–2050. [doi:10.1109/TIT.2010. 2040863]
Mulligan, J., Isler, V., Daniilidis, K., 2002. Trinocular stereo: a real-time algorithm and its evaluation. Int. J. Comput. Vis., 47(1–3):51–61. [doi:10.1023/A:1014525320885]
Salmen, J., Schlipsing, M., Edelbrunner, J., Hegemann, S., Lueke, S., 2009. Real-time stereo vision: making more out of dynamic programming. LNCS, 5702:1096–1103. [doi:10.1007/978-3-642-03767-2_133]
Sara, R., 2010. How to Teach Stereoscopic Matching? Proc. 52nd Int. Symp. ELMAR, p.445–453.
Scharstein, D., Szeliski, R., 2001. Middlebury Stereo Vision Page. Available from http://vision.middlebury.edu/stereo/ [Accessed on Oct. 20, 2011].
Scharstein, D., Szeliski, R., 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis., 47(1):7–42. [doi:10.1023/A:1014573219 977]
Tombari, F., Mattoccia, S., di Stefano, L., Addimanda, E., 2008a. Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence. IEEE Conf. on Computer Vision and Pattern Recognition, p.1–8. [doi:10. 1109/CVPR.2008.4587677]
Tombari, F., Mattoccia, S., di Stefano, L., Addimanda, E., 2008b. Near Real-Time Stereo Based on Effective Cost Aggregation. Int. Conf. on Pattern Recognition, p.1–4. [doi:10.1109/ICPR.2008.4761024]
Veksler, O., 1999. Efficient Graph-Based Energy Minimization Methods in Computer Vision. PhD Thesis, Cornell University, New York, USA.
Veksler, O., 2003. Fast Variable Window for Stereo Correspondence Using Integral Images. IEEE Conf. on Computer Vision and Pattern Recognition, p.556–561. [doi:10. 1109/CVPR.2003.1211403]
Veksler, O., 2005. Stereo Correspondence by Dynamic Programming on a Tree. IEEE Conf. on Computer Vision and Pattern Recognition, p.384–390. [doi:10.1109/CVPR. 2005.334]
Wang, L., Liao, M., Gong, M., Yang, R., Nister, D., 2006. High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming. 3rd Int. Symp. on 3DPVT, p.798–805. [doi:10.1109/3DPVT.2006.75]
Weiss, Y., Freeman, W.T., 2001. On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs. IEEE Trans. Inform. Theory, 47(2):736–744. [doi:10.1109/18.910585]
Woodfill, J., Herzen, B.V., 1997. Real-Time Stereo Vision on the PARTS Reconfigurable Computer. Proc. IEEE Symp. on FPGAs for Custom Computing Machines, p.201–210. [doi:10.1109/FPGA.1997.624620]
Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., Nister, D., 2006. Real-Time Global Stereo Matching Using Hierarchical Belief Propagation. The British Machine Vision Conf., p.989–998.
Yang, R., Welch, G., Bishop, G., 2002. Real-Time Consensus-Based Scene Reconstruction Using Commodity Graphics Hardware. Proc. 10th Pacific Conf. on Computer Graphics and Applications, p.225–234. [doi:10.1109/PCCGA.2002.1167864]
Yoon, K., Kweon, I., 2006. Adaptive support-weight approach for correspondence search. IEEE Trans. Pattern Anal. Mach. Intell., 28(4):650–656. [doi:10.1109/TPAMI.2006.70]
Zhang, K., Lu, J., Lafruit, G., 2009. Cross-based local stereo matching using orthogonal integral images. IEEE Trans. Circ. Syst. Video Technol., 19(7):1073–1079. [doi:10. 1109/TCSVT.2009.2020478]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 60802013 and 61072081), the National Science and Technology Major Project of the Ministry of Science and Technology of China (No. 2009ZX01033-001-007), and the China Postdoctoral Science Foundation (No. 20110491804)
Rights and permissions
About this article
Cite this article
Yao, L., Li, Dx., Zhang, J. et al. Accurate real-time stereo correspondence using intra- and inter-scanline optimization. J. Zhejiang Univ. - Sci. C 13, 472–482 (2012). https://doi.org/10.1631/jzus.C1100311
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1100311