Skip to main content
SpringerLink
Log in

Multi-label Depth Estimation for Graph Cuts Stereo Problems

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We describe here a method to compute the depth of a scene from a set of at least two images taken at known view-points. Our approach is based on an energy formulation of the 3D reconstruction problem which we minimize using a graph-cut approach that computes a local minimum whose energy is comparable (modulo a multiple constant) with the energy of the absolute minimum. As usually done, we treat the input images symmetrically, match pixels using photoconsistency, treat occlusion and visibility problems and consider a spatial regularization term which preserves discontinuities. The details of the graph construction as well as the proof of the correctness of the method are given. Moreover we introduce a multi-label refinement algorithm in order to increase the number of depth labels without significantly increasing the computational complexity. Finally we compared our algorithm with the results available in the Middlebury database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R.: Hierarchical model-based motion estimation. In: Proceedings of the Second European Conference on Computer Vision, pp. 237–252. Springer, Berlin (1992)

    Google Scholar 

  2. Black, M.J., Anandan, P.: The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Comput. Vis. Image Underst. 63(1), 75–104 (1996)

    Article  Google Scholar 

  3. Bobick, A.F., Intille, S.S.: Large occlusion stereo. Int. J. Comput. Vis. 33(3), 181–200 (1999)

    Article  Google Scholar 

  4. Boykov, Y., Veksler, O., Zabih, R.: Markov random fields with efficient approximations. In: IEEE Conf. Comp. Vis. and Pat. Recog. (CVPR’98), pp. 648–655 (1998)

  5. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  6. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: European Conference on Comput. Vis. (ECCV’04). Springer, Berlin (2004)

    Google Scholar 

  7. Collins, R.: A space-sweep approach to true multi-image matching. In: Proc. Comp. Vis. and Pat. Recog. Conf. (CVPR’96), pp. 358–363 (1996)

  8. Ford, L.R., Fulkerson, D.R.: Maximal flow through a network. Can. J. Math. 8, 399–404 (1956)

    MATH  MathSciNet  Google Scholar 

  9. Fua, P.: A parallel stereo algorithm that produces dense depth maps and preserves image features. Mach. Vis. Appl. 6(1), 35–49 (1993)

    Article  Google Scholar 

  10. Gu, Z., Su, X., Liu, Y., Zhang, Q.: Local stereo matching with adaptive support-weight, rank transform and disparity calibration. Pattern Recogn. Lett. Arch. 29(9), 1230–1235 (2008)

    Article  Google Scholar 

  11. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)

    Google Scholar 

  12. Ishikawa, H.: Exact optimization for Markov random fields with convex priors. IEEE Trans. Pattern Anal. Mach. Intell. 25(10), 1333–1336 (2003)

    Article  Google Scholar 

  13. Ishikawa, H., Geiger, D.: Occlusions, discontinuities, and epipolar lines in stereo. In: Europ. Conf. Comp. Vis. (ECCV’98), vol. 1, pp. 232–248 (1998)

  14. Kanade, T., Okutomi, M.: A stereo matching algorithm with an adaptive window: Theory and experiment. IEEE Trans. Pattern Anal. Mach. Intell. 16(9), 920–932 (1994)

    Article  Google Scholar 

  15. Kang, S.B., Szeliski, R.: Extracting view-dependent depth maps from a collection of images. Int. J. Comput. Vis. 58, 139–163 (2004)

    Article  Google Scholar 

  16. Klaus, A., Sormann, M., Karner, K.: Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In: Int. Conf. on Pat. Recogn. (ICPR ’06), pp. 15–18 (2006)

  17. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions via graph cuts. In: IEEE Int. Conf. Comp. Vis. (ICCV’01), vol. 2, pp. 508–515 (2001)

  18. Kolmogorov, V., Zabih, R.: Multi-camera scene reconstruction via graph cuts. In: Europ. Conf. Comp. Vis. (ECCV’02), pp. 82–96 (2002)

  19. Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  Google Scholar 

  20. Kolmogorov, V., Zabih, R., Gortler, S.: Generalized multi-camera scene reconstruction using graph cuts. In: Energy Minimization Methods in Comp. Vis. and Pat. Recog., pp. 501–516 (2003)

  21. Mémin, E., Pérez, P.: Hierarchical estimation and segmentation of dense motion fields. Int. J. Comput. Vis. 46(2), 129–155 (2002)

    Article  MATH  Google Scholar 

  22. Papadakis, N., Caselles, V.: Multi-label depth estimation for graph cuts stereo problems. http://www.dtic.upf.edu/~vcaselles/technicalreport2Web.pdf (2009)

  23. Roy, S., Cox, I.J.: A maximum flow formulation of the n-camera stereo correspondence problem. In: IEEE Int. Conf. Comp. Vis. (ICCV’98), pp. 492–502 (1998)

  24. Sarkis, M., Dörfler, N., Diepold, K.: Fast adaptive graph-cuts based stereo matching. In: Advanced Concepts for Intel. Vis. Sys. (ACIVS’07), pp. 818–827 (2007)

  25. Scharstein, D.: High-accuracy stereo depth maps using structured light. In: Conf. on Comp. Vis. and Pat. Recog. (CVPR’03), pp. 195–202 (2003)

  26. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1), 7–42 (2002)

    Article  MATH  Google Scholar 

  27. Sun, J., Li, Y., Kang, S.B., Shum, H.Y.: Symmetric stereo matching for occlusion handling. In: Conf. Comp. Vis. and Pat. Recogn. (CVPR’05), vol. 2, pp. 399–406 (2005)

  28. Taguchi, Y., Wilburn, B., Zitnick, L.: Stereo reconstruction with mixed pixels using adaptive over-segmentation. In: Conf. Comp. Vis. and Pat. Recog. (CVPR’08) (2008)

  29. Wang, Z.F., Zheng, Z.G.: A region based stereo matching algorithm using cooperative optimization. In: Conf. Comp. Vis. and Pat. Recog. (CVPR’08) (2008)

  30. Xu, L., Jia, J.: Stereo matching: An outlier confidence approach. In: Europ. Conf. Comp. Vis. (ECCV’06) (2008)

  31. Yáng, Q., Wang, L., Yang, R., Stewénius, H., Nistér, D.: Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Trans. Pattern Recogn. Mach. Intell. 31(3), 492–504 (2009)

    Article  Google Scholar 

  32. Zitnick, C.L., Kang, S.B.: Stereo for image-based rendering using image over-segmentation. Int. J. Comput. Vis. 75(1), 49–65 (2007)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Barcelona Media, Avenida Diagonal 177, 08017, Barcelona, Spain

    Nicolas Papadakis

  2. Universitat Pompeu Fabra, Carrer Roc Boronat 138, 08018, Barcelona, Spain

    Vicent Caselles

Authors
  1. Nicolas Papadakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vicent Caselles
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolas Papadakis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Papadakis, N., Caselles, V. Multi-label Depth Estimation for Graph Cuts Stereo Problems. J Math Imaging Vis 38, 70–82 (2010). https://doi.org/10.1007/s10851-010-0212-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-010-0212-8

Keywords

Search

Navigation