Skip to main content
SpringerLink
Log in

Mumford-Shah on the Move: Region-Based Segmentation on Deforming Manifolds with Application to 3-D Reconstruction of Shape and Appearance from Multi-View Images

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We address the problem of estimating the shape and appearance of a scene made of smooth Lambertian surfaces with piecewise smooth albedo. We allow the scene to have self-occlusions and multiple connected components. This class of surfaces is often used as an approximation of scenes populated by man-made objects. We assume we are given a number of images taken from different vantage points. Mathematically this problem can be posed as an extension of Mumford and Shah’s approach to static image segmentation to the segmentation of a function defined on a deforming surface. We propose an iterative procedure to minimize a global cost functional that combines geometric priors on both the shape of the scene and the boundary between smooth albedo regions. We carry out the numerical implementation in the level set framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bertalmio, M., Cheng, L., Osher, S.J., Sapiro, G.: Variational problems and partial differential equations on implicit surfaces. J. Comput. Phys. 174(2), 759–780 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  2. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  3. Cheng, L.-T., Burchard, P., Merriman, B., Osher, S.J.: Motion of curves constrained on surfaces using a level-set approach. J. Comput. Phys. 175(2), 604–644 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  4. Epstein, C.L., Gage, M.: The curve shortening flow. In: Wave Motion: Theory, Modelling, and Computation, Berkeley CA, 1986. Math. Sci. Res. Inst. Publ., vol. 7, pp. 15–59. Springer, New York (1987)

    Google Scholar 

  5. Faugeras, O., Keriven, R.: Variational principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Trans. Image Process. 7(3), 336–344 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  6. Greenberg, M.: Advanced Engineering Mathematics, 2 edn. Prentice Hall, New York (1998)

    Google Scholar 

  7. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)

    MATH  Google Scholar 

  8. Jin, H., Soatto, S., Yezzi, A.J.: Multi-view stereo reconstruction of dense shape and complex appearance. Int. J. Comput. Vis. 63(3), 175–189 (2005)

    Article  Google Scholar 

  9. Jin, H., Yezzi, A.J., Soatto, S.: Region-based segmentation on evolving surfaces with application to 3D reconstruction of shape and piecewise constant radiance. In: Proc. Eur. Conf. on Computer Vision, May 2004

  10. Jin, H., Yezzi, A.J., Tsai, Y.-H., Cheng, L.-T., Soatto, S.: Estimation of 3d surface shape and smooth radiance from 2D images: a level set approach. J. Sci. Comput. 19(1–3), 267–292 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  11. Kimmel, R.: Intrinsic scale space for images on surfaces: the geodesic curvature flow. Graph. Models Image Process. 59(5), 365–372 (1997)

    Article  Google Scholar 

  12. Kutulakos, K.N., Seitz, S.M.: A theory of shape by space carving. Int. J. Comput. Vis. 38(3), 199–218 (2000)

    Article  MATH  Google Scholar 

  13. Laurentini, A.: The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Machine Intell. 16(2), 150–162 (1994)

    Article  Google Scholar 

  14. Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An Invitation to 3-D Vision. Springer, Berlin (2003)

    Google Scholar 

  15. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  16. Osher, S.J., Fedkiw, R.P.: Level Set Methods and Dynamic Implicit Surfaces, 1st edn. Springer, Berlin (2002)

    Google Scholar 

  17. Osher, S.J., Sethian, J.A.: Fronts propagating with curvature dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  18. Sethian, J.A.: Level Set Methods and Fast Marching Methods: Evolving Interfaces in Computational Geometry, Fluid Mechanics, Computer Vision, and Materials Science, 2nd edn. Cambridge University Press, Cambridge (1999)

    MATH  Google Scholar 

  19. Shu, C.-W., Osher, S.J.: Efficient implementation of essentially non-oscillatory shock-capturing schemes. J. Comput. Phys. 77(2), 439–471 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  20. Soatto, S., Yezzi, A.J., Jin, H.: Tales of shape and radiance in multi-view stereo. In: Proc. of Int. Conf. on Computer Vision, October 2003, vol. 1, pp. 171–178 (2003)

  21. Vedaldi, A., Soatto, S.: Features for recognition: Viewpoint invariance for non-planar scenes. In: Proc. of Int. Conf. on Computer Vision, vol. 2, pp. 1474–1481 (2005)

  22. Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)

    Article  MATH  Google Scholar 

  23. Yezzi, A.J., Soatto, S.: Deformotion: deforming motion, shape average and the joint registration and approximation of structures in images. Int. J. Comput. Vis. 53(2), 153–167 (2003)

    Article  Google Scholar 

  24. Yezzi, A.J., Soatto, S.: Stereoscopic segmentation. Int. J. Comput. Vis. 53(1), 31–43 (2003)

    Article  MathSciNet  Google Scholar 

  25. Zhu, S.C., Yuille, A.L.: Region competition: unifying snakes, region growing, and Bayes MDL for multiband image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 18(9), 884–900 (1996)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Office of Technology, Adobe Systems Incorporated, San Jose, CA, 95110, USA

    Hailin Jin

  2. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, 30332, USA

    Anthony J. Yezzi

  3. Computer Science Department, University of California, Los Angeles, CA, 90095, USA

    Stefano Soatto

Authors
  1. Hailin Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anthony J. Yezzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefano Soatto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hailin Jin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Jin, H., Yezzi, A.J. & Soatto, S. Mumford-Shah on the Move: Region-Based Segmentation on Deforming Manifolds with Application to 3-D Reconstruction of Shape and Appearance from Multi-View Images. J Math Imaging Vis 29, 219–234 (2007). https://doi.org/10.1007/s10851-007-0030-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-007-0030-9

Keywords

Search

Navigation