Skip to main content
SpringerLink
Log in

Image Segmentation with Depth Information via Simplified Variational Level Set Formulation

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Image segmentation with depth information can be modeled as a minimization problem with Nitzberg–Mumford–Shiota functional, which can be transformed into a tractable variational level set formulation. However, such formulation leads to a series of complicated high-order nonlinear partial differential equations which are difficult to solve efficiently. In this paper, we first propose an equivalently reduced variational level set formulation without using curvatures by taking level set functions as signed distance functions. Then, an alternating direction method of multipliers (ADMM) based on this simplified variational level set formulation is designed by introducing some auxiliary variables, Lagrange multipliers via using alternating optimization strategy. With the proposed ADMM method, the minimization problem for this simplified variational level set formulation is transformed into a series of sub-problems, which can be solved easily via using the Gauss–Seidel iterations, fast Fourier transform and soft thresholding formulas. The level set functions are treated as signed distance functions during computation process via implementing a simple algebraic projection method, which avoids the traditional re-initialization process for conventional variational level set methods. Extensive experiments have been conducted on both synthetic and real images, which validate the proposed approach, and show advantages of the proposed ADMM projection over algorithms based on traditional gradient descent method in terms of computational efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Nitzberg, M., Mumford, D.: The 2.1D sketch. In: Proceedings of the Third IEEE International Conference on Computer Vision, pp. 138–144 (1990)

  2. Yu, C.-C., Liu, Y.-J., Wu, M.T., Li, K.-Y., Fu, X.: A global energy optimization framework for 2.1D sketch extraction from monocular images. Graph. Models 76(5), 507–521 (2014)

    Article  Google Scholar 

  3. Amer, M.R., Yousefi, S., Raich, R., Todorovic, S.: Monocular extraction of 2.1D sketch using constrained convex optimization. Int. J. Comput. Vis. 112(1), 23 (2015)

    Article  Google Scholar 

  4. Zhu, W., Chan, T.F.: A variational model for capturing illusory contours using curvature. J. Math. Imaging Vis. 27(1), 29–40 (2007)

    Article  MathSciNet  Google Scholar 

  5. Kang, S.-H., Zhu, W., Shen, J(.J.-H).: Illusory shapes via corner fusion. SIAM J. Imaging Sci. 7(4), 1907–1936 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  6. Nitzberg, M., Mumford, D., Shiota, T.: Filtering, Segmentation, and Depth. Lecture Notes in Computer Sciences, vol. 662. Springer-Verlag, Berlin (1993)

    Book  MATH  Google Scholar 

  7. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  8. Mumford, D.: Elastica and computer vision. In: Bajaj, C.L. (ed.) Algebraic Geometry and Its Applications, pp. 491–506. Springer-Verlag, New York (1994)

    Chapter  Google Scholar 

  9. Esedoglu, S., March, R.: Segmentation with depth but without detecting junctions. J. Math. Imaging Vis. 18(1), 7–15 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  10. Ambrosio, L.A., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via \(\Gamma \)-convegence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990)

    Article  MATH  Google Scholar 

  11. Loreti, P., March, R.: Propagation of fronts in a nonlinear fourth order equation. Eur. J. Appl. Math. 2, 203–213 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  12. Zhu, W., Chan, T.F., Esedoglu, S.: Segmentation with depth: a level set approach. SIAM J. Sci. Comput. 28(5), 1957–1973 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhao, H.-K., Chan, T.F., Merriman, B., Osher, S.: A variational level set approach to multiphase motion. J. Comput. Phys. 127(1), 179–195 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  14. Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  15. Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)

    Article  MATH  Google Scholar 

  16. Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  17. Smereka, P.: Semi-implicit level set methods for curvature and surface diffusion motion. J. Sci. Comput. 19(1–3), 439–456 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhu, W., Tai, X.-C., Chan, T.F.: Image segmentation using Euler’s elastica as the regularization. J. Sci. Comput. 57(2), 414–438 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Wu, C., Tai, X.-C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imaging Sci. 7(3), 1588–1623 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of denoising and segmentation models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  22. Lie, J., Lysaker, M., Tai, X.-C.: A binary level set model and some applications to Mumford–Shah image segmentation. IEEE Trans. Image Process. 15(5), 1171–1181 (2006)

    Article  MATH  Google Scholar 

  23. Chan, T.F., Kang, S.-H., Shen, J(.J.-H).: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)

    MathSciNet  MATH  Google Scholar 

  24. Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. Image Process. 11(2), 68–76 (2002)

    Article  MathSciNet  Google Scholar 

  25. Zhu, W., Chan, T.F.: Image denoising using mean curvature of image surface. SIAM J. Imaging Sci. 5(1), 1–32 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  26. Yip, A., Zhu, W.: A fast modified Newton’s method for curvature based denoising of 1D signals. Inverse Probl. Imaging 7(3), 1075–1097 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  27. Zhu, W., Tai, X.-C., Chan, T.F.: Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Probl. Imaging 7(4), 1409–1432 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Myllykoski, M., Glowinski, R., Karkkainen, T., Rossi, T.: A new augmented Lagrangian approach for L1-mean curvature image denoising. SIAM J. Imaging Sci. 8(1), 95–125 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  29. Tai, X.-C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  30. Tai, X.-C.: Fast Numerical Schemes Related to Curvature Minimization: A Brief and Elementary Review, UCLA CAM Report14-40 (May, 2014)

  31. Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2003)

    Book  MATH  Google Scholar 

  32. Zhao, H.-K.: Fast sweeping method for eikonal equations. Math. Comput. 74, 603–627 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  33. Li, C., Xu, C., Gui, C., Fox, M. D.: Level set evolution without re-initialization: a new variational formulation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 430–436, June 20 (2005)

  34. Liu, C., Dong, F., Zhu, S., Kong, D., Liu, K.: New variational formulations for level set evolution without re-initialization with applications to image segmentation. J. Math. Imaging Vis. 41(3), 194–209 (2011)

    Article  MATH  Google Scholar 

  35. Duan, J., Pan, Z., Yin, X., Wei, W., Wang, G. (2014) Some fast projection methods based on Chan–Vese model for image segmentation. EURASIP J. Image Video Process. 10.1186/1687-5281-2014-7

  36. Yashtini, M.: Alternating Direction Method of Multiplier for Euler’s Elastica-Based Denoising, Scale Space and Variational Methods in Computer Vision, pp. 690–701. Springer, Berlin (2015)

    Google Scholar 

  37. Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  38. Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (1970)

    Article  Google Scholar 

  39. Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. arXiv preprint arXiv:1511.06324, (2015)

  40. Glowinski, R., Pan, T.W., Tai, X.C.: Some facts about operator splitting and alternating direction methods. UCLA CAM Report: 16-10 (2016)

Download references

Acknowledgements

The work has been partially supported by the National Natural Science Foundation of China with Grant numbers 61305045, 61170106, 61363066 and 61303079.

Author information

Authors and Affiliations

  1. College of Information Engineering, Qingdao University, Qingdao, China

    Lu Tan, Zhenkuan Pan, Weibo Wei & Guodong Wang

  2. School of Computer Science, University of Nottingham, Nottingham, UK

    Jinming Duan

  3. Department of Computing, Curtin University, Bentley, Perth, Australia

    Wanquan Liu

Authors
  1. Lu Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenkuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanquan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinming Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weibo Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guodong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanquan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tan, L., Pan, Z., Liu, W. et al. Image Segmentation with Depth Information via Simplified Variational Level Set Formulation. J Math Imaging Vis 60, 1–17 (2018). https://doi.org/10.1007/s10851-017-0735-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0735-3

Keywords

Search

Navigation