Skip to main content
SpringerLink
Log in

Linear convergence analysis of the use of gradient projection methods on total variation problems

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

Optimization problems using total variation frequently appear in image analysis models, in which the sharp edges of images are preserved. Direct gradient descent methods usually yield very slow convergence when used for such optimization problems. Recently, many duality-based gradient projection methods have been proposed to accelerate the speed of convergence. In this dual formulation, the cost function of the optimization problem is singular, and the constraint set is not a polyhedral set. In this paper, we establish two inequalities related to projected gradients and show that, under some non-degeneracy conditions, the rate of convergence is linear.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aujol, J.F.: Some first-order algorithms for total variation based image restoration. J. Math. Imaging Vis. 34, 307–327 (2009)

    Article  MathSciNet  Google Scholar 

  2. Barzilai, J., Borwein, J.M.: Two-point step size gradient methods. IMA J. Numer. Anal. 8, 141–148 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertsekas, D.P.: On the Goldstein-Levitin-Polyak gradient projection method. IEEE Trans. Autom. Control AC-21(2), 174–184 (1976)

    Article  MathSciNet  Google Scholar 

  4. Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)

    MATH  Google Scholar 

  5. Birgin, E.G., Mart’inez, J.M., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196–1211 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  6. Bobson, D.C., Vogel, C.R.: Convergence of an iterative method for total variation denoising. SIAM J. Numer. Anal. 34(5), 1779–1791 (1997)

    Article  MathSciNet  Google Scholar 

  7. Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28(2), 151–167 (2007)

    Article  MathSciNet  Google Scholar 

  8. Calamai, P.H., More, J.J.: Projected gradient methods for linear constrained problems. Math. Program. 39, 93–116 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)

    MathSciNet  Google Scholar 

  10. Chambolle, A.: Total variation minimization and a class of binary MRF models. In: EMMCVPR, vol. 3757(1–2), pp. 136–152 (2005)

    Google Scholar 

  11. Chan, T.F., Mulet, P.: On the convergence of the lagged diffusivity fixed point method in total variation image restoration. SIAM J. Numer. Anal. 36(2), 354–367 (1999)

    Article  MathSciNet  Google Scholar 

  12. Chan, T.F., Vese, L.A.: An active contour model without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  13. Chan, T.F., Wong, C.K.: Total variation blind deconvolution. IEEE Trans. Image Process. 7(3), 370–375 (1998)

    Article  Google Scholar 

  14. Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20(6), 1964–1977 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chan, T.F., Esedoḡlu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006) (electronic)

    Article  MathSciNet  MATH  Google Scholar 

  16. Dai, Y.H., Liao, L.Z.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1–10 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  17. Dai, Y.H., Hager, W.W., Schittkowski, K., Zhang, H.: The cyclic Barzilai-Borwein method for unconstrained optimization. IMA J. Numer. Anal. 26, 604–627 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  18. Dunn, J.C.: On the convergence of projected gradient processes to singular critical points. J. Optim. Theory Appl. 55, 203–216 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  19. Evans, L.C.: Measure Theory and Fine Properties of Functions. CRC Press, Boca Raton (1992)

    MATH  Google Scholar 

  20. Figueiredo, M.A.T., Nowak, R.D., Wright, S.J.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1, 586–597 (2007)

    Article  Google Scholar 

  21. Friedlander, A., Martinez, J.M., Molina, B., Raydan, M.: Gradient method with retards and generalizations. SIAM J. Numer. Anal. 36, 275–289 (1999)

    Article  MathSciNet  Google Scholar 

  22. Gafni, E.M., Bertsekas, D.P.: Convergence of a gradient projection method. Report LIDS-P-1201, Lab. for Info. and Dec. Systems, M.I.T. (1982)

  23. Goldstein, A.A.: Convex programming in Hilbert space. Bull. Am. Math. Soc. 70, 709–710 (1964)

    Article  MATH  Google Scholar 

  24. Goldstein, T., Osher, S.: The split Bregman method for l 1 regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  25. Kolmogorov, V., Boykob, Y., Rother, C.: Applications of parametric maxflow in computer vision. In: International Conference on Computer Vision, pp. 1–8 (2007)

    Google Scholar 

  26. Levitin, E.S., Poljak, B.T.: Constrained minimization methods. U.S.S.R. Comput. Math. Math. Phys. 6, 1–50 (1965)

    Article  Google Scholar 

  27. Luo, Z.Q., Tseng, P.: On the linear convergence of descent methods for convex essentially smooth minimization. SIAM J. Control Optim. 30(2), 408–425 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  28. Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic, Dordrecht (2004)

    MATH  Google Scholar 

  29. Nesterov, Y.: Gradient methods for minimizing composite objective function. Core discussion paper (2007)

  30. O’Neil, B.: Elementary Differential Geometry. Academic Press, San Diego (1966)

    Google Scholar 

  31. Rudin, L., Osher, S., Fatemi, C.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  32. Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50, 271–293 (2002)

    Article  MATH  Google Scholar 

  33. Vogel, C.R., Oman, M.E.: Iterative methods for total variation denoising. SIAM J. Sci. Comput. 17(1), 227–238 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  34. Vogel, C.R., Oman, M.E.: Fast, robust total variation-based reconstruction of noisy, blurred images. IEEE Trans. Image Process. 7(6), 813–824 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  35. Wright, S.J.: Identifiable surfaces in constrained optimization. SIAM J. Control Optim. 31(4), 1063–1079 (1992)

    Article  Google Scholar 

  36. Yu, G., Qi, L., Dai, Y.: On nonmonotone Chambolle gradient projection algorithms. J. Math. Imaging Vis. 35, 143–154 (2009)

    Article  MathSciNet  Google Scholar 

  37. Zarantonello, E.H.: Contributions to Nonlinear Functional Analysis: Proceedings. Academic Press, New York (1971)

    MATH  Google Scholar 

  38. Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total variation image restoration. UCLA computational and applied mathematics report, 08-33 (2008)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, National Taiwan University, Taipei, Taiwan

    Pengwen Chen

  2. Department of Mathematics, University of Connecticut, Storers, CT, 06268, USA

    Changfeng Gui

  3. School of Mathematics and Econometrics, Hunan University, Changsha, P.R. China

    Changfeng Gui

Authors
  1. Pengwen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changfeng Gui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pengwen Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, P., Gui, C. Linear convergence analysis of the use of gradient projection methods on total variation problems. Comput Optim Appl 54, 283–315 (2013). https://doi.org/10.1007/s10589-011-9412-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-011-9412-4

Keywords

Search

Navigation