Skip to main content
Springer Nature Link
Log in

Convergence Analysis of Primal–Dual Based Methods for Total Variation Minimization with Finite Element Approximation

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

We consider a minimization model with total variational regularization, which can be reformulated as a saddle-point problem and then be efficiently solved by the primal–dual method. We utilize the consistent finite element method to discretize the saddle-point reformulation; thus possible jumps of the solution can be captured over some adaptive meshes and a generic domain can be easily treated. Our emphasis is analyzing the convergence of a more general primal–dual scheme with a combination factor for the discretized model. We establish the global convergence and derive the worst-case convergence rate measured by the iteration complexity for this general primal–dual scheme. This analysis is new in the finite element context for the minimization model with total variational regularization under discussion. Furthermore, we propose a prediction–correction scheme based on the general primal–dual scheme, which can significantly relax the step size for the discretization in the time direction. Its global convergence and the worst-case convergence rate are also established. Some preliminary numerical results are reported to verify the rationale of considering the general primal–dual scheme and the primal–dual-based prediction–correction scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://dsec.pku.edu.cn/~rli/source_code/AFEPack.tar.gz.

References

  1. Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10(6), 12171229 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  2. Allen, S.M., Cahn, J.W.: A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening. Acta Metall. 27(6), 1085–1095 (1979)

    Article  Google Scholar 

  3. Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Clarendon Press, Oxford (2000)

    MATH  Google Scholar 

  4. Andreu, F., Ballester, C., Caselles, V., Mazón, J.M.: Minimizing total variation flow. Differ. Integral Equ. 14(3), 321–360 (2001)

    MathSciNet  MATH  Google Scholar 

  5. Arrow, K.J., Hurwicz, L., Uzawa, H.: Studies in Linear and Non-linear Programming. Stanford University Press, Stanford (1958)

    MATH  Google Scholar 

  6. Attouch, H., Buttazzo, G., Michaille, G.: Variational Analysis in Sobolev and BV Spaces: Applications to PDEs and Optimization. SIAM/MPS, Philadelphia (2006)

    Book  MATH  Google Scholar 

  7. Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd edn. Springer, New York (2006)

    MATH  Google Scholar 

  8. Bartels, S.: Total variation minimization with finite elements: convergence and iterative solution. SIAM J. Numer. Anal. 50(3), 1162–1180 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bartels, S.: Broken Sobolev space iteration for total variation regularized minimization problems. IMA J. Numer. Anal. 36(2), 493–502 (2016)

    Article  MathSciNet  Google Scholar 

  10. Bartels, S., Nochetto, R.H., Salgado, A.J.: Discrete total variation flows without regularization. SIAM J. Numer. Anal. 52(1), 363–385 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  11. Bellettini, G., Caselles, V., Novaga, M.: The total variation flow in \(\mathbb{R}^n\). J. Differ. Equ. 184(2), 475–525 (2002)

    Article  MATH  Google Scholar 

  12. Blum, E., Oettli, W.: Mathematische Optimierung. Springer, Berlin (1975)

    Book  MATH  Google Scholar 

  13. Boţ, R.I., Csetnek, E.R.: On the convergence rate of a forward-backward type primal-dual splitting algorithm for convex optimization problems. Optimization 64(1), 5–23 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  14. Bramble, J.H., Pasciak, J.E., Vassilev, A.T.: Analysis of the inexact Uzawa algorithm for saddle point problems. SIAM J. Numer. Anal. 34(3), 1072–1092 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  15. Brenner, S.C., Scott, L.R.: The Mathematical Theory of Finite Element Methods, 3rd edn. Springer, New York (2008)

    Book  MATH  Google Scholar 

  16. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)

    MathSciNet  MATH  Google Scholar 

  17. Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76(2), 167–188 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chan, R.H., Chen, K.: A multilevel algorithm for simultaneously denoising and deblurring images. SIAM J. Sci. Comput. 32(2), 1043–1063 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Chan, T.F., Mulet, P.: On the convergence of the lagged diffusivity fixed point method in total variation image restoration. SIAM J. Numer. Anal. 36(2), 354–367 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chan, T.F., Tai, X.-C.: Identification of discontinuous coefficients in elliptic problems using total variation regularization. SIAM J. Sci. Comput. 25(3), 881–904 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20(6), 1964–1977 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  23. Chavent, G., Kunisch, K.: Regularization of linear least squares problems by total bounded variation. ESAIM Control Optim. Calc. Var. 2, 359–376 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  24. Chen, Z., Zou, J.: An augmented Lagrangian method for identifying discontinuous parameters in elliptic systems. SIAM J. Control Optim. 37(3), 892–910 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  25. Dobson, D., Scherzer, O.: Analysis of regularized total variation penalty methods for denoising. Inverse Probl. 12(5), 601–617 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  26. Dobson, D.C., Santosa, F.: An image-enhancement technique for electrical impedance tomography. Inverse Probl. 10(2), 317–334 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  27. Dobson, D.C., Santosa, F.: Recovery of blocky images from noisy and blurred data. SIAM J. Appl. Math. 56(4), 1181–1198 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  28. Dobson, D.C., Vogel, C.R.: Convergence of an iterative method for total variation denoising. SIAM J. Numer. Anal. 34(5), 1779–1791 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  29. Elliott, C.M., Smitheman, S.A.: Numerical analysis of the TV regularization and \({H}^{-1}\) fidelity model for decomposing an image into cartoon plus texture. IMA J. Numer. Anal. 29(3), 651–689 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  30. Elman, H.C., Golub, G.H.: Inexact and preconditioned Uzawa algorithms for saddle point problems. SIAM J. Numer. Anal. 31(6), 1645–1661 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  31. Esser, E., Zhang, X., Chan, T.F.: A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. SIAM J. Imaging Sci. 3(4), 1015–1046 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  32. Facchinei, F., Pang, J.-S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. I. Springer, New York (2003)

    MATH  Google Scholar 

  33. Feng, X.B., Prohl, A.: Analysis of total variation flow and its finite element approximations. ESAIM Math. Model. Numer. Anal. 37(3), 533–556 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  34. Feng, X., von Oehsen, M., Prohl, A.: Rate of convergence of regularization procedures and finite element approximations for the total variation flow. Numer. Math. 100(3), 441–456 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  35. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary Value Problems. North-Holland Publishing Co., Amsterdam (1983)

    MATH  Google Scholar 

  36. Glowinski, R.: Numerical Methods for Nonlinear Variational Problems. Springer, New York (1984)

    Book  MATH  Google Scholar 

  37. He, B.S., Yuan, X.M.: Convergence analysis of primal-dual algorithms for a saddle-point problem: from contraction perspective. SIAM J. Imaging Sci. 5(1), 119–149 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  38. He, B.S., Yuan, X.M.: On the \(O(1/n)\) convergence rate of the Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50(2), 700–709 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  39. He, B.S., Liu, H., Wang, Z., Yuan, X.M.: A strictly contractive Peaceman–Rachford splitting method for convex programming. SIAM J. Optim. 24(3), 1011–1040 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  40. Keung, Y.L., Zou, J.: Numerical identifications of parameters in parabolic systems. Inverse Probl. 14(1), 83–100 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  41. Li, B., Sun, W.: Linearized FE approximations to a nonlinear gradient flow. SIAM J. Numer. Anal. 52(6), 2623–2646 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  43. Nemirovsky, A.S., Yudin, D.B.: Problem Complexity and Method Efficiency in Optimization. Wiley, New York (1983)

    Google Scholar 

  44. Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125–161 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  45. Nesterov, Y.E.: A method for solving the convex programming problem with convergence rate \(O(1/k^{2})\). Dokl. Akad. Nauk SSSR 269(3), 543–547 (1983). (in Russian. Translated in Soviet Math. Dokl., 27 (1983), pp. 372–376.)

    MathSciNet  Google Scholar 

  46. Ng, M.K., Qi, L., Yang, Y.-F., Huang, Y.-M.: On semismooth Newton-methods for total variation minimization. J. Math. Imaging Vis. 27(3), 265–276 (2007)

    Article  MathSciNet  Google Scholar 

  47. Osher, S., Burger, M., Goldfarb, D., Xu, J., Yin, W.: An iterative regularization method for total variation-based image restoration. Multiscale Model. Simul. 4(2), 460–489 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  48. Pock, T., Chambolle, A.: Diagonal preconditioning for first order primal-dual algorithms in convex optimization. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1762–1769 (2011)

  49. Queck, W.: The convergence factor of preconditioned algorithms of the Arrow–Hurwicz type. SIAM J. Numer. Anal. 26(4), 1016–1030 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  50. Ring, W.: Structural properties of solutions to total variation regularization problems. ESAIM Math. Model. Numer. Anal. 34(04), 799–810 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  51. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  52. Sapiro, G., Caselles, V.: Histogram modification via differential equations. J. Differ. Equ. 135(2), 238–268 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  53. Strong, D., Chan, T.: Edge-preserving and scale-dependent properties of total variation regularization. Inverse Probl. 19(6), 165–187 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  54. Vogel, C.R., Oman, M.E.: Iterative methods for total variation denoising. SIAM J. Sci. Comput. 17(1), 227–238 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhang, J., Chen, K., Yu, B.: An iterative Lagrange multiplier method for constrained total-variation-based image denoising. SIAM J. Numer. Anal. 50(3), 983–1003 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  56. Zhang, X., Burger, M., Osher, S.: A unified primal-dual algorithm framework based on Bregman iteration. J. Sci. Comput. 46(1), 20–46 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  57. Zhu, M., Chan, T.F.: An efficient primal-dual hybrid gradient algorithm for total variation image restoration. CAM Report 08-34, UCLA, Los Angeles, CA (2008)

  58. Ziemer, W.P.: Weakly Differentiable Functions: Sobolev Spaces and Functions of Bounded Variation. Springer, New York (1989)

    Book  MATH  Google Scholar 

  59. Zulehner, W.: Analysis of iterative methods for saddle point problems: a unified approach. Math. Comput. 71(238), 479–505 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Applied Mathematics, Tianjin University, Tianjin, 300072, China

    WenYi Tian

  2. Department of Mathematics, Hong Kong Baptist University, Hong Kong, China

    WenYi Tian

  3. Department of Mathematics, The University of Hong Kong, Hong Kong, China

    Xiaoming Yuan

Authors
  1. WenYi Tian
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaoming Yuan
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Yuan.

Additional information

WenYi Tian was partially supported by National Natural Science Foundation of China (Nos. 11626250, 11701416). Xiaoming Yuan was partially supported by the General Research Fund from Hong Kong Research Grants Council: HKBU 12300515.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tian, W., Yuan, X. Convergence Analysis of Primal–Dual Based Methods for Total Variation Minimization with Finite Element Approximation. J Sci Comput 76, 243–274 (2018). https://doi.org/10.1007/s10915-017-0623-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-017-0623-4

Keywords