Skip to main content
SpringerLink
Log in

An Accelerated Smoothing Gradient Method for Nonconvex Nonsmooth Minimization in Image Processing

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we propose a fast and provably convergent smoothing gradient descent type algorithm with extrapolation for solving a general class of nonsmooth and nonconvex inverse problems arising from image processing. Our algorithm has a localizer selective policy to switch between gradient descent scheme with or without extrapolation to possibly speed up the decreasing of the smoothed objective function and ensure the convergence. Moreover, the algorithm adaptively reduces the smoothing factor to guarantee that any accumulation point of the generated sequence is an (affine-scaled) Clarke stationary point of the original nonsmooth and nonconvex problem. Extensive numerical experiments and comparisons indicate the effectiveness of the proposed algorithm in natural image deblurring, CT and MRI reconstruction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bao, C., Dong, B., Hou, L., Shen, Z., Zhang, X., Zhang, X.: Image restoration by minimizing zero norm of wavelet frame coefficients. Inverse Probl. 32, 115004 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  3. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2, 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bian, W., Chen, X.: Smoothing SQP algorithm for non-Lipschitz optimization with complexity analysis, Preprint (2012)

  5. Bian, W., Chen, X., Ye, Y.: Complexity analysis of interior point algorithms for non-Lipschitz and nonconvex minimization. Math. Program. 149, 301–327 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Block, K.T., Uecker, M., Frahm, J.: Undersampled radial MRI with multiple coils. Iterative image reconstruction using a total variation constraint. Magn. Reson. Med. 57, 1086–1098 (2007)

    Article  Google Scholar 

  7. Burke, J.V., Ferris, M.C., Qian, M.: On the Clarke subdifferential of the distance function of a closed set. J. Math. Anal. Appl. 166, 199–213 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  8. Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134, 71–99 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chen, X., Ng, M.K., Zhang, C.: Non-Lipschitz \(\ell _{p}\)-regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21, 4709–4721 (2012)

    MathSciNet  MATH  Google Scholar 

  10. Chen, X., Niu, L., Yuan, Y.: Optimality conditions and a smoothing trust region Newton method for nonLipschitz optimization. SIAM J. Optim. 23, 1528–1552 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3, 765–790 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Chen, X., Zhou, W.: Convergence of the reweighted \(\ell _{1}\) minimization algorithm for \(\ell _{2}\)-\(\ell _{p}\) minimization. Comput. Optim. Appl. 59, 47–61 (2014)

    Article  MathSciNet  Google Scholar 

  13. Chen, Y., Hager, W., Huang, F., Phan, D., Ye, X., Yin, W.: Fast algorithms for image reconstruction with application to partially parallel MR imaging. SIAM J. Imaging Sci. 5, 90–118 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  14. Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, Hoboken (1990)

    Book  MATH  Google Scholar 

  15. Cocosco, C.A., Kollokian, V., Kwan, R.K., Pike, G.B., Evans, A.C.: Brainweb: online interface to a 3D MRI simulated brain database. In: NeuroImage, Citeseer (1997)

  16. Dong, B., Li, J., Shen, Z.: X-ray CT image reconstruction via wavelet frame based regularization and radon domain inpainting. J. Sci. Comput. 54, 333–349 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Foucart, S., Lai, M.J.: Sparsest solutions of underdetermined linear systems via \(\ell _{q}\)-minimization for \(0<q<1\). Appl. Comput. Harmon. Anal. 26, 395–407 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  18. Gao, Y., Wu, C.: On a general smoothly truncated regularization for variational piecewise constant image restoration: construction and convergent algorithms. Inverse Probl. 36, 045007 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hintermüller, M., Wu, T.: Nonconvex \(TV^{q}\)-models in image restoration: analysis and a trust-region regularization based superlinearly convergent solver. SIAM J. Imaging Sci. 6, 1385–1415 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Knoll, F., Bredies, K., Pock, T., Stollberger, R.: Second order total generalized variation (TGV) for MRI. Magn. Reson. Med. 65, 480–491 (2011)

    Article  Google Scholar 

  21. Lai, M.J., Xu, Y., Yin, W.: Improved iteratively reweighted least squares for unconstrained smoothed \(\ell _{q}\) minimization. SIAM J. Numer. Anal. 51, 927–957 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  22. Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25, 2434–2460 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  23. Li, H., Lin, Z.: Accelerated proximal gradient methods for nonconvex programming. In: NIPS, pp. 379–387 (2015)

  24. Liu, S., Cao, J., Liu, H., Zhou, X., Zhang, K., Li, Z.: MRI reconstruction via enhanced group sparsity and nonconvex regularization. Neurocomputing 272, 108–121 (2018)

    Article  Google Scholar 

  25. Lou, Y., Yan, M.: Fast \(\ell _{1}\)-\(\ell _{2}\) minimization via a proximal operator. J. Sci. Comput. 74, 767–785 (2018)

    Article  MathSciNet  Google Scholar 

  26. Lou, Y., Zeng, T., Osher, S., Xin, J.: A weighted difference of anisotropic and isotropic total variation model for image processing. SIAM J. Imaging Sci. 8, 1798–1823 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  27. Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: the application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58, 1182–1195 (2007)

    Article  Google Scholar 

  28. Nesterov, Y.: A method for solving the convex programming problem with convergence rate O(\({1}/{k^2}\)). In: Dokl. Akad. Nauk. SSSR, pp. 543–547 (1983)

  29. Nikolova, M.: Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. SIAM J. Multiscale Model. Simul. 4, 960–991 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  30. Nikolova, M., Ng, M.K., Tam, C.P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19, 3073–3088 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  31. Nikolova, M., Ng, M.K., Zhang, S., Ching, W.K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1, 2–25 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  32. Ochs, P., Chen, Y., Brox, T., Pock, T.: IPiano: inertial proximal algorithm for nonconvex optimization. SIAM J. Imaging Sci. 7, 1388–1419 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  33. Ochs, P., Dosovitskiy, A., Brox, T., Pock, T.: On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision. SIAM J. Imaging Sci. 8, 331–372 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  34. Odonoghue, B., Candes, E.: Adaptive restart for accelerated gradient schemes. Found. Comput. Math. 15, 715–732 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  35. Rasch, J., Kolehmainen, V., Nivajärvi, R., Kettunen, M., Gröhn, O., Burger, M., Brinkmann, E.-M.: Dynamic MRI reconstruction from undersampled data with an anatomical prescan. Inverse Probl. 34, 074001 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  36. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D, Nonlinear Phenomena 60, 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  37. Sidky, E.Y., Pan, X.: Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization. Phys. Med. Biol. 53, 4777–4807 (2008)

    Article  Google Scholar 

  38. Souza, J., Oliveira, P., Soubeyran, A.: Global convergence of a proximal linearized algorithm for difference of convex functions. Optim. Lett. 10, 1529–1539 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  39. Themelis, A., Patrinos, P.: Douglas–Rachford splitting and ADMM for nonconvex optimization: tight convergence results. SIAM J. Optim. 30, 149–181 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  40. Tseng, P.: Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125, 263–295 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  41. Wang, W., Wu, C., Gao, Y.: A nonconvex truncated regularization and box-constrained model for CT reconstruction. Inverse Probl. Imag. 14, 867–890 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  42. Wang, W., Wu, C., Tai, X.C.: A globally convergent algorithm for a constrained non-Lipschitz image restoration model. J. Sci. Comput. 83, 1–29 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  43. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1, 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  44. Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 29–63 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  45. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004)

    Article  Google Scholar 

  46. Wen, B., Chen, X., Pong, T.K.: Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems. SIAM J. Optim. 27, 124–145 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  47. Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69, 297–324 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  48. Wu, C., Tai, X.C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3, 300–339 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  49. Wu, Z., Li, M.: General inertial proximal gradient method for a class of nonconvex nonsmooth optimization problems. Comput. Optim. Appl. 73, 129–158 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  50. Xu, Z., Chang, X., Xu, F., Zhang, H.: \(\ell _{\frac{1}{2}}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23, 1013–1027 (2012)

    Google Scholar 

  51. Yang, L.: Proximal gradient method with extrapolation and line search for a class of nonconvex and nonsmooth problems (2017). arXiv:1711.06831

  52. Yu, P., Pong, T.K.: Iteratively reweighted \(\ell _{1}\) algorithms with extrapolation. Comput. Optim. Appl. 73, 353–386 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  53. Zeng, C., Jia, R., Wu, C.: An iterative support shrinking algorithm for non-Lipschitz optimization in image restoration. J. Math. Imaging Vis. 61, 122–139 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  54. Zeng, C., Wu, C.: On the edge recovery property of noncovex nonsmooth regularization in image restoration. SIAM J. Numer. Anal. 56, 1168–1182 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhan, R., Dong, B.: CT image reconstruction by spatial-radon domain data-driven tight frame regularization. SIAM J. Imaging Sci. 9, 1063–1083 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  56. Zhang, X., Zhang, X.: A new proximal iterative hard thresholding method with extrapolation for \(\ell _{0}\) minimization. J. Sci. Comput. 79, 809–826 (2019)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers for providing us valuable suggestions to revise this paper. This work was supported by the National Natural Science Foundation of China (NSFC) (No. 12001144), Zhejiang Provincial Natural Science Foundation of China (No. LQ20A010007) and NSF/DMS 1719932.

Author information

Authors and Affiliations

  1. Department of Mathematics, Hangzhou Dianzi University, Hangzhou, China

    Weina Wang

  2. Department of Mathematics, University of Florida, Gainesville, USA

    Yunmei Chen

Authors
  1. Weina Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunmei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yunmei Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Chen, Y. An Accelerated Smoothing Gradient Method for Nonconvex Nonsmooth Minimization in Image Processing. J Sci Comput 90, 31 (2022). https://doi.org/10.1007/s10915-021-01677-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-021-01677-8

Keywords

Mathematics Subject Classification

Search

Navigation