Skip to main content
SpringerLink
Log in

An Iterative Support Shrinking Algorithm for Non-Lipschitz Optimization in Image Restoration

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We consider a class of non-Lipschitz regularization problems that include the \(\hbox {TV}^p\) model as a special case. A lower bound theory of the non-Lipschitz regularization is obtained, which inspires us to propose an algorithm guaranteeing the non-expansiveness of the images gradient support set. After being proximally linearized, this algorithm can be easily implemented. Some standard techniques in image processing, like the fast Fourier transform, could be utilized. The global convergence is also established. Moreover, we prove that the restorations by the algorithm have edge preservation property. Numerical examples are given to show good performance of the algorithm and the rationality of the theories.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116(1), 5–16 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the kurdyka–łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  3. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized gauss-seidel methods. Math. Program. 137(1–2), 91–129 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Beck, A., Teboulle, M.: A fast iterative shrinkage–thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bian, W., Chen, X.: Linearly constrained non-lipschitz optimization for image restoration. SIAM J. Imaging Sci. 8(4), 2294–2322 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Blumensath, T., Davies, M.E.: Iterative hard thresholding for compressed sensing. Appl. Comput. Harmon. Anal. 27(3), 265–274 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bolte, J., Daniilidis, A., Lewis, A.: The łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2007)

    Article  MATH  Google Scholar 

  8. Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18(2), 556–572 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1–2), 459–494 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bondy, J., Murty, U.: Graph theory (graduate texts in mathematics). Springer, New York (2008)

    Google Scholar 

  11. Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  12. Candes, E.J., Wakin, M.B., Boyd, S.P.: Enhancing sparsity by reweighted \(\ell_1\) minimization. J. Fourier Anal. Appl. 14(5), 877–905 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chartrand, R.: Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Process. Lett. 14(10), 707–710 (2007)

    Article  Google Scholar 

  14. Chartrand, R., Staneva, V.: Restricted isometry properties and nonconvex compressive sensing. Inverse Probl. 24(3), 035020 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  15. Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2008. ICASSP 2008, pp. 3869–3872. IEEE (2008)

  16. Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129–159 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  17. Chen, X.: Smoothing methods for nonsmooth, nonconvex minimization. Math. Program. 134, 71–99 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  18. Chen, X., Ng, M.K., Zhang, C.: Non-lipschitz-regularization and box constrained model for image restoration. IEEE Trans. Image Process. 21(12), 4709–4721 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  19. Chen, X., Niu, L., Yuan, Y.: Optimality conditions and a smoothing trust region newton method for nonlipschitz optimization. SIAM J. Optim. 23(3), 1528–1552 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  20. Chen, X., Xu, F., Ye, Y.: Lower bound theory of nonzero entries in solutions of \(\ell _2-\ell _p\) minimization. SIAM J. Sci. Comput. 32(5), 2832–2852 (2010)

    Article  MathSciNet  Google Scholar 

  21. Chen, X., Zhou, W.: Smoothing nonlinear conjugate gradient method for image restoration using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 3(4), 765–790 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  22. Chen, X., Zhou, W.: Convergence of the reweighted \(\ell_1\) minimization algorithm for \(\ell_2-\ell_p\) minimization. Comput. Optim. Appl. 59(1–2), 47–61 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  23. Daubechies, I., DeVore, R., Fornasier, M., Güntürk, C.S.: Iteratively reweighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. 63(1), 1–38 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  24. Donoho, D.L.: For most large underdetermined systems of linear equations the minimal \(\ell _1\)-norm solution is also the sparsest solution. Commun. Pure Appl. Math. 59(6), 797–829 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  25. Foucart, S., Lai, M.-J.: Sparsest solutions of underdetermined linear systems via \(\ell_q\)-minimization for \(0<q<1\). Appl. Comput. Harmon. Anal. 26(3), 395–407 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Gribonval, R., Nielsen, M.: Sparse representations in unions of bases. IEEE Trans. Inf. Theory 49(12), 3320–3325 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  27. Hintermüller, M., Wu, T.: Nonconvex tv\(^q\)-models in image restoration: Analysis and a trust-region regularization-based superlinearly convergent solver. SIAM J. Imaging Sci. 6(3), 1385–1415 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  28. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Annales de l’institut Fourier 48, 769–784 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lai, M.-J., Wang, J.: An unconstrained \(\ell _q\) minimization with \(0<q \le 1\) for sparse solution of underdetermined linear systems. SIAM J. Optim. 21(1), 82–101 (2011)

    Article  MathSciNet  Google Scholar 

  30. Lai, M.-J., Xu, Y., Yin, W.: Improved iteratively reweighted least squares for unconstrained smoothed \(\ell_q\) minimization. SIAM J. Numer. Anal. 51(2), 927–957 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Lanza, A., Morigi, S., Sgallari, F.: Constrained tv\(_p-\ell _2\) model for image restoration. J. Sci. Comput. 68(1), 64–91 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  32. Li, G., Pong, T.K.: Global convergence of splitting methods for nonconvex composite optimization. SIAM J. Optim. 25(4), 2434–2460 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  33. Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Les équations aux dérivées partielles 117, 87–89 (1963)

    MATH  Google Scholar 

  34. Lu, Z.: Iterative reweighted minimization methods for \(\ell _p\) regularized unconstrained nonlinear programming. Math. Program. 147(1–2), 277–307 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  35. Natarajan, B.K.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  36. Ng, M.K., Chan, R.H., Tang, W.-C.: A fast algorithm for deblurring models with neumann boundary conditions. SIAM J. Sci. Comput. 21(3), 851–866 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  37. Nikolova, M.: Analysis of the recovery of edges in images and signals by minimizing nonconvex regularized least-squares. Multiscale Model. Simul. 4(3), 960–991 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  38. Nikolova, M., Ng, M.K., Tam, C.-P.: Fast nonconvex nonsmooth minimization methods for image restoration and reconstruction. IEEE Trans. Image Process. 19(12), 3073–3088 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  39. Nikolova, M., Ng, M.K., Zhang, S., Ching, W.-K.: Efficient reconstruction of piecewise constant images using nonsmooth nonconvex minimization. SIAM J. Imaging Sci. 1(1), 2–25 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  40. Ochs, P., Dosovitskiy, A., Brox, T., Pock, T.: On iteratively reweighted algorithms for nonsmooth nonconvex optimization in computer vision. SIAM J. Imaging Sci. 8(1), 331–372 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  41. Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis, vol. 317. Springer, Berlin (2009)

    MATH  Google Scholar 

  42. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1–4), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  43. Sidky, E.Y., Chartrand, R., Boone, J.M., Pan, X.: Constrained t\(_p\)v minimization for enhanced exploitation of gradient sparsity: application to ct image reconstruction. IEEE J. Transl. Eng. Health Med. 2(6), 1–18 (2014)

    Article  Google Scholar 

  44. Storath, M., Weinmann, A., Frikel, J., Unser, M.: Joint image reconstruction and segmentation using the potts model. Inverse Probl. 31(2), 025003 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  45. Sun, Q.: Recovery of sparsest signals via \(\ell_q\)-minimization. Appl. Comput. Harmon. Anal. 32(3), 329–341 (2012)

    Article  MathSciNet  Google Scholar 

  46. Van den Dries, L., Miller, C., et al.: Geometric categories and o-minimal structures. Duke Math. J. 84(2), 497–540 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  47. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  48. Wu, C., Tai, X.-C.: Augmented lagrangian method, dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  49. Xu, Z., Chang, X., Xu, F., Zhang, H.: \(l_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans. Neural Netw. Learn. Syst. 23(7), 1013–1027 (2012)

    Article  Google Scholar 

  50. Zeng, C., Wu, C.: On the edge recovery property of noncovex nonsmooth regularization in image restoration. SIAM J. Numer. Anal. 56(2), 1168–1182 (2018)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The work of the Chao Zeng was supported by Postdoctoral Science Foundation of China (2016M601248). The work of the Chunlin Wu was supported by National Natural Science Foundation of China (Grants 11301289 and 11531013) and Recruitment Programm of Global Young Expert.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, NanKai University, Tianjin, China

    Chao Zeng, Rui Jia & Chunlin Wu

Authors
  1. Chao Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunlin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Zeng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, C., Jia, R. & Wu, C. An Iterative Support Shrinking Algorithm for Non-Lipschitz Optimization in Image Restoration. J Math Imaging Vis 61, 122–139 (2019). https://doi.org/10.1007/s10851-018-0830-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-018-0830-0

Keywords

Search

Navigation