Skip to main content
Springer Nature Link
Log in

Convergence of proximal algorithms with stepsize controls for non-linear inverse problems and application to sparse non-negative matrix factorization

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

We consider a general ill-posed inverse problem in a Hilbert space setting by minimizing a misfit functional coupling with a multi-penalty regularization for stabilization. For solving this minimization problem, we investigate two proximal algorithms with stepsize controls: a proximal fixed point algorithm and an alternating proximal algorithm. We prove the decrease of the objective functional and the convergence of both update schemes to a stationary point under mild conditions on the stepsizes. These algorithms are then applied to the sparse and non-negative matrix factorization problems. Based on a priori information of non-negativity and sparsity of the exact solution, the problem is regularized by corresponding terms. In both cases, the implementation of our proposed algorithms is straight-forward since the evaluation of the proximal operators in these problems can be done explicitly. Finally, we test the proposed algorithms for the non-negative sparse matrix factorization problem with both simulated and real-world data and discuss reconstruction performance, convergence, as well as achieved sparsity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alexandrov, T., Becker, M., Deininger, S. O., Ernst, G., Wehder, L., Grasmair, M., von Eggeling, F., Thiele, H., Maass, P.: Spatial segmentation of imaging mass spectrometry data with edge-preserving image denoising and clustering. J. Proteome Res. 9(12), 6535–6546 (2010)

    Article  Google Scholar 

  2. Alexandrov, T., Kobarg, J. H.: Efficient spatial segmentation of large imaging mass spectrometry datasets with spatially aware clustering. Bioinformatics 27(13), i230–i238 (2011)

    Article  Google Scholar 

  3. Attouch, H., Bolte, J., Svaiter, B. F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137(1-2), 91–129 (2013)

    Article  MathSciNet  Google Scholar 

  4. Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., Plemmons, R. J.: Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Analys. 52(1), 155–173 (2007)

    Article  MathSciNet  Google Scholar 

  5. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1-2), 459–494 (2014)

    Article  MathSciNet  Google Scholar 

  6. Brunet, J., Tamayo, P., Golub, T. R., Mesirov, J. P.: Metagenes and molecular pattern discovery using matrix factorization. Proc. Natl. Acad. Sci. 101 (12), 4164–4169 (2004)

    Article  Google Scholar 

  7. Daubechies, I., Defrise, M., De Mol, C.: Sparsity-enforcing regularisation and ISTA revisited. Inverse Problems 32(10), 104001 (2016)

    Article  MathSciNet  Google Scholar 

  8. Daubechies, I., Defrise, M., Demol, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 57, 1413–1541 (2004)

    Article  MathSciNet  Google Scholar 

  9. Drakakis, K., Rickard, S., Fréin, R. D., Cichocki, A.: Analysis of financial data using non-negative matrix factorization. Int. Math. Forum 3, 1853–1870 (2008)

    MathSciNet  MATH  Google Scholar 

  10. Engl, H. W., Hanke, M., Neubauer, A.: Regularization of inverse problems. Kluwer, Dordrecht (1996)

    Book  Google Scholar 

  11. Engl, H. W., Kunisch, K., Neubauer, A.: Convergence rates for Tikhonov regularization of non-linear ill-posed problems. Inverse Problems 5(4), 523 (1989)

    Article  MathSciNet  Google Scholar 

  12. Greene, D., Cagney, G., Krogan, N., Cunningham, P.: Ensemble non-negative matrix factorization methods for clustering protein–protein interactions. Bioinformatics 24(15), 1722–1728 (2008)

    Article  Google Scholar 

  13. Guo, Z., Lin, S., Shi, L.: Distributed learning with multi-penalty regularization. Appl. Comput. Harmon. Anal. 46, 478–499 (2019)

    Article  MathSciNet  Google Scholar 

  14. Hào, D. N., Quyen, T. N. T.: Convergence rates for total variation regularization of coefficient identification problems in elliptic equations I. Inverse Problems 27, 075008 (2011)

    Article  MathSciNet  Google Scholar 

  15. Hào, D. N., Quyen, T. N. T.: Convergence rates for total variation regularization of coefficient identification problems in elliptic equations ii. J. Math. Anal. Appl. 388 (1), 593–616 (2012)

    Article  MathSciNet  Google Scholar 

  16. Hoyer, P.O.: Non-negative sparse coding. In: Proceedings of the 2002 12th IEEE workshop on neural networks for signal processing, 2002, pp. 557–565. IEEE (2002)

  17. Ito, K., Jin, B., Takeuchi, T.: Multi-parameter Tikhonov regularization. Methods and Applications of Analysis 18(1), 31–46 (2011)

    Article  MathSciNet  Google Scholar 

  18. Lee, D. D., Seung, H. S.: Learning the parts of objects by non-negative matrix factorization. Nature 401(6755), 788–791 (1999)

    Article  Google Scholar 

  19. Lorenz, D.A.: Convergence rates and source conditions for Tikhonov regularization with sparsity constraints. J. Inv Ill-posed problems 16, 463–478 (2008)

    Article  MathSciNet  Google Scholar 

  20. Lorenz, D. A., Maass, P., Muoi, P. Q.: Gradient descent methods based on quadratic approximations of Tikhonov functionals with sparsity constraints: theory and numerical comparison of stepsize rules. Electron. Trans. Numer. Anal. 39, 437–463 (2012)

    MathSciNet  MATH  Google Scholar 

  21. Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)

    MathSciNet  MATH  Google Scholar 

  22. Moreau, J.: Proximité et dualité dans un espace hilbertien. Bulletin de la Société Mathématique de France 93, 273–299 (1965)

    Article  MathSciNet  Google Scholar 

  23. Muoi, P. Q.: Reconstructing conductivity coefficients based on sparsity regularization and measured data in electrical impedance tomography. Inverse Problems in Science and Engineering 23(8), 1366–1387 (2015)

    Article  MathSciNet  Google Scholar 

  24. Muoi, P. Q., Hào, D. N., Maass, P., Pidcock, M.: Semismooth Newton and quasi-Newton methods in weighted l1-regularization. Journal of Inverse and Ill-Posed Problems 21(5), 665–693 (2013)

    Article  MathSciNet  Google Scholar 

  25. Muoi, P. Q., Hào, D. N., Maass, P., Pidcock, M.: Descent gradient methods for nonsmooth minimization problems in ill-posed problems. J. Comput. Appl. Math. 298, 105–122 (2016)

    Article  MathSciNet  Google Scholar 

  26. Muoi, P.Q., Hào, D.N., Sahoo, S.K., Tang, D., Cong, N. H., Dang, C.: Inverse problems with nonnegative and sparse solutions: Algorithms and application to the phase retrieval problem. Inverse Problems 34(5), 055007 (2018)

    Article  MathSciNet  Google Scholar 

  27. Nesterov, Y.: Gradient methods for minimizing composite objective function CORE discussion papers, 2007076, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE (2007)

  28. Sivananthan, S., et al.: Multi-penalty regularization in learning theory. J. Complex. 36, 141–165 (2016)

    Article  MathSciNet  Google Scholar 

  29. Smaragdis, P., Brown, J.C.: Non-negative matrix factorization for polyphonic music transcription. In: 2003 IEEE workshop on applications of signal processing to audio and acoustics, pp. 177–180. IEEE (2003)

  30. Wang, W., Lu, S., Mao, H., Cheng, J.: Multi-parameter Tikhonov regularization with the l0 sparsity constraint. Inverse Problems 29(6), 065018 (2013)

    Article  MathSciNet  Google Scholar 

  31. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

We gratefully thank Dr. Michael Becker (Bruker Daltonik) for doing the IMS measurement of the rat brain. Part of this work has been done during the stay of Pham Quy Muoi and Dinh Nho Hào at Vietnam Institute for Advance Study in Mathematics. The authors sincerely thank a referee for his/her constructive comments, which help us to improve the first version of this manuscript.

Funding

This research was financially supported by the German Federal Ministry of Education and Research (“KMU-innovativ: Medizintechnik” program, contract number 13GW0081), by The University of Da Nang - University of Science and Education under grant T2018-TD and by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 101.02-2017.318.

Author information

Authors and Affiliations

  1. Department of Mathematics, The University of Danang - University of Science and Education, 459 Ton Duc Thang, Danang, Vietnam

    Quy Muoi Pham

  2. Center for Industrial Mathematics, University of Bremen, Bibliothekstr. 1, 28359, Bremen, Germany

    Delf Lachmund

  3. Hanoi Institute of Mathematics, VAST, 18 Hoang Quoc Viet Road, Hanoi, Vietnam

    Dinh Nho Hào

Authors
  1. Quy Muoi Pham
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Delf Lachmund
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Dinh Nho Hào
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Quy Muoi Pham.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham, Q.M., Lachmund, D. & Hào, D.N. Convergence of proximal algorithms with stepsize controls for non-linear inverse problems and application to sparse non-negative matrix factorization. Numer Algor 85, 1255–1279 (2020). https://doi.org/10.1007/s11075-019-00864-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-019-00864-x

Keywords

Mathematics Subject Classification (2010)