Skip to main content

Advertisement

SpringerLink
Log in

Alternating DC algorithm for partial DC programming problems

  • Published:
Journal of Global Optimization Aims and scope Submit manuscript

Abstract

DC (Difference of Convex functions) programming and DCA (DC Algorithm) play a key role in nonconvex programming framework. These tools have a rich and successful history of thirty five years of development, and the research in recent years is being increasingly explored to new trends in the development of DCA: design novel DCA variants to improve standard DCA, to deal with the scalability and with broader classes than DC programs. Following these trends, we address in this paper the two wide classes of nonconvex problems, called partial DC programs and generalized partial DC programs, and investigate an alternating approach based on DCA for them. A partial DC program in two variables \((x,y)\in \mathbb {R}^{n}\times {\mathbb {R}}^{m}\) takes the form of a standard DC program in each variable while fixing other variable. A so-named alternating DCA and its inexact/generalized versions are developed. The convergence properties of these algorithms are established: both exact and inexact alternating DCA converge to a weak critical point of the considered problem, in particular, when the Kurdyka–Łojasiewicz inequality property is satisfied, the algorithms furnish a Fréchet/Clarke critical point. The proposed algorithms are implemented on the problem of finding an intersection point of two nonconvex sets. Numerical experiments are performed on an important application that is robust principal component analysis. Numerical results show the efficiency and the superiority of the alternating DCA comparing with the standard DCA as well as a well known alternating projection algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. A multifunction \(T:X\rightrightarrows Y\) is called closed at \(x\in X\) if for any sequence \(\{x^k\}\rightarrow x\), any \(\{u^k\}\rightarrow u\) with \(u^k\in T(x^k)\), one has \(u\in T(x).\)

References

  1. Aragón Artacho, F.J., Vuong, P.T.: The boosted difference of convex functions algorithm for nonsmooth functions. SIAM J. Optim. 30(1), 980–1006 (2020)

    Article  MathSciNet  Google Scholar 

  2. Attouch, H., Bolte, J.: On the convergence of the proximal algorithm for nonsmooth functions involving analytic features. Math. Program. 116, 5–16 (2009)

    Article  MathSciNet  Google Scholar 

  3. Attouch, H., Bolte, J., Redont, P., Soubeyran, A.: Proximal alternating minimization and projection methods for nonconvex problems: an approach based on the Kurdyka–Łojasiewicz inequality. Math. Oper. Res. 35(2), 438–457 (2010)

    Article  MathSciNet  Google Scholar 

  4. Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods. Math. Program. 137, 91–129 (2013)

    Article  MathSciNet  Google Scholar 

  5. Bauschke, H.H., Borwein, J.M.: On projection algorithms for solving convex feasibility problems. SIAM Rev. 38(3), 367–426 (1996)

    Article  MathSciNet  Google Scholar 

  6. Bierstone, E., Milman, P.: Semianalytic and subanalytic sets. Public. Math. l’IHÉS 67, 5–42 (1988)

    Article  MathSciNet  Google Scholar 

  7. Bolte, J., Daniilidis, A., Lewis, A.: The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM J. Optim. 17(4), 1205–1223 (2007)

    Article  Google Scholar 

  8. Bolte, J., Daniilidis, A., Lewis, A., Shiota, M.: Clarke subgradients of stratifiable functions. SIAM J. Optim. 18(2), 556–572 (2007)

    Article  MathSciNet  Google Scholar 

  9. Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146, 459–494 (2014)

    Article  MathSciNet  Google Scholar 

  10. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3) 11, 1–37 (2011)

    MATH  Google Scholar 

  11. Chandrasekaran, V., Sanghavi, S., Parrilo, P.A., Willsky, A.S.: Rank-sparsity incoherence for matrix decomposition. SIAM J. Optim. 21(2), 572–596 (2011)

    Article  MathSciNet  Google Scholar 

  12. Chatterji, N., Bartlett, P.L.: Alternating minimization for dictionary learning with random initialization. In: I. Guyon, U.V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, R. Garnett (eds.) Advances in Neural Information Processing Systems 30, pp. 1997–2006. Curran Associates, Inc. (2017)

  13. Clarke, F.: Optimization and Nonsmooth Analysis. Canadian Mathematical Society Series of Monographs and Advanced Texts, Wiley, New York (1983)

    MATH  Google Scholar 

  14. Comon, P., Luciani, X., de Almeida, A.L.F.: Tensor decompositions, alternating least squares and other tales. J. Chemom. 23(7–8), 393–405 (2009)

    Article  Google Scholar 

  15. Cruz Neto, J.X., Lopes, J.O., Santos, P.S.M., Souza, J.C.O.: An interior proximal linearized method for dc programming based on Bregman distance or second-order homogeneous kernels. Optimization 68(7), 1305–1319 (2019)

    Article  MathSciNet  Google Scholar 

  16. Eckart, C., Young, G.: The approximation of one matrix by another of lower rank. Psychometrika 1, 211–218 (1936)

    Article  Google Scholar 

  17. Ekeland, I.: On the variational principle. J. Math. Anal. Appl. 47(2), 324–353 (1974)

    Article  MathSciNet  Google Scholar 

  18. Ho, V.T., Le Thi, H.A., Pham Dinh, T.: DCA-based algorithms for DC fitting. J. Comput. Appl. Math. 389(113353) (2021)

  19. Ioffe, A., Tihomirov, V.: Theory of Extremal Problems. North-Holland (1979)

  20. Jain, P., Netrapalli, P., Sanghavi, S.: Low-rank matrix completion using alternating minimization. In: Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC’13, pp. 665–674. Association for Computing Machinery, New York, NY, USA (2013)

  21. Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth dc optimization utilizing nonconvex cutting planes. J. Global Optim. 68(3), 501–535 (2017)

    Article  MathSciNet  Google Scholar 

  22. Kurdyka, K.: On gradients of functions definable in o-minimal structures. Annales de l’Institut Fourier 48(3), 769–783 (1998)

    Article  MathSciNet  Google Scholar 

  23. Le Thi, H.A., Ho, V.T.: Online learning based on online DCA and application to online classification. Neural Comput. 32(4), 759–793 (2020)

    Article  MathSciNet  Google Scholar 

  24. Le Thi, H.A., Huynh, V.N., Pham Dinh, T.: Convergence analysis of dc algorithm for dc programming with subanalytic data. J. Optim. Theory Appl. 179, 103–126 (2018)

    Article  MathSciNet  Google Scholar 

  25. Le Thi, H.A., Le, H.M., Phan, D.N., Tran, B.: Stochastic DCA for the large-sum of non-convex functions problem and its application to group variable selection in classification. In: Proceedings of the 34th International Conference on Machine Learning, Vol. 70, pp. 3394–3403. JMLR.org (2017)

  26. Le Thi, H.A., Le, H.M., Phan, D.N., Tran, B.: Novel DCA based algorithms for a special class of nonconvex problems with application in machine learning. Appl. Math. Comput. (2020). https://doi.org/10.1016/j.amc.2020.125904

    Article  Google Scholar 

  27. Le Thi, H.A., Pham Dinh, T.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1–4), 23–48 (2005)

    MathSciNet  MATH  Google Scholar 

  28. Le Thi, H.A., Pham Dinh, T.: DC programming and DCA: thirty years of developments. Math. Program. Special Issue DC Program. Theory Algor. Appl. 169(1), 5–68 (2018)

    MathSciNet  MATH  Google Scholar 

  29. Le Thi, H.A., Phan, D.N.: DC programming and DCA for sparse optimal scoring problem. Neurocomputing 186, 170–181 (2016)

    Article  Google Scholar 

  30. Le Thi, H.A., Vo, X.T., Pham Dinh, T.: Efficient nonnegative matrix factorization by DC programming and DCA. Neural Comput. 28(6), 1163–1216 (2016)

    Article  MathSciNet  Google Scholar 

  31. Li, Q., Zhu, Z., Tang, G.: Alternating minimizations converge to second-order optimal solutions. In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, pp. 3935–3943. PMLR, Long Beach, California, USA (2019)

  32. Liu, T., Pong, T.K., Takeda, A.: A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems. Math. Program. 176(1), 339–367 (2019)

    Article  MathSciNet  Google Scholar 

  33. Łojasiewicz, S.: Sur le problème de la division. Stud. Math. 18(1), 87–136 (1959)

    Article  Google Scholar 

  34. Łojasiewicz, S.: Une propriété topologique des sous-ensembles analytiques réels. Colloques internationaux du C.N.R.S sur les Equations aux dérivées Partielles pp. 87–89 (1963)

  35. Łojasiewicz, S.: Sur la géométrie semi- et sous-analytique. Anal de l’institute Fourier 43(5), 1575–1595 (1993)

    Article  Google Scholar 

  36. Lu, Z., Zhang, Y.: Sparse approximation via penalty decomposition methods. SIAM J. Optim. 23(4), 2448–2478 (2013)

    Article  MathSciNet  Google Scholar 

  37. Mordukhovich, B.S.: Variational analysis and generalized differentiation. I. Basic theory, Grundlehren der Mathematischen Wissenschaften, vol. 330. Springer, Berlin (2006)

  38. Netrapalli, P., U N, N., Sanghavi, S., Anandkumar, A., Jain, P.: Non-convex robust PCA. In: Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, K.Q. Weinberger (eds.) Advances in Neural Information Processing Systems 27, pp. 1107–1115. Curran Associates, Inc. (2014)

  39. de Oliveira, W.: Sequential difference-of-convex programming. J. Optim. Theory Appl. 186, 936–959 (2020)

    Article  MathSciNet  Google Scholar 

  40. Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)

    Article  MathSciNet  Google Scholar 

  41. Pham Dinh, T., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietn. 22(1), 289–355 (1997)

    MATH  Google Scholar 

  42. Phan, D.N., Le, H.M., Le Thi, H.A.: Accelerated difference of convex functions algorithm and its application to sparse binary logistic regression. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 1369–1375. International Joint Conferences on Artificial Intelligence Organization (2018)

  43. Rockafellar, R.T.: Convex Analysis. Princeton University Press (1970)

  44. Rockafellar, R.T., Wets, R.J.B.: Variational Analysis. Springer, Berlin (1998)

    Book  Google Scholar 

  45. Shen, Y., Xu, H., Liu, X.: An alternating minimization method for robust principal component analysis. Optim. Methods Softw. 34(6), 1251–1276 (2019)

    Article  MathSciNet  Google Scholar 

  46. Shiota, M.: Geometry of Subanalytic and Semialgebraic Sets, Progress in Mathematics, vol. 150. Birkhäuser, Basel (1997)

    Book  Google Scholar 

  47. Wang, Y., Yang, J., Yin, W., Zhang, Y.: A new alternating minimization algorithm for total variation image reconstruction. SIAM J. Imaging Sci. 1(3), 248–272 (2008)

    Article  MathSciNet  Google Scholar 

  48. Wen, B., Chen, X., Pong, T.K.: A proximal difference-of-convex algorithm with extrapolation. Comput. Optim. Appl. 69(2), 297–324 (2018)

    Article  MathSciNet  Google Scholar 

  49. Xu, Y., Qi, Q., Lin, Q., Jin, R., Yang, T.: Stochastic optimization for DC functions and non-smooth non-convex regularizers with non-asymptotic convergence. In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, pp. 6942–6951. PMLR (2019)

  50. Xu, Y., Yin, W.: A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion. SIAM J. Imaging Sci. 6(3), 1758–1789 (2013)

    Article  MathSciNet  Google Scholar 

  51. Yuan, X., Yang, J.: Sparse and low rank matrix decomposition via alternating direction method. Pac. J. Optim. 9(1), 167–180 (2013)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research is funded by Foundation for Science and Technology Development of Ton Duc Thang University (FOSTECT), website: http://fostect.tdtu.edu.vn, under Grant FOSTECT.2017.BR.10

Author information

Authors and Affiliations

  1. Department for Management of Science and Technology Development, Ton Duc Thang University, Ho Chi Minh City, Vietnam

    Tao Pham Dinh

  2. Faculty of Mathematics and Statistics, Ton Duc Thang University, Ho Chi Minh City, Vietnam

    Tao Pham Dinh

  3. Department of Mathematics, University of Quynhon, 170 An Duong Vuong, Qui Nhon, Vietnam

    Van Ngai Huynh

  4. Computer science and Applications Department, LGIPM, University of Lorraine, 57070, Metz, France

    Hoai An Le Thi & Vinh Thanh Ho

Authors
  1. Tao Pham Dinh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Van Ngai Huynh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hoai An Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vinh Thanh Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vinh Thanh Ho.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pham Dinh, T., Huynh, V.N., Le Thi, H.A. et al. Alternating DC algorithm for partial DC programming problems. J Glob Optim 82, 897–928 (2022). https://doi.org/10.1007/s10898-021-01043-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10898-021-01043-w

Keywords

Search

Navigation