Abstract
Optimization methods for difference-of-convex programs iteratively solve convex subproblems to define iterates. Although convex, depending on the problem’s structure, these subproblems are very often challenging and require specialized solvers. This work investigates a new methodology that defines iterates as approximate critical points of significantly easier difference-of-convex subproblems approximating the original one. Since there is considerable freedom to choose such more accessible subproblems, several algorithms can be designed from the given approach. In some cases, the resulting algorithm boils down to a straightforward process with iterates given in an analytic form. In other situations, decomposable subproblems can be chosen, opening the way for parallel computing even when the original program is not decomposable. Depending on the problem’s assumptions, a possible variant of the given approach is the Josephy–Newton method applied to the system of (necessary) optimality conditions of the original difference-of-convex program. In such a setting, local convergence with superlinear and even quadratic rates can be achieved under certain conditions.
Similar content being viewed by others
References
Toland, J.F.: Duality in nonconvex optimization. J. Math. Anal. Appl. 66(2), 399–415 (1978)
Hiriart-Urruty, J.B.: Generalized differentiability / duality and optimization for problems dealing with differences of convex functions. In: Ponstein, J. (ed.) Convexity and Duality in Optimization: Proceedings of the Symposium on Convexity and Duality in Optimization Held at the University of Groningen, The Netherlands June 22, 1984, pp. 37–70. Springer Berlin Heidelberg (1985)
Bomze, I.M., Lemaréchal, C.: Necessary conditions for local optimality in difference-of-convex programming. J. Convex Anal. 17(2), 673–680 (2010)
Bagirov, A.M., Yearwood, J.: A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems. Eur. J. Oper. Res. 170(2), 578–596 (2006)
Khalaf, W., Astorino, A., d’Alessandro, P., Gaudioso, M.: A DC optimization-based clustering technique for edge detection. Optim. Lett. 11(3), 627–640 (2017)
Astorino, A., Miglionico, G.: Optimizing sensor cover energy via DC programming. Optim. Lett. 10(2), 355–368 (2016)
Le Thi, H.A., Le, H.M., Nguyen, V.V., Pham Dinh, T.: A DC programming approach for feature selection in support vector machines learning. Adv. Data Anal. Classif. 2(3), 259–278 (2008)
Le Thi, H.A., Le, H.M., Pham Dinh, T., Van Huynh, N.: Binary classification via spherical separator by DC programming and DCA. J. Global Optim. 56(4), 1393–1407 (2013)
Gaudioso, M., Giallombardo, G., Miglionico, G., Vocaturo, E.: Classification in the multiple instance learning framework via spherical separation. Soft. Comput. 24(7), 5071–5077 (2020)
Astorino, A., Fuduli, A., Giallombardo, G., Miglionico, G.: SVM-based multiple instance classification via DC optimization. Algorithms 12(12), 249 (2019)
Rakotomamonjy, A., Flamary, R., Gasso, G.: DC proximal Newton for nonconvex optimization problems. IEEE Trans. Neural Netw. Learn. Syst. 27(3), 636–647 (2016)
Li, X., Yang, L., Ge, J., Haupt, J., Zhang, T., Zhao, T.: On quadratic convergence of DC proximal Newton algorithm in nonconvex sparse learning. Adv. Neural Inf. Process. Syst. 1, 2743–2753 (2017)
de Oliveira, W., Tcheou, M.P.: An inertial algorithm for DC programming. Set-Valued Var. Anal. 27(4), 895–919 (2019)
Tuy, H.: Convex Analysis and Global Optimization, Nonconvex Optimization and Its Applications, vol. 22. Springer, Berlin (2016)
Gaudioso, M., Giallombardo, G., Miglionico, G.: Minimizing piecewise-concave functions over polyhedra. Math. Oper. Res. 43(2), 580–597 (2018)
An, L.T.H., Tao, P.D.: The DC (difference of convex functions) programming and DCA revisited with DC models of real world nonconvex optimization problems. Ann. Oper. Res. 133(1), 23–46 (2005)
Le Thi, H.A., Tao, P.D.: DC programming and DCA: thirty years of developments. Math. Program. 169(1), 5–68 (2018)
Tao, P.D., Le Thi, H.A.: Convex analysis approach to DC programming: theory, algorithms and applications. Acta Math. Vietnam. 22(1), 289–355 (1997)
Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M.: A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes. J. Global Optim. 68(3), 501–535 (2017)
Joki, K., Bagirov, A.M., Karmitsa, N., Mäkelä, M.M., Taheri, S.: Double bundle method for finding Clarke stationary points in nonsmooth DC programming. SIAM J. Optim. 28(2), 1892–1919 (2018)
Montonen, O., Joki, K.: Bundle-based descent method for nonsmooth multiobjective DC optimization with inequality constraints. J. Global Optim. 72(3), 403–429 (2018)
Gaudioso, M., Giallombardo, G., Miglionico, G., Bagirov, A.M.: Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations. J. Global Optim. 71(1), 37–55 (2018)
de Oliveira, W.: Proximal bundle methods for nonsmooth DC programming. J. Global Optim. 75(2), 523–563 (2019)
Izmailov, A.F., Solodov, M.V.: Newton-type methods: a broader view. J. Optim. Theory Appl. 164(2), 577–620 (2015)
van Ackooij, W., de Oliveira, W.: Nonsmooth and nonconvex optimization via approximate difference-of-convex decompositions. J. Optim. Theory Appl. 182(1), 49–80 (2019)
Pang, J.S., Razaviyayn, M., Alvarado, A.: Computing B-stationary points of nonsmooth DC programs. Math. Oper. Res. 42(1), 95–118 (2017)
Aragón Artacho, F.J., Campoy, R., Vuong, P.T.: Using positive spanning sets to achieve d-stationarity with the boosted dc algorithm. Vietnam J. Math (2020). https://doi.org/10.1007/s10013-020-00400-8
Souza, J.C.O., Oliveira, P.R., Soubeyran, A.: Global convergence of a proximal linearized algorithm for difference of convex functions. Optim. Lett. 10(7), 1529–1539 (2016)
Clarke, F.: Optimisation and nonsmooth analysis. Classics in applied mathematics. Soc. Ind. Appl. Math. (1990). https://doi.org/10.1137/1.9781611971309
Artacho, F.J.A., Fleming, R.M.T., Vuong, P.T.: Accelerating the DC algorithm for smooth functions. Math. Program. 169(1), 95–118 (2018)
Hiriart-Urruty, J., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II. No. 306 in Grundlehren der mathematischen Wissenschaften, 2nd edn. Springer, Berlin (1996)
Izmailov, A.F., Solodov, M.V.: Newton-Type Methods for Optimization and Variational Problems. Springer Series in Operations Research and Financial Engineering, 1st edn. Springer, Berlin (2014)
Acknowledgements
The author is grateful to four anonymous reviewers for their remarks and constructive suggestions that considerably improved the first version of this article. The author also acknowledges financial support from the Gaspard-Monge program for Optimization and Operations Research (PGMO) project “Models for planning energy investment under uncertainty.”
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
de Oliveira, W. Sequential Difference-of-Convex Programming. J Optim Theory Appl 186, 936–959 (2020). https://doi.org/10.1007/s10957-020-01721-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-020-01721-x