Abstract
We consider minimization of nonsmooth functions which can be represented as the composition of a positively homogeneous convex function and a smooth mapping. This is a sufficiently rich class that includes max-functions, largest eigenvalue functions, and norm-1 regularized functions. The bundle method uses an oracle that is able to compute separately the function and subgradient information for the convex function, and the function and derivatives for the smooth mapping. With this information, it is possible to solve approximately certain proximal linearized subproblems in which the smooth mapping is replaced by its Taylor-series linearization around the current serious step. Our numerical results show the good performance of the Composite Bundle method for a large class of problems.
Similar content being viewed by others
References
Bonnans, J., Gilbert, J., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization. Theoretical and Practical Aspects. Universitext, 2nd edn, pp xiv+423. Springer, Berlin (2006)
Bonnans, J.F., Shapiro, A.: Perturbation Analysis of Optimization Problems. Springer Series in Operations Research, Springer, New York (2000)
Burke, J., Lewis, A., Overton, M.: Two numerical methods for optimizing matrix stability. Linear Algebra Appl. 351–352, 117–145 (2002)
Burke, J.V., Ferris, M.C.: A Gauss-Newton method for convex composite optimization. Math. Program. 71, 179–194 (1995). doi:10.1007/BF01585997
Correa, R., Lemaréchal, C.: Convergence of some algorithms for convex minimization. Math. Program. 62(2), 261–275 (1993)
Daniilidis, A., Sagastizábal, C., Solodov, M.: Identifying structure of nonsmooth convex functions by the bundle technique. SIAM J. Optim. 20(2), 820–840 (2009). doi:10.1137/080729864. http://link.aip.org/link/?SJE/20/820/1
Emiel, G., Sagastizábal, C.: Incremental-like bundle methods with application to energy planning. Comput. Optim. Appl. 46, 305–332 (2010). doi:10.1007/s10589-009-9288-8
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987)
Haarala, M., Miettinen, K., Makela, M.M.: New limited memory bundle method for large-scale nonsmooth optimization. Optim. Methods Softw. (6), 673–692 (2004). http://www.informaworld.com/10.1080/10556780410001689225
Hare, W.: Nonsmoth Optimization with Smooth Substructure. Ph.D. thesis, Department of Mathematics, Simon Fraser University (2003)
Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010). doi:10.1137/090754595
Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms. No. 305–306 in Grund. der math. Wiss, vol. 2. Springer, Berlin (1993)
Hiriart-Urruty, J.B., Lemaréchal, C.: Fundamentals of Convex Analysis. Grundlehren Text Editions, Springer, Berlin (2001)
Karmitsa, N., Bagirov, A., Makela, M.M.: Comparing different nonsmooth minimization methods and software. Optim. Methods Softw. (2010). http://www.informaworld.com/10.1080/10556788.2010.526116
Kiwiel, K.: A method for solving certain quadratic programming problems arising in nonsmooth optimization. IMA J. Numer. Anal. 6, 137–152 (1986)
Lemaréchal, C., Oustry, F., Sagastizábal, C.: The \({\cal U}\)-Lagrangian of a convex function. Trans. Am. Math. Soc. 352(2), 711–729 (2000)
Lemaréchal, C., Sagastizábal, C.: Practical aspects of the Moreau-Yosida regularization: theoretical preliminaries. SIAM J. Optim. 7(2), 367–385 (1997). http://link.aip.org/link/?SJE/7/367/1
Lewis, A.: Active sets, nonsmoothness and sensitivity. SIAM J. Optim. 13(3), 702–725 (2002)
Lewis, A., Wright, S.: A Proximal Method for Composite Minimization (2008). Available at http://www.optimization-online.org/DB_HTML/2008/12/2162.html
Lewis, A.S., Overton, M.L.: Nonsmooth optimization via quasi-Newton methods. Math. Program. (2011). Accepted for publication
Li, C., Wang, X.: On convergence of the gauss-newton method for convex composite optimization. Math. Program. 91, 349–356 (2002). doi:10.1007/s101070100249
Luksan, L., Vlcek, J.: Test Problems for Nonsmooth Unconstrained and Linearly Constrained Optimization. Technical Report 798, Institute of Computer Science, Academy of Sciences of the, Czech Republic (2000)
Mifflin, R.: Semi-smooth and semi-convex functions in constrained optimization. SIAM J. Control Optim. 15, 959–972 (1977)
Mifflin, R., Sagastizábal, C.: On \({\cal VU}\)-theory for functions with primal-dual gradient structure. SIAM J. Optim. 11(2), 547–571 (2000). http://siamdl.aip.org/getabs/servlet/GetabsServlet?prog=normal&id=SJOPE8000011000002000547000001&idtype=cvips&gifs=Yes
Mifflin, R., Sagastizábal, C.: Primal-dual gradient structured functions: second-order results; links to epi-derivatives and partly smooth functions. SIAM J. Optim. 13(4), 1174–1194 (2003)
Mifflin, R., Sagastizábal, C.: \({\cal VU}\)-smoothness and proximal point results for some nonconvex functions. Optim. Methods Softw. 19(5), 463–478 (2004)
Mifflin, R., Sagastizábal, C.: A \({\cal V}U\)-algorithm for convex minimization. Math. Program. Ser. A 104(2–3), 583–608 (2005). doi:10.1007/s10107-005-0630-3
Nemirovsky, A., Yudin, D.: Problem Complexity and Method Efficiency in Optimization. A Wiley-Interscience Publication (1983)
Nesterov, Y.: Gradient methods for minimizing composite objective functions. Math. Program. (2011). Accepted for publication
Noll, D., Prot, O., Rondepierre, A.: A proximity control algorithm to minimize nonsmooth and nonconvex functions. Pac. J. Optim. 4(3), 569–602 (2008)
Oustry, F.: A second-order bundle method to minimize the maximum eigenvalue function. Math. Program. 89(1, Ser. A), 1–33 (2000)
Shapiro, A.: On a class of nonsmooth composite functions. Math. Oper. Res. 28(4), 677–692 (2003)
Skajaa, A.: Limited Memory BFGS for Nonsmooth Optimization. Master’s thesis, Courant Institute of Mathematical Science (2010). http://cs.nyu.edu/overton/mstheses/skajaa/msthesis.pdf
Womersley, R.: Local properties of algorithms for minimizing nonsmooth composite functions. Math. Program. 32, 69–89 (1985)
Author information
Authors and Affiliations
Corresponding author
Additional information
Research partially supported by Grants CNPq 303840/2011-0, AFOSR FA9550-08-1-0370, and NSF DMS 0707205, as well as by PRONEX-Optimization and FAPERJ.
To C. Lemaréchal, a nonsmooth optimization giant who kindly guided the author through ridges of nondifferentiability until a superlinear path of knowledge. Merci, Claude.
On leave from INRIA, France Visiting researcher at IMPA.
Rights and permissions
About this article
Cite this article
Sagastizábal, C. Composite proximal bundle method. Math. Program. 140, 189–233 (2013). https://doi.org/10.1007/s10107-012-0600-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-012-0600-5