Skip to main content
SpringerLink
Log in

A proximal bundle method for nonsmooth nonconvex functions with inexact information

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

For a class of nonconvex nonsmooth functions, we consider the problem of computing an approximate critical point, in the case when only inexact information about the function and subgradient values is available. We assume that the errors in function and subgradient evaluations are merely bounded, and in principle need not vanish in the limit. We examine the redistributed proximal bundle approach in this setting, and show that reasonable convergence properties are obtained. We further consider a battery of difficult nonsmooth nonconvex problems, made even more difficult by introducing inexactness in the available information. We verify that very satisfactory outcomes are obtained in our computational implementation of the inexact algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Apkarian, P., Noll, D., Prot, O.: A proximity control algorithm to minimize nonsmooth and nonconvex semi-infinite maximum eigenvalue functions. J. Convex Anal. 16(3–4), 641–666 (2009)

    MATH  MathSciNet  Google Scholar 

  2. Astorino, A., Frangioni, A., Gaudioso, M., Gorgone, E.: Piecewise-quadratic approximations in convex numerical optimization. SIAM J. Optim. 21, 1418–1438 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  3. Bagirov, A.M., Karasözen, B., Sezer, M.: Discrete gradient method: derivative-free method for nonsmooth optimization. J. Optim. Theory Appl. 137(2), 317–334 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bihain: Optimization of upper semidifferentiable functions. J. Optim. Theory Appl., 44, (1985)

  5. Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization, volume 8 of MPS/SIAM Book Series on Optimization. SIAM, New Delhi (2009)

    Book  Google Scholar 

  6. Daniilidis, A., Georgiev, P.: Approximate convexity and submonotonicity. J. Math. Anal. Appl. 291(1), 292–301 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  7. Daniilidis, A., Sagastizábal, C., Solodov, M.: Identifying structure of nonsmooth convex functions by the bundle technique. SIAM J. Optim. 20(2), 820–840 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  8. d’Antonio, G., Frangioni, A.: Convergence analysis of deflected conditional approximate subgradient methods SIAM. J. Optim. 20(1), 357–386 (2009)

    MATH  MathSciNet  Google Scholar 

  9. de Oliveira, W., Sagastizábal, C., Lemaréchal, C.: Convex proximal bundle methods in depth: a unified analysis for inexact oracles. Math. Program. 148(1–2), 241–277 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  10. Emiel, G., Sagastizábal, C.: Incremental-like bundle methods with application to energy planning. Comp. Optim. Appl. 46, 305–332 (2010)

    Article  MATH  Google Scholar 

  11. Ferrier, C.: Bornes Duales de Problémes d’Optimisation Polynomiaux, Ph.D. thesis, Laboratoire Approximation et Optimisation, Université Paul Sabatier, Toulouse, France (1997)

  12. Ferrier, C.: Computation of the distance to semi-algebraic sets. ESAIM Control Optim. Calc. Var. 5, 139–156 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  13. Fletcher, R., Leyffer, S., Ralph, D., Scholtes, S.: Local convergence of SQP methods for mathematical programs with equilibrium constraints. SIAM J. Optim. 17, 259–286 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  14. Gupal, A.M.: A method for the minimization of almost differentiable functions. Kibernetika (Kiev) 1, 114–116 (1977)

    MathSciNet  Google Scholar 

  15. Hintermüller, M.: A proximal bundle method based on approximate subgradients. Comp. Optim. Appl. 20, 245–266 (2001)

    Article  MATH  Google Scholar 

  16. Hare, W., Macklem, M.: Derivative-free optimization methods for finite minimax problems. Opt. Methods Soft. 28(2), 300–312 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  17. Hare, W., Nutini, J.: A derivative-free approximate gradient sampling algorithm for finite minimax problems. Comput. Optim. Appl. 56(1), 1–38 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  18. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20(5), 2442–2473 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  19. Hiriart-Urruty, J.-B., Lemaréchal, C.: Convex analysis and minimization algorithms. II, Volume 306 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer, Berlin (1993). Advanced theory and bundle methods

    Google Scholar 

  20. Kiwiel, K.C.: A linearization algorithm for nonsmooth minimization. Math. Oper. Res. 10(2), 185–194 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  21. Kiwiel, K.C.: Methods of Descent for Nondifferentiable Optimization. Springer, Berlin (1985)

    MATH  Google Scholar 

  22. Kiwiel, K.C.: Approximations in proximal bundle methods and decomposition of convex programs. J. Optim. Theory Appl. 84, 529–548 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  23. Kiwiel, K.C.: Restricted step and Levenberg-Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6(1), 227–249 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  24. Kiwiel, K.C.: Convergence of approximate and incremental subgradient methods for convex optimization SIAM. J. Optim. 14(3), 807–840 (2004)

    MATH  MathSciNet  Google Scholar 

  25. Kiwiel, K.C.: A proximal bundle method with approximate subgradient linearizations. SIAM J. Optim. 16(4), 1007–1023 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  26. Kiwiel, K.C.: A nonderivative version of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 20(4), 1983–1994 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  27. Lemaréchal, C.: An extension of Davidon methods to non differentiable problems. Math. Progr. Study 3, 95–109 (1975). Nondifferentiable optimization

    Article  Google Scholar 

  28. Lemaréchal, C.: Bundle Methods in Nonsmooth Optimization. In: Nonsmooth Optimization (Proceedings of IIASA Workshop, Laxenburg, 1977), volume 3 of IIASA Proceeding Series, pp. 79–102. Pergamon, Oxford (1978)

  29. Lemaréchal, C.: Lagrangian relaxation. In: Computational Combinatorial Optimization (Schloß Dagstuhl, 2000), volume 2241 of Lecture Notes in Computer Science, pp. 112–156. Springer, Berlin (2001)

  30. Lemaréchal, C., Strodiot, J.-J., Bihain, A.: On a bundle algorithm for nonsmooth optimization. In: Nonlinear Programming, 4 (Madison, Wis., 1980), pp. 245–282. Academic Press, New York (1981)

  31. Lukšan, L., Vlček, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83(3, Ser. A), 373–391 (1998)

    MATH  Google Scholar 

  32. Lukšan, L., Vlček, J.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 151(3), 425–454 (2011)

    Article  MathSciNet  Google Scholar 

  33. Luz, C.J., Schrijver, A.: A convex quadratic characterization of the Lovász theta number. SIAM J. Discret. Math. 19(2), 382–387 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  34. Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15(6), 959–972 (1977)

    Article  MATH  MathSciNet  Google Scholar 

  35. Mifflin, R.: Convergence of a modification of Lemaréchal’s algorithm for nonsmooth optimization. In: Progress in Nondifferentiable Optimization, volume 8 of IIASA Collaborative Proceeding Series CP-82, pp. 85–95. Int. Inst. Appl. Systems Anal., Laxenburg (1982)

  36. Mifflin, R.: A modification and extension of Lemarechal’s algorithm for nonsmooth minimization. Math. Program. Study 17, 77–90 (1982). Nondifferential and variational techniques in optimization (Lexington, Ky., 1980)

    Article  MATH  MathSciNet  Google Scholar 

  37. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization. Analysis and algorithms with applications to optimal control. World Scientific Publishing Co., Inc, River Edge (1992)

    Book  MATH  Google Scholar 

  38. Nedić, A., Bertsekas, D.P.: The effect of deterministic noise in subgradient methods. Math. Program. 125, 75–99 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  39. Noll, D.: Bundle method for non-convex minimization with inexact subgradients and function values. In: Computational and Analytical Mathematics, vol. 50, pp. 555–592. Springer Proceedings in Mathematics (2013)

  40. Pinar, M., Teboulle, M.: On semidefinite bounds for maximization of a non-convex quadratic objective over the L1-unit ball. RAIRO Oper. Res. 40(3), 253–265 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  41. Polyak, B.T.: Introduction to Optimization. Optimization Software, Inc., Publications Division, New York (1987)

    Google Scholar 

  42. Robert, C.P., Casella, G.: Monte Carlo Statistical Methods. Springer Texts in Statistics, 2nd edn. Springer, New York (2004)

    Book  Google Scholar 

  43. Rockafellar, R.T., Wets, J.J.-B.: Variational Analysis, Volume 317 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer, Berlin (1998)

    Google Scholar 

  44. Sagastizábal, C.: Divide to conquer: decomposition methods for energy optimization. Math. Program. B 134(1), 187–222 (2012)

    Article  MATH  Google Scholar 

  45. Sagastizábal, C., Solodov, M.: An infeasible bundle method for nonsmooth convex constrained optimization without a penalty function or a filter. SIAM J. Optim. 16, 146–169 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  46. Solodov, M.V.: On approximations with finite precision in bundle methods for nonsmooth optimization. J. Optim. Theory Appl. 119, 151–165 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  47. Solodov, M.V.: Constraint qualifications. In: Cochran, James J., et al. (eds.) Wiley Encyclopedia of Operations Research and Management Science. Wiley, New York (2010)

    Google Scholar 

  48. Solodov, M.V., Zavriev, S.K.: Error stabilty properties of generalized gradient-type algorithms. J. Optim. Theory Appl. 98, 663–680 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  49. Spingarn, J.E.: Submonotone subdifferentials of Lipschitz functions. Trans. Am. Math. Soc. 264, 77–89 (1981)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors thank the referees for many useful and insightful comments. In fact, this version looks very much different from (and is much better than) the original, thanks to the input received. Research of the first author is supported in part by NSERC DG program and UBC IRF. The second author is supported by CNPq 303840/2011-0, AFOSR FA9550-08-1-0370, NSF DMS 0707205, PRONEX-Optimization, and FAPERJ. The third author is supported in part by CNPq Grant 302637/2011-7, PRONEX-Optimization, and by FAPERJ.

Author information

Authors and Affiliations

  1. University of British Columbia, Okanagan Campus, 3333 University Way, Kelowna, BC, V1Y 8C5, Canada

    W. Hare

  2. IMPA – Instituto de Matemática Pura e Aplicada, Estrada Dona Castorina 110, Jardim Botânico, Rio de Janeiro, RJ, 22460-320, Brazil

    C. Sagastizábal & M. Solodov

Authors
  1. W. Hare
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. C. Sagastizábal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Solodov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Solodov.

Additional information

C. Sagastizábal—Visiting Researcher.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hare, W., Sagastizábal, C. & Solodov, M. A proximal bundle method for nonsmooth nonconvex functions with inexact information. Comput Optim Appl 63, 1–28 (2016). https://doi.org/10.1007/s10589-015-9762-4

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-015-9762-4

Keywords

Mathematics Subject Classification

Search

Navigation