Skip to main content
SpringerLink
Log in

A New Nonsmooth Trust Region Algorithm for Locally Lipschitz Unconstrained Optimization Problems

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, a new nonsmooth trust region algorithm is proposed for solving unconstrained minimization problems with locally Lipschitz objective functions. At first, by using an approximation of the steepest descent direction, a local model is presented for locally Lipschitz functions. More precisely, in the quadratic model of classical trust region methods, the gradient vector is replaced by an approximation of the steepest descent direction. We then apply one of the efficient approaches of classical trust region methods in order to solve the obtained model. Using the BFGS updating formula for the Hessian approximation of the model, we show that the proposed algorithm is convergent under some mild and standard conditions on the objective function. Finally, the presented algorithm is implemented in the MATLAB environment and applied on some nonsmooth test problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Shor, N.Z.: Minimization Methods for Non-differentiable Functions. Springer, Berlin (1985)

    Book  MATH  Google Scholar 

  2. Frangioni, A.: Generalized bundle methods. SIAM J. Optim. 13, 117–156 (2003)

    Article  MathSciNet  Google Scholar 

  3. Makela, M.: Survey of bundle methods for nonsmooth optimization. Optim. Methods Softw. 17(1), 1–29 (2002)

    Article  MathSciNet  Google Scholar 

  4. Hiriart-Urruty, J.B., Lemaréchal, C.: Convex Analysis and Minimization Algorithms II: Advanced Theory and Bundle Methods. Springer, Berlin (1993)

    MATH  Google Scholar 

  5. Schramm, H., Zowe, J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis, numerical results. SIAM J. Optim. 2, 121–152 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  6. Polak, E., Royset, J.: Algorithms for finite and semi-infinite minmaxmin problems using adaptive smoothing techniques. J. Optim. Theory Appl. 119(3), 421–457 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  7. Bagirov, A.M.: Continuous subdifferential approximations and their applications. J. Math. Sci. 115, 2567–2609 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  8. Lukan, L., Vlcek, J.: A bundle-newton method for nonsmooth unconstrained minimization. Math. Program. 83(1), 373–391 (1998)

    Google Scholar 

  9. Luksan, L., Vlcek, J.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 111, 407–430 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  10. Lewis, A.S., Overton, M.L.: Nonsmooth optimization via quasi-Newton methods. Math. Program. Ser. A 141, 135–163 (2013)

    Article  MATH  MathSciNet  Google Scholar 

  11. Burke, J., Lewis, A., Overton, M.: A robust gradient sampling algorithm for nonsmooth, nonconvex optimization. SIAM J. Optim. 15, 571–779 (2005)

    Article  MathSciNet  Google Scholar 

  12. Kiwiel, K.C.: Convergence of the gradient sampling algorithm for nonsmooth nonconvex optimization. SIAM J. Optim. 18(2), 379–388 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  13. Karmitsa, N., Bagirov, A.: Limited memory discrete gradient bundle method for nonsmooth derivative-free optimization. Optimization 61(12), 1491–1509 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  14. Conn, A., Gould, N., Toint, PhL: Trust-Region Methods. Society for Industrial and Applied Mathematics, Philadelphia (2000)

    Book  MATH  Google Scholar 

  15. Kiwiel, K.: Methods of Descent for Nondifferentiable Optimization. Lecture Notes in Mathematics, vol. 1133. Springer, Berlin (1985)

  16. Fletcher, R.: A model algorithm for composite nondifferentiable optimization problem. Math. Program. Studies 17, 67–76 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  17. Fletcher, R.: An \(\ell _1\)-penalty method for nonlinear constraints. In: Boggs, P.T., Byrd, R.H., Schnabel, R.B. (eds.) Numerical Optimization, pp. 26–40. SIAM, Philadelphia (1984)

  18. Powell, M.: General algorithms for discrete nonlinear approximation calculations. In: Chui, C.K., Schumaker, L.L., Wards, J.D. (eds.) Approximation Theory IV, pp. 187–218. Academic Press, London (1983)

  19. Yuan, Y.: Global convergence of trust region algorithms for nonsmooth optimization. Techical Report DAMTP1983/NA13, Department of Applied Mathematics and Theoretical Physics, Cambridge University (1983)

  20. Zhang, J., Kim, N., Lasdon, L.: An improved successive linear programming algorithm. Manag. Sci. 31, 1312–1331 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  21. Duff, I., Nocedal, J., Reid, J.: The use of linear programming for the solution of sparse sets of nonlinear equations. SIAM J. Sci. Stat. Comput. 8, 99–108 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  22. Hallabi, M.E., Tapia, R.: A global convergence theory for arbitraty norm trust region methods for nonlinear equations. Mathematical Sciences Technical Report, TR 87–25, Rice University, Houston, TX (1987)

  23. Clarke, F.: Optim. Nonsmooth Anal. Society for Industrial and Applied Mathematics, Philadelphia (1987)

    Google Scholar 

  24. Dennis, J., Li, S., Tapia, R.: A unified approach to global convergence of trust region methods for nonsmooth optimization. Math. Program. 68, 319–346 (1995)

    MATH  MathSciNet  Google Scholar 

  25. Qi, L., Sun, J.: A trust region algorithm for minimization of locally Lipschitzian functions. Math. Program. 66, 25–43 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  26. Goldstein, A.: Optimization of Lipschitz continuous functions. Math. Program. 13, 14–22 (1977)

    Article  MATH  Google Scholar 

  27. Bertsekas, D., Mitter, S.: A descent numerical method for optimization problems with nondifferentiable cost functionals. SIAM J. Control Optim. 11, 637–652 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  28. Gaudioso, M., Monaco, M.: A bundle type approach to the unconstrained minimization of convex nonsmooth functions. Math. Program. 23(2), 216–226 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  29. Mahdavi-Amiri, N., Yousefpour, R.: An effective nonsmooth optimization algorithm for locally Lipschitz functions. J. Optim. Theory Appl. 155, 180–195 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  30. Wolfe, P.: A method of conjugate subgradients for minimizing non-differentiable functions. Math. Program. Studies 3, 145–173 (1975)

    Article  MathSciNet  Google Scholar 

  31. Nocedal, J., Wright, S.: Numerical Optimization, 2nd edn. Springer, Berlin (1999)

  32. Dolan, E., More, J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  33. Haarala, N., Miettinen, K., Mäkelä, M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109, 181–205 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  34. Luksan, L., Tuma, M., Siska, M., Vlcek, J., Ramesova, N.: UFO 2002: interactive system for universal functional optimization. Technical Report, Academy of Sciences of the Czech Republic (2002). Available online at http://www.cs.cas.cz/luksan/ufo

  35. Luksan, L., Vlcek, J.: Globally convergent variable metric method for convex nonsmooth unconstrained minimization. J. Optim. Theory Appl. 102, 593–613 (1999)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the research councils of K.N. Toosi University of Technology and Mazandaran University for supporting this work.

Author information

Authors and Affiliations

  1. Department of Mathematics, K.N. Toosi University of Technology, P.O. Box 16315-1618, Tehran, Iran

    Z. Akbari & M. Reza Peyghami

  2. Department of Mathematical Sciences, University of Mazandaran, Babolsar, Iran

    R. Yousefpour

  3. Scientific Computations in Optimization and Systems Engineering (SCOPE), K.N. Toosi University of Technology, Tehran, Iran

    M. Reza Peyghami

Authors
  1. Z. Akbari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. R. Yousefpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. M. Reza Peyghami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Reza Peyghami.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Akbari, Z., Yousefpour, R. & Reza Peyghami, M. A New Nonsmooth Trust Region Algorithm for Locally Lipschitz Unconstrained Optimization Problems. J Optim Theory Appl 164, 733–754 (2015). https://doi.org/10.1007/s10957-014-0534-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-014-0534-6

Keywords

Mathematics Subject Classification

Search

Navigation