Skip to main content
SpringerLink
Log in

Constrained Nonconvex Nonsmooth Optimization via Proximal Bundle Method

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

In this paper, we consider a constrained nonconvex nonsmooth optimization, in which both objective and constraint functions may not be convex or smooth. With the help of the penalty function, we transform the problem into an unconstrained one and design an algorithm in proximal bundle method in which local convexification of the penalty function is utilized to deal with it. We show that, if adding a special constraint qualification, the penalty function can be an exact one, and the sequence generated by our algorithm converges to the KKT points of the problem under a moderate assumption. Finally, some illustrative examples are given to show the good performance of our algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Balinski, M.L., Wolfe, P.: Nondifferentiable Optimization. North-Holland, Amsterdam (1975)

    Book  MATH  Google Scholar 

  2. Bonnans, J.F., Gilbert, J.C., Lemaréchal, C., Sagastizábal, C.: Numerical Optimization: Theoretical and Pratical Aspects, 2nd edn. Springer, Berlin (2000)

    Google Scholar 

  3. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  4. Kiwiel, K.C.: Proximity control in bundle methods for convex nondifferentiable minimization. Math. Program. 46, 105–122 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  5. Lemaréchal, C.: An extension of davidon methods to nondifferentiable problems. Nondiffer. Optim. Math. Program. Study 3, 95–109 (1975)

    Article  Google Scholar 

  6. Kiwiel, K.C.: A linearization algorithm for nonsmooth minimization. Math. Oper. Res. 10(2), 185–194 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  7. Fuduli, A., Gaudioso, M., Giallombardo, G.: Minimizing nonconvex nonsmooth functions via cutting planes and proximity control. SIAM J. Optim. 14, 743–756 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  8. Haarala, N., Miettinen, K., Mäkelä, M.M.: Globally convergent limited memory bundle method for large-scale nonsmooth optimization. Math. Program. 109, 181–205 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Mifflin, R.: A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization. Math. Program. Study 17, 77–90 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  10. Vlček, J., Lukšan, L.: Globally convergent variable metric method for nonconvex nondifferentiable unconstrained minimization. J. Optim. Theory Appl. 111(2), 407–430 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  11. Hare, W., Sagastizábal, C.: A redistributed proximal bundle method for nonconvex optimization. SIAM J. Optim. 20, 2442–2473 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  12. Hare, W., Sagastizábal, C.: Computing proximal points of nonconvex functions. Math. Program. 116(1–2, Ser.B), 221–258 (2009)

    Google Scholar 

  13. Karmitsa, N., Mäkelä, M.M.: Adaptive limited memory bundle method for bound constrained large-scale nonsmooth optimization. Optimization 59, 945–962 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Karmitsa, N., Mäkelä, M.M.: Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis. Optim. Methods Softw. 25, 895–916 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  15. Fuduli, A., Gaudioso, M., Giallombardo, G.: A DC piecewise and model and a bundling technique in nonconvex nonsmooth minimization. Optim. Methods Softw. 19, 89–102 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  16. Lukšan, L., Vlček, J.: A bundle-Newton method for nonsmooth unconstrained minimization. Math. Program. 83, 373–391 (1998)

    MATH  Google Scholar 

  17. Schramm, H.: Zowe. J.: A version of the bundle idea for minimizing a nonsmooth function: conceptual idea, convergence analysis. SIAM J Optim. 1, 121–152 (1992)

    Article  MathSciNet  Google Scholar 

  18. Kiwiel, K.C.: Restricted step and Levenberg-Marquardt techniques in proximal bundle methods for nonconvex nondifferentiable optimization. SIAM J. Optim. 6(1), 227–249 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  19. Kiwiel, K.C.: Exact penalty functions in proximal bundle methods for constrained convex nondifferentiable minimization. Math. Program. 52, 285–302 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  20. Mäkelä, M.M., Neittaanmäki, P.: Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control. Utopia Press, Singapore (1992)

    Book  MATH  Google Scholar 

  21. Rockafellar, R.T., Wets, J.J.-B.: Variational Analysis. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  22. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton, NJ (1970)

    MATH  Google Scholar 

  23. Cheney, E.W., Goldstein, A.A.: Newton’s method for convex programming and tchebycheff approximation. Numerische Mathematik 1, 253–268 (1959)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kelley, J.E.: The cutting plane method for solving convex programs. SIAM J. Optim. 8, 703–712 (1960)

    MathSciNet  Google Scholar 

  25. Kuntsevich, A., Kappel, F.: SolvOpt—The Solver for Local Nonlinear Optimization Problems: Matlab, C and Fortran Source Codes. Institute for Mathematics, Karl-Franzens University of Graz (1997)

Download references

Acknowledgments

The authors are grateful to the anonymous referees for their helpful suggestions and comments. This work was partially supported by the Natural Science Foundation of China, Grant 11171049, 11301246.

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Dalian University of Technology, Dalian, China

    Yang Yang & Liping Pang

  2. Department of Mathematics, Syracuse University, Syracuse, NY, USA

    Xuefei Ma

  3. School of Mathematics, Liaoning Normal University, Dalian, China

    Jie Shen

Authors
  1. Yang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liping Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuefei Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liping Pang.

Appendices

Appendix A

Case 1 Convex constraint functions with \(n=3\).

\( F_{i}(x)=<a^i,x>-b_i; i=1,\ldots ,n\,\),

where \(a_j^i=1/(i+j),b_i=\sum _{j=1}^n a^i_j,c_i=-(b_i+1/(1+i))\). Since constraint functions are convex, the “augment” Slater constraint qualification here degenerates Slater constraint qualification, i.e. \(F(\tilde{x})<0\).

Case 2 Nonconvex constraint functions with \(n=2\).

$$\begin{aligned}&\mathop {F_{1}: A_{1}=\left( \begin{array}{l@{\quad }l} -1 &{} 0\\ -2 &{} -1 \end{array} \right) ;}\nolimits _{B_{1}=(-14,-18)^{\top }; C_{1}=-9.} \\&\mathop {F_{2}: A_{2}=\left( \begin{array}{l@{\quad }l} -1 &{} 0\\ -1 &{} -1 \end{array} \right) ;}\nolimits _{B_{2}=(-17,-12)^{\top }; C_{2}=-13.} \end{aligned}$$

Case 3 Nonconvex constraint functions with \(n=3\).

$$\begin{aligned}&\mathop {F_{1}: A_{1}=\left( \begin{array}{l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0\\ 0 &{} -2 &{} 0\\ 0 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{1}=(-17,-13,-19)^{\top }; C_{1}=-35.}\\&\mathop {F_{2}: A_{2}=\left( \begin{array}{l@{\quad }l@{\quad }l} 0 &{} 0 &{} 0\\ -2 &{} 0 &{} 0\\ 0 &{} 0 &{} -1 \end{array} \right) ;}\nolimits _{B_{2}=(-20,-13,-21)^{\top }; C_{2}=-39.} \\&\mathop {F_{3}: A_{3}=\left( \begin{array}{l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0\\ 0 &{} -1 &{} 0\\ -1 &{} 0 &{} 0 \end{array}\right) ;}\nolimits _{B_{3}=(-21,-13,-18)^{\top }; C_{3}=-33.} \end{aligned}$$

Case 4 Nonconvex constraint functions with \(n=4\).

$$\begin{aligned}&\mathop {F_{1}: A_{1}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} 0\\ 0 &{} -1 &{} 0 &{}0 \\ 0 &{} -1 &{} 0 &{} 0\\ 0 &{} 0 &{} 0&{} -2 \end{array}\right) ;}\nolimits _{B_{1}=(-27,-23,-21,-22)^{\top }; C_{1}=-9.} \\&\mathop {F_{2}: A_{2}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} 0\\ 0 &{} -2 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} 0&{} -1 \end{array}\right) ;}\nolimits _{B_{2}=(-28,-29,-21,-21)^{\top }; C_{2}=-3.} \end{aligned}$$
$$\begin{aligned}&\mathop {F_{3}: A_{3}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} -1 &{} -1 &{} 0\\ 0 &{} 0 &{} -2 &{} 0\\ 0 &{} 0 &{} 0&{} 0 \end{array}\right) ;}\nolimits _{B_{3}=(-27,-22,-21,-24)^{\top }; C_{3}=-5.} \\&\mathop {F_{4}: A_{4}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l} -1 &{} -1 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} 0\\ -1 &{} 0 &{} -1 &{} 0\\ 0 &{} 0 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{4}=(-22,-23,-31,-22)^{\top }; C_{4}=-3.} \end{aligned}$$

Case 5 Nonconvex constraint functions with \(n=5\).

$$\begin{aligned}&\mathop {F_{1}:A_{1}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} -1 &{} 0\\ 0 &{} 0 &{} 0 &{} 0 &{} -1\\ 0 &{} 0 &{} -1 &{} 0 &{} 0\\ 0 &{} -1 &{} 0&{} 0 &{} 0 \\ -1 &{} 0 &{} 0 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{1}=(-27,-33,-21,-32,-23)^{\top }; C_{1}=-39.} \\&\mathop {F_{2}:A_{2}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} -2 &{} 0\\ 0 &{} -1 &{} -2 &{} 0 &{} 0\\ 0 &{} 0 &{} 0 &{} -1 &{} 0\\ 0 &{} -1&{} 0&{} -1 &{} 0 \\ 0 &{} 0 &{} -2 &{} 0 &{} 0 \end{array}\right) ;}\nolimits _{B_{2}=(-29,-52,-37,-12,-26)^{\top }; C_{2}=-41.}\\&\mathop {F_{3}:A_{3}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l@{\quad }l} 0 &{} 0 &{} -1 &{} 0 &{} 0\\ 0 &{} 0 &{} -1 &{} 0 &{} 0\\ 0 &{} 0 &{} -2 &{} 0 &{} -1\\ 0 &{} -1 &{} 0&{} -1 &{} 0 \\ 0 &{} -1 &{} -1 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{3}=(-17,-14,-41,-32,-21)^{\top }; C_{3}=-35.}\\&\mathop {F_{4}: A_{4}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} 0 &{} 0\\ 0 &{} -3 &{} 0 &{} 0 &{} 0\\ 0 &{} 0 &{} -1 &{} 0 &{} 0\\ 0 &{} 0 &{} 0&{} 0 &{} 0 \\ 0 &{} 0 &{} 0 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{4}=(-17,-13,-11,-12,-19)^{\top }; C_{4}=-49.} \\&\mathop {F_{5}: A_{5}=\left( \begin{array}{l@{\quad }l@{\quad }l@{\quad }l@{\quad }l} -1 &{} 0 &{} 0 &{} 0 &{} -1\\ 0 &{} 0 &{} -2 &{} 0 &{} 0\\ 0 &{} 0 &{} -1 &{} 0 &{} 0\\ 0 &{} -2 &{} 0&{} -1 &{} 0 \\ 0 &{} -1 &{} 0 &{} 0 &{} -1 \end{array}\right) ;}\nolimits _{B_{5}=(-12,-24,-29,-41,-14)^{\top }; C_{5}=-43.} \end{aligned}$$

Appendix B

Table 4 Results for dimension 3 (\(f_2(x_{0})=4\))
Full size table
Table 5 Results for dimension 4 (\(f_2(x_{0})=6\))
Full size table
Table 6 Results for dimension 5 (\(f_2(x_{0})=8\))
Full size table
Table 7 Results for dimension 6 (\(f_2(x_{0})=10\))
Full size table
Table 8 Results for dimension 7 (\(f_2(x_{0})=12\))
Full size table
Table 9 Results for dimension 8 (\(f_2(x_{0})=14\))
Full size table
Table 10 Results for dimension 9 (\(f_2(x_{0})=16\))
Full size table
Table 11 Results for dimension 10 (\(f_2(x_{0})=18\))
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, Y., Pang, L., Ma, X. et al. Constrained Nonconvex Nonsmooth Optimization via Proximal Bundle Method. J Optim Theory Appl 163, 900–925 (2014). https://doi.org/10.1007/s10957-014-0523-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-014-0523-9

Keywords

Mathematics Subject Classification (2000)

Search

Navigation