Skip to main content
SpringerLink
Log in

A reduced proximal-point homotopy method for large-scale non-convex BQP

  • Published:
Computational Optimization and Applications Aims and scope Submit manuscript

Abstract

In this paper, a reduced proximal-point homotopy (RPP-Hom) method is presented for large-scale non-convex box constrained quadratic programming (BQP) problems. As the outer iteration, at each step, the reduced proximal-point (RPP) algorithm applies the proximal point algorithm to a reduced BQP problem. The variables of the reduced subproblem include all free variables and variables at bound with respect to which the optimality conditions are violated. The RPP subproblem is solved by, as the inner iteration, an efficient piecewise linear homotopy path following method. A special termination criterion for the RPP algorithm is given and the global convergence as well as the locally linear convergence to a Karush-Kuhn-Tucker point is proved. Furthermore, a random perturbation procedure is given to modify RPP such that it converges to a local minimizer with probability 1. An accelerated version of RPP is also presented. Numerical experiments show that the RPP-Hom method outperforms the state-of-the-art algorithms for most of the benchmark problems, especially for training non-convex support vector machine.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

All data included in this study are available upon request by contact with the corresponding author.

Notes

  1. http://www.minlp.com/nlp-and-minlp-test-problems

  2. http://yann.lecun.com/exdb/mnist/

References

  1. Hanke, M., Nagy, J.G., Vogel, C.: Quasi-Newton approach to nonnegative image restorations. Linear Algebra Appl. 316, 223–236 (2000)

    Article  MathSciNet  Google Scholar 

  2. Ciarlet,P.G.: The finite element method for elliptic problems, Classics in Applied Mathematics, 40, SIAM (2002)

  3. Glowinski, R., Oden, J.T.: Numerical methods for nonlinear variational problems. J. Appl. Mech. 52, 739 (1985)

    Article  Google Scholar 

  4. Capriz,G., Cimatti,G.: Free boundary problems in the theory of hydrodynamic lubrication: A survey, Free Boundary Problems: Theory and Applications, A. Fasano and M. Primicerio, eds (1983), pp. 613–635

  5. Cimatti, G.: On a problem of the theory of lubrication governed by a variational inequality. Appl. Math. Optim. 3, 227–242 (1976)

    Article  MathSciNet  Google Scholar 

  6. Fan, R.-E., Chang, K.-W., Hsieh, C.-J., Wang, X.-R., Lin, C.-J.: Liblinear: A LIBRARY for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)

    MATH  Google Scholar 

  7. Hungerlander, P., Kaltenbacher, B., Rendl, F.: Regularization of inverse problems via box constrained minimization, arXiv preprint arXiv:1807.11316 (2018)

  8. Conn, A.R., Gould, N.I., Toint, P.: A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J. Numer. Anal. 28, 545–572 (1991)

    Article  MathSciNet  Google Scholar 

  9. Rosen, J.B.: The gradient projection method for nonlinear programming. Part I: linear constraints. J Soc. Ind. Appl. Math. 8, 181–217 (1960)

    Article  Google Scholar 

  10. Nesterov, Y.: Smooth minimization of non-smooth functions. Math. Program. 103, 127–152 (2005)

    Article  MathSciNet  Google Scholar 

  11. Nesterov, Y.: Gradient methods for minimizing composite objective function, tech. report, UCL (2007)

  12. Lin, C.J., Moré, J.J.: Newton’s method for large bound-constrained optimization problems. SIAM J. Optim. 9, 1100–1127 (1999)

    Article  MathSciNet  Google Scholar 

  13. Bertsekas, D.P.: Projected newton methods for optimization problems with simple constraints. SIAM J. Control Optim. 20, 221–246 (1982)

    Article  MathSciNet  Google Scholar 

  14. Steihaug, T.: The conjugate gradient method and trust regions in large scale optimization. SIAM J. Numer. Anal. 20, 626–637 (1983)

    Article  MathSciNet  Google Scholar 

  15. Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16, 1190–1208 (1995)

    Article  MathSciNet  Google Scholar 

  16. Kim, D., Sra, S., Dhillon, I.S.: Tackling box-constrained optimization via a new projected quasi-newton approach. SIAM J. Sci. Comput. 32, 3548–3563 (2010)

    Article  MathSciNet  Google Scholar 

  17. Hager, W.W., Zhang, H.C.: A new active set algorithm for box constrained optimization. SIAM J. Optim. 17, 526–557 (2006)

    Article  MathSciNet  Google Scholar 

  18. Bongartz, I., Conn, A.R., Gould, N.I.M., Toint, P.L.: CUTE: constrained and unconstrained testing environment. ACM Trans. Math. Softw. (TOMS) 21, 123–160 (1995)

    Article  Google Scholar 

  19. Averick, B.M., Carter, R.G., Xue, G.-L., Moré, J.J.: The MINPACK-2 test problem collection, tech. report, Argonne National Lab., IL (United States) (1992)

  20. Ritter, K., Meyer, M.: A method for solving nonlinear maximum-problems depending on parameters. Naval Res. Logist. 14, 147–162 (1967)

    Article  MathSciNet  Google Scholar 

  21. Ritter, K.: On parametric linear and quadratic programming problems., Tech. Report, DTIC Document (1981)

  22. Best, M.J.: An algorithm for the solution of the parametric quadratic programming problem, CORR 82–14, Department of Combinatorics and Optimization, University of Waterloo, Canada (1982)

  23. Ferreau, H.J., Kirches, C., Potschka, A., Bock, H.G., Diehl, M.: qpOASES: a parametric active-set algorithm for quadratic programming. Math. Program. Comput. 6, 327–363 (2014)

    Article  MathSciNet  Google Scholar 

  24. Bartels, R.H., Golub, G.H.: The simplex method of linear programming using LU decomposition. Commun. of the ACM 12, 266–268 (1969)

    Article  Google Scholar 

  25. Gill, P.E., Golub, G.H., Murray, W., Saunders, M.A.: Methods for modifying matrix factorizations. Math. Comput. 28, 505–535 (1974)

    Article  MathSciNet  Google Scholar 

  26. Eldersveld, S.K., Saunders, M.A.: A block-LU update for large-scale linear programming. SIAM J. Matrix Anal. Appl. 13, 191–201 (1992)

    Article  MathSciNet  Google Scholar 

  27. Fletcher, R., Matthews, S.: Stable modification of explicit LU factors for simplex updates. Math. Program. 30, 267–284 (1984)

    Article  MathSciNet  Google Scholar 

  28. Gill, P.E., Murray, W., Saunders, M., Wright, M.: Maintaining LU factors of a general sparse matrix. Linear Algebra Appl. 88, 239–270 (1987)

    Article  MathSciNet  Google Scholar 

  29. Björck, Å.: Numerical methods for least squares problems, SIAM (1996)

  30. Ferreau, H.J., Bock, H.G., Diehl, M.: An online active set strategy to overcome the limitations of explicit MPC. Int. J. Robust Nonlinear Control 18, 816–830 (2008)

    Article  MathSciNet  Google Scholar 

  31. Nocedal, J., Wright, S.: Numerical Optimization. Springer Science & Business Media, Berlin (2006)

    MATH  Google Scholar 

  32. Luo, Z.Q., Tseng, P.: Error bounds and convergence analysis of feasible descent methods: a general approach. Annal. Oper. Res. 46, 157–178 (1993)

    Article  MathSciNet  Google Scholar 

  33. Kaplan, A., Tichatschke, R.: Proximal point methods and nonconvex optimization. J. Global Optim. 13, 389–406 (1998)

    Article  MathSciNet  Google Scholar 

  34. Wang, G.-Q. , Yu, B., Chen, Z.-X.: App-hom method for box constrained quadratic programming, arXiv preprint arXiv:1703.05001v2 (2020)

  35. Wang, G., Yu, B.: Pal-hom method for QP and an application to LP. Comput. Optim. Appl. 73, 311–352 (2019)

    Article  MathSciNet  Google Scholar 

  36. Haeser, G., deMelo, V.V.: Approximate-KKT stopping criterion when lagrange multipliers are not available, Optimization-online (2013)

  37. Vandenbussche, D., Nemhauser, G.L.: A branch-and-cut algorithm for nonconvex quadratic programs with box constraints. Math. Program. 102, 559–575 (2005)

    Article  MathSciNet  Google Scholar 

  38. Huyer, W., Neumaier, A.: MINQ8: general definite and bound constrained indefinite quadratic programming. Comput. Optim. Appl. 69, 351–381 (2018)

    Article  MathSciNet  Google Scholar 

  39. Sra, S., Nowozin, S., Wright, S.J.: Optimization for Machine Learning. Mit Press, Cambridge (2012)

    Google Scholar 

  40. Shalev-Shwartz, S., Singer, Y., Srebro, N., Cotter, A.: Pegasos: Primal estimated sub-gradient solver for SVM. Math. Program. 127, 3–30 (2011)

    Article  MathSciNet  Google Scholar 

  41. Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2, 27 (2011)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (11971092, 11571061) and the China Postdoctoral Science Foundation (2019M660444).

Author information

Authors and Affiliations

  1. School of Mathematical Sciences, Dalian University of Technology, Dalian, 116024, Liaoning, People’s Republic of China

    Xiubo Liang & Bo Yu

  2. Business Research and Development department JD.com, Beijing, China

    Guoqiang Wang

Authors
  1. Xiubo Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guoqiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bo Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, X., Wang, G. & Yu, B. A reduced proximal-point homotopy method for large-scale non-convex BQP. Comput Optim Appl 81, 539–567 (2022). https://doi.org/10.1007/s10589-021-00330-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10589-021-00330-2

Keywords

Search

Navigation