Abstract
In this paper, we propose a limited-memory trust-region method for solving large-scale nonlinear optimization problems with many equality constraints. Within the framework of the Byrd–Omojokun algorithm, we adopt the technique proposed by Burdakov et al. (Math Program Comput 9:101–134, 2017) to solve the accompanying trust-region subproblems. To successfully deal with the difficulties arising in the case of many constraints, we reduce the number of constraints in the normal subproblem, so that the computational cost at each iteration is suitable for large-scale problems. Furthermore, we establish the global convergence of the proposed method in that case. Numerical experiments on some test problems are given to verify the soundness and effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60:223–311. https://doi.org/10.1137/16M1080173
Brust JJ, Erway JB, Marcia RF (2017) On solving L-SR1 trust-region subproblems. Comput Optim Appl 66:245–266. https://doi.org/10.1007/s10589-016-9868-3
Brust JJ, Marcia RF, Petra CG (2019) Large-scale quasi-newton trust-region methods with low-dimensional linear equality constraints. Comput Optim Appl 74:669–701. https://doi.org/10.1007/s10589-019-00127-4
Brust JJ, Burdakov O, Erway JB, Marcia RF, Yuan YX (2022) Algorithm 1030: SC-SR1: MATLAB software for limitedmemory SR1 trust-region methods. ACM Trans. Math. Softw. 48, 1–33. https://doi.org/10.1145/3550269
Brust JJ, Erway JB, Marcia RF (2022b) Shape-changing trust-region methods using multipoint symmetric secant matrices. arXiv:2209.12057
Brust JJ, Marcia RF, Petra CG, Saunders MA (2022c) Large-scale optimization with linear equality constraints using reduced compact representation. SIAM J Sci Comput. https://doi.org/10.1137/21M1393819
Burdakov O, Gong L, Zikrin S, Yuan YX (2017) On efficiently combining limited-memory and trust-region techniques. Math Program Comput 9:101–134. https://doi.org/10.1007/s12532-016-0109-7
Byrd RH, Omojokun EO (1987) Robust trust-region methods for nonlinearly constrained optimization. Paper presented at the SIAM conference on optimization, Houston, TX, USA
Byrd RH, Nocedal J, Schnabel RB (1994) Representations of quasi-newton matrices and their use in limited memory methods. Math Program 63:129–156. https://doi.org/10.1007/BF01582063
Conn AR, Gould NIM, Toint PL (2000) Trust-region methods. SIAM, Philadelphia
Costa CM, Grapiglia GN (2019) A subspace version of the Wang–Yuan augmented Lagrangian trust region method for equality constrained optimization. Appl Math Comput. https://doi.org/10.1016/j.amc.2019.124861
Courant R (1943) Variational methods for the solution of problems of equilibrium and vibrations. Bull Am Math Soc 49:1–23. https://doi.org/10.1090/S0002-9904-1943-07818-4
Dennis JE Jr, El-Alem MM, Maciel MC (1997) A global convergence theory for general trust-region-based algorithms for equality constrained optimization. SIAM J Optim 7:177–207. https://doi.org/10.1137/S1052623492238881
Dong J, Shi J, Wang S, Xue Y, Liu S (2003) A trust-region algorithm for equality-constrained optimization via a reduced dimension approach. J Comput Appl Math 152:99–118. https://doi.org/10.1016/S0377-0427(02)00699-4
El-Alem MM (1991) A global convergence theory for the Celis–Dennis–Tapia trust-region algorithm for constrained optimization. SIAM J Numer Anal 28:266–290. https://doi.org/10.1137/0728015
El-Alem MM (1995) Global convergence without the assumption of linear independence for a trust-region algorithm for constrained optimization. J Optim Theory Appl 87:563–577. https://doi.org/10.1007/BF02192134
Erway JB, Marcia RF (2015) On efficiently computing the eigenvalues of limited-memory quasi-newton matrices. SIAM J Matrix Anal Appl 36:1338–1359. https://doi.org/10.1137/140997737
Erway JB, Rezapour M (2021) A new multipoint symmetric secant method with a dense initial matrix. Optim. Methods Softw.https://doi.org/10.1080/10556788.2023.2167993
Erway JB, Griffin J, Marcia RF, Omheni R (2020) Trust-region algorithms for training responses: machine learning methods using indefinite hessian approximations. Optim Methods Softw 35:460–487. https://doi.org/10.1080/10556788.2019.1624747
Fletcher R, Leyffer S (2002) Nonlinear programming without a penalty function. Math Program 91:239–269. https://doi.org/10.1007/s101070100244
Gould NIM, Orban D, Toint PL (2005) Numerical methods for large-scale nonlinear optimization. Acta Numer 14:299–361. https://doi.org/10.1017/S0962492904000248
Gould NIM, Orban D, Toint PL (2015) CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization. Comput Optim Appl 60:545–557. https://doi.org/10.1007/s10589-014-9687-3
Grapiglia GN, Yuan J, Yuan YX (2013) A subspace version of the Powell–Yuan trust-region algorithm for equality constrained optimization. J Oper Res Soc China 1:425–451. https://doi.org/10.1017/S0962492904000248
Lalee M, Nocedal J, Plantenga T (1998) On the implementation of an algorithm for large-scale equality constrained optimization. SIAM J Optim 8:682–706. https://doi.org/10.1137/S1052623493262993
Lee JH, Jung YM, Yuan YX, Yun S (2019) A subspace SQP method for equality constrained optimization. Comput Optim Appl 74:177–194. https://doi.org/10.1007/s10589-019-00109-6
Moré JJ, Sorensen DC (1983) Computing a trust region step. SIAM J Sci Stat Comput 4:553–572. https://doi.org/10.1137/0904038
Nocedal J (1980) Updating quasi-newton matrices with limited storage. Math Comput 35:773–782. https://doi.org/10.1090/S0025-5718-1980-0572855-7
Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, New York
Omojokun EO (1991) Trust region algorithms for optimization with nonlinear equality and inequality constraints. Dissertation, University of Colorado Boulder
Powell MJD (1970) A new algorithm for unconstrained optimization. In: Rosen JB, Mangasarian OL, Ritter K (eds) Nonlinear programming. Academic Press, New York, pp 31–65. https://doi.org/10.1016/B978-0-12-597050-1.50006-3
Powell MJD (1978) A fast algorithm for nonlinearly constrained optimization calculations. In: Watson GA (ed) Numerical analysis. Springer, Berlin, pp 144–157. https://doi.org/10.1007/BFb0067703
Powell MJD, Yuan YX (1990) A trust region algorithm for equality constrained optimization. Math Program 49:189–211. https://doi.org/10.1007/BF01588787
Steihaug T (1983) The conjugate gradient method and trust regions in large scale optimization. SIAM J Numer Anal 20:626–637. https://doi.org/10.1137/0720042
Tröltzsch A (2016) A sequential quadratic programming algorithm for equality-constrained optimization without derivatives. Optim Lett 10:383–399. https://doi.org/10.1007/s11590-014-0830-y
Ulbrich M, Ulbrich S (2003) Non-monotone trust region methods for nonlinear equality constrained optimization without a penalty function. Math Program 95:103–135. https://doi.org/10.1007/s10107-002-0343-9
Acknowledgements
The work of J. H. Lee was supported by Korea Initiative for fostering University of Research and Innovation Program of the National Research Foundation (NRF) funded by the Korean government (MSIT) (No. 2020M3H1A1077095). Y. M. Jung was supported by the National Research Foundation of Korea NRF-2016R1A5A1008055, NRF-2019R1F1A1047134, and NRF-2022R1A2C1010537. S. Yun was supported by the National Research Foundation of Korea NRF-2016R1A5A1008055, NRF-2019R1F1A1057051, and NRF-2022R1A2C1011503.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Paulo J. S. Silva.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lee, J.H., Jung, Y.M. & Yun, S. A limited-memory trust-region method for nonlinear optimization with many equality constraints. Comp. Appl. Math. 42, 109 (2023). https://doi.org/10.1007/s40314-023-02251-8
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s40314-023-02251-8