Abstract
Optimization algorithms usually rely on the setting of parameters, such as barrier coefficients. We have developed a generic meta-control procedure to optimize the behavior of given iterative optimization algorithms. In this procedure, an optimal continuous control problem is defined to compute the parameters of an iterative algorithm as control variables to achieve a desired behavior of the algorithm (e.g., convergence time, memory resources, and quality of solution). The procedure is illustrated with an interior point algorithm to control barrier coefficients for constrained nonlinear optimization. Three numerical examples are included to demonstrate the enhanced performance of this method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M. Athans P.L. Falb (1966) Optimal Control McGraw-Hill Book Company New York
J.C. Butcher (2003) Numerical Methods for Ordinary Differential Equations EditionNumber2 John Wiley, Hoboken New Jersey
J.E. Dennis R.B. Schnabel (1996) Numerical Methods for Unconstrained Optimization and Nonlinear Equations Society for Industrial and Applied Mathematics (SIAM) Philadelphia
K. Eriksson D. Estep P. Hansbo C. Johnson (1996) Computational Differential Equations Cambridge University Press Lund, Sweden
L. C. Evans (1998) Partial Differential Equations American Mathematical Society Providence, Rhode Island
A.V. Fiacco G.P. McCormick (1990) Nonlinear Programming Society for Industrial and Applied Mathematics (SIAM) Philadelphia
A. Forsgren P.E. Gill M.H. Wright (2002) ArticleTitleInterior methods for nonlinear optimization SIAM Review 44 IssueID4 525–597 Occurrence Handle10.1137/S0036144502414942
T. Frankel (1997) The Geometry of Physics Cambridge University Press New York
X. Ge W. Kohn A. Nerode J.B. Remmel (1996) Hybrid systems: chattering approximations to relaxed controls R. Alur T.A. Henzinger E.D. Sontag (Eds) Hybrid Systems III. number 1066 in Lecture Notes in Computer Science Springer New York 76
I.M. Gelfand S.V. Fomin (1963) Calculus of variations Prentice-Hall Inc. Englewood Cliffs, N.J
Kohn W. and Brayman V. (2003). Convergence analysis of the hynomics incremental optimizer, Technical report, Hynomics Corporation.
Kohn W., Brayman V., Cholewinski P. and Nerode A. (2004). Control in hybrid systems, International Journal of Hybrid Systems.
Kohn, W., Nerode, A. and Remmel, J.B. (1996), Continualization: A hybrid systems control technique for computing, Proceedings of CESA’96 IMACS Multiconference. 517–521.
H.J. Kushner D.S. Clark (1978) Stochastic Approximation Methods for Constrained and Unconstrained Systems Springer-Verlag New York
H.J. Kushner G.G. Yin (1997) Stochastic Approximation Algorithms and Applications Springer-Verlag New York
H. Kwakernaak R. Sivan (1972) Linear Optimal Control Systems John Wiley and Sons New York
G.P. McCormick (1983) Nonlinear Programming: Theory, Algorithms, and Applications John Wiley and Sons New York
R.D.C. Monteiro I. Adler (1989) ArticleTitleInterior path following primal-dual algorithms Part I: Linear programming Mathematical Programming 44 27–41
R.D.C. Monteiro I. Adler (1989) ArticleTitleInterior path following primal-dual algorithms. Part II: Convex quadratic programming Mathematical Programming 44 43–66
Y. Nesterov Nemirovskii (1994) Interior-point polynomial algorithms in convex programming Society for Industrial and Applied Mathematics Philadelphia
Olver P.J. (1993). Applications of Lie Groups to Differential Equations, Springer-Verlag, second edition.
L.S. Pontryagin V.G. Boltyanskii R.V. Gamkrelidze E.F. Mischenko (1962) The Mathematical Theory of Optimal Processes Interscience New York
K. Schittkowski (1987) More Test Examples for Nonlinear Programming Codes Springer-Verlag Berlin, Heidelberg
H.D. Sherali B.O. Skarpness B. Kim (1988) ArticleTitleAn assumption-free convergence analysis for a perturbation of the scaling algorithm for linear programs, with application to the L1 estimation problem Naval Research Logistics 35 473–492
P. Tseng Y. Ye (2002) ArticleTitleOn some interior-point algorithms for nonconvex quadratic optimization Mathematical Programming 93 217–225 Occurrence Handle10.1007/s10107-002-0310-5
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was primarily done when Z. Zabinsky was visiting Clearsight Systems Inc.
Rights and permissions
About this article
Cite this article
Kohn, W., Zabinsky, Z.B. & Brayman, V. Optimization of Algorithmic Parameters using a Meta-Control Approach*. J Glob Optim 34, 293–316 (2006). https://doi.org/10.1007/s10898-005-1655-0
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10898-005-1655-0