Abstract.
We consider some algorithms for unconstrained minimization without derivatives that form linear or quadratic models by interpolation to values of the objective function. Then a new vector of variables is calculated by minimizing the current model within a trust region. Techniques are described for adjusting the trust region radius, and for choosing positions of the interpolation points that maintain not only nonsingularity of the interpolation equations but also the adequacy of the model. Particular attention is given to quadratic models with diagonal second derivative matrices, because numerical experiments show that they are often more efficient than full quadratic models for general objective functions. Finally, some recent research on the updating of full quadratic models is described briefly, using fewer interpolation equations than before. The resultant freedom is taken up by minimizing the Frobenius norm of the change to the second derivative matrix of the model. A preliminary version of this method provides some very promising numerical results.
Similar content being viewed by others
References
Conn, A.R., Gould, N.J.M., Lescrenier, M., Toint, Ph.L.: Performance of a multifrontal scheme for partially separable optimization. In: Advances in Optimization and Numerical Analysis, eds. S. Gomez & J-P. Hennart, Kluwer Academic (Dordrecht), 1994, pp. 79–96
Conn, A.R., Scheinberg, K., Toint, Ph.L.: Recent progress in unconstrained nonlinear optimization without derivatives. Math. Programming 79, 397–414 (1997)
Fletcher, R.: Practical Methods of Optimization. John Wiley & Sons (Chichester), 1987
Fletcher, R., Powell, M.J.D.: A rapidly convergent descent method for minimization. Comput. J. 6, 163–168 (1963)
Moré, J.J., Sorensen, D.C.: Computing a trust region step. SIAM J. Sci. Stat. Comput. 4, 553–572 (1983)
Powell, M.J.D.: A direct search optimization method that models the objective and constraint functions by linear interpolation. In: Advances in Optimization and Numerical Analysis. eds. S. Gomez & J-P. Hennart, Kluwer Academic (Dordrecht), 1994, pp. 51–67
Powell, M.J.D.: On the Lagrange functions of quadratic models that are defined by interpolation. Optim. Meth. Software 16, 289–309 (2001)
Powell, M.J.D.: UOBYQA: unconstrained optimization by quadratic approximation. Math. Programming 92, 555–582 (2002)
Toint, Ph.L.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comp. 32, 839–851 (1978)
Winfield, D.: Function minimization by interpolation in a data table. J. Inst. Maths Applics 12, 339–347 (1973)
Author information
Authors and Affiliations
Additional information
Presented at NTOC 2001, Kyoto, Japan.
Rights and permissions
About this article
Cite this article
Powell, M. On trust region methods for unconstrained minimization without derivatives. Math. Program., Ser. B 97, 605–623 (2003). https://doi.org/10.1007/s10107-003-0430-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-003-0430-6