Abstract
Several recent papers have proposed the use of grids for solving unconstrained optimisation problems. Grid-based methods typically generate a sequence of grid local minimisers which converges to stationary points under mild conditions.
In this paper the location and number of grid local minimisers is calculated for strictly convex quadratic functions in two dimensions with certain types of grids. These calculations show it is possible to construct a grid with an arbitrary number of grid local minimisers. The furthest of these can be an arbitrary distance from the quadratic's minimiser. These results have important implications for the design of practical grid-based algorithms.
Grids based on conjugate directions do not suffer from these problems. For such grids only the grid points closest (depending on the choice of metric) to the minimiser are grid local minimisers. Furthermore, conjugate grids are shown to be reasonably stable under mild perturbations so that in practice, only approximately conjugate grids are required.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
I.D. Coope and C.J. Price, “A direct search conjugate directions algorithm for unconstrained minimization,” ANZIAM Journal, vol. 42, no. E, pp. C478–C498, 2000. http://anziamj.austms.org.au/V42/CTAC99/.
I.D. Coope and C.J. Price, “On the convergence of grid-based methods for unconstrained optimization,” SIAM Journal on Optimization, vol. 11, no. 4, pp. 859–869, 2001.
I.D. Coope and C.J. Price, “Positive bases in numerical optimization,” Computational Optimization and Applications, vol. 21, pp. 169–175, 2002.
C. Davis, “Theory of positive linear dependence,” American Journal of Mathematics, pp. 733–746, 1954.
E.D. Dolan, R.M. Lewis, and V. Torczon, “On the convergence of pattern search,” SIAM Journal on Optimization, vol. 14, no. 2, pp. 567–583, 2003.
P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization, Academic Press: London, 1981.
R. Hooke and T.A. Jeeves, “Direct search solution of numerical and statistical problems,” Journal of the Association for Computing Machinery (ACM), vol. 8, no. 2, pp. 212–219, 1961.
J. Nocedal and S.J. Wright, Numerical Optimization, Springer: New York, 1999.
C.J. Price, I.D. Coope, and D. Byatt, “A convergent variant of the Nelder-Mead algorithm,” Journal of Optimization Theory and Applications, vol. 113, no. 1, pp. 5–19, 2002.
H.H. Rosenbrock, “An automatic method for finding the greatest or least value of a function,” The Computer Journal, vol. 3, no. 3, pp. 175–184, 1960.
V. Torczon, “On the convergence of pattern search algorithms,” SIAM Journal on Optimization, vol. 7, no. 1, pp. 1–25, 1997.
M.H. Wright, “Direct search methods: once scorned, now respectable,” in D.F. Griffiths and G.A. Watson (eds.), Numerical Analysis 1995, number 344 in Pitman Research Notes in Mathematics Series, Addison Wesley Longman Ltd: Harlow, UK, 1996, pp. 191–208.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Byatt, D., Coope, I. & Price, C. Conjugate Grids for Unconstrained Optimisation. Computational Optimization and Applications 29, 49–68 (2004). https://doi.org/10.1023/B:COAP.0000039488.79258.d7
Issue Date:
DOI: https://doi.org/10.1023/B:COAP.0000039488.79258.d7