Abstract
Direct search methods have been an area of active research in recent years. On many real-world problems involving computationally expensive and often noisy functions, they are one of the few applicable alternatives. However, although these methods are usually easy to implement, robust and provably convergent in many cases, they suffer from a slow rate of convergence.
Usually these methods do not take the local topography of the objective function into account. We present a new algorithm for unconstrained optimisation which is a modification to a basic generating set search method. The new algorithm tries to adapt its search directions to the local topography by accumulating curvature information about the objective function as the search progresses.
The curvature information is accumulated over a region thus smoothing out noise and minor discontinuities. We present some theory regarding its properties, as well as numerical results. Preliminary numerical testing shows that the new algorithm outperforms the basic method most of the time, sometimes by significant relative margins, on noisy as well as smooth problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abramson, M.A.: Mixed variable optimization of a load-bearing thermal insulation system using a filter pattern search algorithm. Optim. Eng. 5, 157–177 (2004)
Audet, C., Dennis Jr., J.E.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188–217 (2006)
Audet, C., Dennis Jr., J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980–1010 (2004)
Booker, A.J., Dennis Jr., J.E., Frank, P.D., Serafini, D.B., Torczon, V.: Optimization using surrogate objectives on a helicopter test example. In: Borggaard, J.T., Burns, J., Cliff, E., Sherk, S. (eds.) Computational Methods for Optimal Design and Control. Birkhäuser, Boston (1998)
Borggaard, J., Pelletier, D., Vugrin, K.: On sensitivity analysis for problems with numerical noise. AIAA paper 2002–5553. Presented at the 9th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Atlanta, GA (2002)
Burkardt, J., Gunzburger, M., Peterson, J.: Insensitive functionals, inconsistent gradients, spurious minima, and regularized functionals in flow optimization problems. Int. J. Comput. Fluid Dyn. 16(3), 171–185 (2002)
Coope, I.D., Price, C.J.: A direct search conjugate directions algorithm for unconstrained minimization. ANZIAM J. 42(E), C478–C498 (2000)
Davis, C.: Theory of positive linear dependence. Am. J. Math. 76, 733–746 (1954)
Dennis Jr., J.E., Price, C.J., Coope, I.D.: Direct search methods for nonlinearly constrained optimization using filters and frames. Optim. Eng. 5, 123–144 (2004)
Dennis Jr., J.E., Torczon, V.: Direct search methods on parallel machines. SIAM J. Optim. 1(4), 448–474 (1991)
Dolan, E.D., Lewis, R.M., Torczon, V.: On the local convergence of pattern search. SIAM J. Optim. 14(2), 567–583 (2003)
Edwards, C.H.: Advanced Calculus of Several Variables. Academic, New York (1973). ISBN 0-12-232550-8
Frimannslund, L., Steihaug, T.: A generating set search method exploiting curvature and sparsity. In: Proceedings of the Ninth Meeting of the Nordic Section of the Mathematical Programming Society, Linköping, Sweden, pp. 57–71. Linköping University, Linköping (2004)
Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic, New York (1981). ISBN 0-12-283950-1
Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385–482 (2003)
Lewis, R.M., Torczon, V.: Pattern search methods for linearly constrained minimization. SIAM J. Optim. 10(3), 917–941 (2000)
Lewis, R.M., Torczon, V.: A globally convergent augmented Lagrangian pattern search algorithm for optimization with general constraints and simple bounds. SIAM J. Optim. 12(4), 1075–1089 (2002)
Lucidi, S., Sciandrone, M.: On the global convergence of derivative-free methods for unconstrained optimization. SIAM J. Optim. 13(1), 97–116 (2002)
Marsden, A.L., Wang, M., Dennis Jr., J.E., Moin, P.: Optimal aeroacoustic shape design using the surrogate management framework. Optim. Eng. 5, 235–263 (2004)
Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)
Price, C.P., Toint, P.: Exploiting problem structure in pattern search methods for unconstrained optimization. Optim. Methods Softw. 21(3), 479–491 (2006)
Rosenbrock, H.H.: An automatic method for finding the greatest or least value of a function. Comput. J. 3(3), 175–184 (1960)
Torczon, V.: Multi-directional search: a direct search algorithm for parallel machines. Ph.D. thesis, Department of Mathematical Sciences, Rice University, Houston, TX (1989). Available as Technical report 90-07, Department of Computational and Applied Mathematics, Rice University, Houston, TX
Torczon, V.: On the convergence of pattern search algorithms. SIAM J. Optim. 7(1), 1–25 (1997)
Torczon, V., Trosset, M.W.: From evolutionary operation to parallel direct search: pattern search algorithms for numerical optimization. Comput. Sci. Stat. 29, 396–401 (1998)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the Norwegian Research Council (NFR).
Rights and permissions
About this article
Cite this article
Frimannslund, L., Steihaug, T. A generating set search method using curvature information. Comput Optim Appl 38, 105–121 (2007). https://doi.org/10.1007/s10589-007-9038-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-007-9038-8