Abstract
In this paper we introduce the LeGO (Learning for Global Optimization) approach for global optimization in which machine learning is used to predict the outcome of a computationally expensive global optimization run, based upon a suitable training performed by standard runs of the same global optimization method. We propose to use a Support Vector Machine (although different machine learning tools might be employed) to learn the relationship between the starting point of an algorithm and the final outcome (which is usually related to the function value at the point returned by the procedure). Numerical experiments performed both on classical test functions and on difficult space trajectory planning problems show that the proposed approach can be very effective in identifying good starting points for global optimization.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Addis, B., Cassioli, A., Locatelli, M., Schoen, F.: A global optimization method for the design of space trajectories. COAP, published on line (2009). doi:10.1007/s10589-009-9261-6
Addis, B., Locatelli, M., Schoen, F.: Local optima smoothing for global optimization. Optim. Methods Softw. 20(45), 417–437 (2005)
Ampatzis, C., Izzo, D.: Machine learning techniques for approximation of objective functions in trajectory optimisation. In: Proceedings of the IJCAI-09 Workshop on Artificial Intelligence in Space, pp. 1–6 (2009)
Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Min. Knowl. Discov. 2(2), 121–167 (1998)
Byrd, R.H., Nocedal, J., Lu, P., Zhu, C.: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. Tech. Rep., Northwestern University, Department of Electrical Engineering and Computer Science (1995)
Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm (2001)
Dill, K.A., Phillips, A.T., Rosen, J.B.: CGU: an algorithm for molecular structure prediction. In: Bigler, L.T., Coleman, T.F., Conn, A.R., Santosa, F.N. (eds.) Large-Scale Optimization with Applications. Part 3: Molecular Structure and Optimization, vol. 94, pp. 1–21. Springer, Berlin (1997)
Dill, K.A., Phillips, A.T., Rosen, J.B.: Protein structure and energy landscape dependence on sequence using a continuous energy function. J. Comput. Biol. 4(3), 227–240 (1997)
Grosso, A., Locatelli, M., Schoen, F.: A population based approach for hard global optimization problems based on dissimilarity measures. Math. Programm. 110(2), 373–404 (2007)
Izzo, D., Becerra, V., Myatt, D., Nasuto, S., Bishop, J.: Search space pruning and global optimisation of multiple gravity assist spacecraft trajectories. J. Global Optim. 38, 283–296 (2007)
Jin, Y.: A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput. Fusion Found. Methodol. Appl. 9(1), 3–12 (2005)
Jin, Y., Olhofer, M., Sendhoff, B.: A framework for evolutionary optimization with approximate fitness functions. IEEE Trans. Evol. Comput. 6(5), 481–494 (2002)
Leary, R.H.: Global optimization on funneling landscapes. J. Global Optim. 18, 367–383 (2000)
Lourenço, H.R., Martin, O.C., Stülze, T.: Iterated local search. In: Glover, F.W., Kochenberger, G.A. (eds.) Handbook of Metaheuristics, pp. 321–353. Kluwer Academic, Dordrecht (2003)
Mangasarian, O.L., Rosen, J.B., Thompson, M.E.: Global minimization via piecewise-linear underestimation. J. Global Optim. 32(1), 1–9 (2005)
Olympio, J.T., Marmorat, J.-P.: Global trajectory optimization: can we prune the solution space when considering deep space manoeuvres? Final Report, ESA (2008)
Rinnooy Kan, A.H.G., Timmer, G.T.: Stochastic global optimization methods. Part I: Clustering methods. Math. Programm. 39, 27–56 (1987)
Rinnooy Kan, A.H.G., Timmer, G.T.: Stochastic global optimization methods. Part II: Multi level methods. Math. Programm. 39, 57–78 (1987)
Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2002)
Vapnik, V.N.: Statistical Learning Theory. Wiley, New York (1998)
Vasile, M.: Design of Earth-Mars transfer trajectories using evolutionary-branching technique. Acta Astronaut. 56, 705–720 (2005)
Vaz, I.F., Vicente, L.N.: A particle swarm pattern search method for bound constrained global optimization. J. Glob. Optim. 39, 197–219 (2007)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cassioli, A., Di Lorenzo, D., Locatelli, M. et al. Machine learning for global optimization. Comput Optim Appl 51, 279–303 (2012). https://doi.org/10.1007/s10589-010-9330-x
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-010-9330-x