Abstract
Single-thread algorithms for global optimization differ in the way computational effort between exploitation and exploration is allocated. This allocation ultimately determines overall performance. For example, if too little emphasis is put on exploration, the globally optimal solution may not be identified. Increasing the allocation of computational effort to exploration increases the chances of identifying a globally optimal solution but it also slows down convergence. Thus, in a single-thread implementation of model-based search exploration and exploitation are substitutes. In this paper we propose a new algorithmic design for global optimization based upon multiple interacting threads. In this design, each thread implements a model-based search in which the allocation of exploration versus exploitation effort does not vary over time. Threads interact through a simple acceptance-rejection rule preventing duplication of search efforts. We show the proposed design provides a speedup effect which is increasing in the number of threads. Thus, in the proposed algorithmic design, exploration is a complement rather than a substitute to exploitation.
Similar content being viewed by others
References
Ackely, D.H.: A Connectionist Machine for Genetic Hillclimbing. Academic Publishers, Boston (1987)
Back, T.: Evolutionary Algorithms in Theory an Practice. Oxford University Press, Oxford (1996)
Bekkera, J., Aldrich, C.: The cross-entropy method in multi-objective optimisation: an assessment. Eur. J. Oper. Res. 211(2), 112–121 (2011)
Bonnans, J.F., Gilbert, J.C., Lemarechal, C., Sagastizabal, C.A.: Numerical Optimization: Theoretical and Practical Aspects, 2nd edn. Springer, Berlin (2006)
Botev, Z.I., Kroese, D.P., Rubinstein, R.Y., L’Ecuyer, P.: The cross-entropy method for optimization. In: Govindaraju, V., Rao, C.R. (eds.) Handbook of Statistics: Machine Learning, vol. 31. North-Holland, Amsterdam (2011)
Chen, J., Xin, B., Peng, Z., Dou, L.: Optimal contraction theorem for exploration-exploitation tradeoff in search and optimization. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 39(3), 680–691 (2009)
Estrada, J.F.S., Casado, L.G., Garcia, I.: Adaptive parallel interval global optimization algorithms based on their performance for non-dedicated multicore architectures. In: 2011 19th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), pp. 252–256 (2011)
Evtushenko, Y., Posypkin, M., Sigal, I.: A framework for parallel large-scale global optimization. Comput. Sci. Res. Dev. 23(3–4), 211–215 (2009)
Griewank, A.O.: Generalized decent for global optimization. J. Optim. Theory Appl. 34(1), 11–39 (1981)
Grosso, A., Locatelli, M., Fabio, S.: A population-based approach for hard global optimization problems based on dissimilarity measures. Math. Program. 110(2), 373–404 (2007)
Hong, S., Hyesoon, K.: An analytical model for a GPU architecture with memory-level and thread-level parallelism awareness. In: Proceedings of the 36th Annual International Symposium on Computer Architecture, pp. 152–163 (2009)
Hu, J., Fu, M., Marcus, S.: A model reference adaptive search method for global optimization. Oper. Res. 55(3), 549–568 (2007)
Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. Learn. Intell. Optim. 6683, 507–523 (2011)
Jin, Y.: A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput. 9(1), 3–12 (2005)
Kennedy J. Eberhart, R.: Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks, vol. IV, pp. 1942–1948 (1995)
Leary, R.: Global optimization on funneling landscapes. J. Glob. Optim. 18(4), 367–383 (2000)
Qin, A.K., Huang, V.L., Suganthan, P.N.: Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans. Evol. Comput. 13(2), 398–417 (2009)
Schutte, J.F., Reinbolt, J.A., Fregly, B.J., Haftka, R.T., George, A.D.: Parallel global optimization with the particle swarm algorithm. Comput. Sci. Res. Dev. 61(13), 2296–2315 (2004)
Srinivas, M., Patnaik, L.: Generic algorithms: a survey. Methodol. Comput. Appl. Probab. 2, 127–190 (1994)
Suzuki, J.: A Markov chain analysis on simple genetic algorithms. IEEE Trans. Syst. Man Cybern. 4, 655–659 (1995)
Zhang, W., Calder B., Tullsen, D.M.: An event-driven multithreaded dynamic optimization framework. In: 14th International Conference on Parallel Architectures and Compilation Techniques, PACT 2005. pp. 87–98 (2005)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wang, Y., Garcia, A. Interactive model-based search for global optimization. J Glob Optim 61, 479–495 (2015). https://doi.org/10.1007/s10898-014-0188-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-014-0188-9