Abstract
The efficiency of metaheuristics depends on parameters. Often this relation is defined by statistical simulation and have many local minima. Therefore, methods of stochastic global optimization are needed to optimize the parameters. The traditional numerical analysis considers optimization algorithms that guarantee some accuracy for all functions to be optimized. This includes the exact algorithms. Limiting the maximal error requires a computational effort that often increases exponentially with the size of the problem [Horst and Pardalos (1995), Handbook of Global Optimization, Kluwer Academic Publisher, Dordrecht/Boston/London]. That limits practical applications. An alternative is the average analysis where the expected error is made as small as possible [Calvin and Zilinskas (2000), JOTA Journal of Optimization Theory and Applications, 106, 297–307]. The average is taken over a set of functions to be optimized. The average analysis is called the Bayesian Approach (BA) [Diaconis (1988), Statistical Decision Theory and Related Topics, Springer-Verlag, Berlin, pp. 163–175, Mockus and Mockus (1987), Theory of Optimal Decision, Vol. 12, Institute of Mathematics and Cybernetics, Akademia Nauk Lithuanian SSR, Vilnius, Lithuania pp. 57–70]. Application of BA to optimization of heuristics is called the Bayesian Heuristic Approach (BHA) [Mockus (2000), A Set of Examples of Global and Discrete Optimization: Application of Bayesian Heuristic Approach, Kluwer Academic Publishers, Dordrecht, ISBN 0-7923-6359-0]. If the global minimum is known then the traditional stopping condition is applied: stop if the distance to the global minimum is within acceptable limits. If the global minimum is not known then the different approach is natural: minimize the average deviation during the fixed time limit because there is no reason to stop before. If the distance from the global minimum is not known the efficiency of method is tested by comparing with average results of some other method. “Pure” Monte Carlo is a good candidate for such comparison because it converges and does not depend on parameters that can be adjusted to a given problem by using some expert knowledge or additional test runs. In this paper a short presentation of the basic ideas of BHA [described in detail in Mockus (2000, A Set of Examples of Global and Discrete Optimization: Application of Bayesian Heuristic Approach, Kluwer Academic Publishers, Dordrecht, ISBN 0-7923-6359-0) and Mockus (1989, Bayesian Approach to Global Optimization, Kluwer Academic Publishers, Dordrec ht-London-Boston)] is given. The simplest knapsack problem is for initial explanation of BHA. The possibilities of application are illustrated by a school scheduling problem and other examples. Designed for distance graduate studies of the theory of games and markets in the Internet environment. All the algorithms are implemented as platform independent Java applets or servlets therefore readers can easily verify and apply the results for studies and for real life heuristic optimization problems. To address this idea, the paper is arranged in a way convenient for the direct reader participation. Therefore, a part of the paper is written as some “user guide”. The rest is a short description of optimization algorithms and models. All the remaining information is on web-sites, for example http://pilis.if.ktu.lt/~mockus.
Similar content being viewed by others
References
R. Horst P. Pardalos (1995) Handbook of Global Optimization Kluwer Academic Publishers Dordrecht/Boston/London
J. Calvin A. Zilinskas (2000) ArticleTitleOne-dimensional p-algorithm with convergence rate o (n 3c) for smooth functions JOTA Journal of Optimization Theory and Applications 106 297–307 Occurrence Handle10.1023/A:1004699313526
P. Diaconis (1988) Bayesian numerical analysis Statistical Decision Theory and Related Topics Springer-Verlag Berlin 163–175
Mockus, J. and Mockus, L. (1987), Bayesian approach to global optimization and applications, Theory of Optimal Decision, vol. 12, Institute of Mathematics and Cybernetics, Akademia Nauk Lithuanian SSR, Vilnius, Lithuania, pp. 57–70 (in Russian).
J. Mockus (2000) A Set of Examples of Global and Discrete Optimization: Application of Bayesian Heuristic Approach Kluwer Academic Publishers Dordrecht
J. Mockus (1989) Bayesian Approach to Global Optimization Kluwer Academic Publishers Dordrecht-London-Boston
M. DeGroot (1970) Optimal Statistical Decisions McGraw-Hill New York
D. Lindley (1971) Bayesian Statistics: A Review SIAM Philadelphia
L. Savage (1954) Foundations of Statistics Wiley New York
T.L. Fine (1973) Theories of Probability Academic Press New York
Zilinskas, A. (1986), Global Optimization: Axiomatic of Statistical Models, Algorithms and Their Applications, Mokslas, Vilnius, Lithuania (in Russian).
H. Kushner (1964) ArticleTitleA new method of locating the maximum point of an arbitrary multi-peak curve in the presence of noise Journal of Basic Engineering 86 97–100
Saltenis, V. (1971), On a method of multi-extremal optimization. Automatics and Computers (Avtomatika i Vychislitelnayya Tekchnika) 33–38 (in Russian).
A. Torn A. Zilinskas (1989) Global optimization Springer-Verlag Berlin
J. Mockus W. Eddy A. Mockus L. Mockus G. Reklaitis (1997) Bayesian Heuristic Approach to Discrete and Global Optimization Kluwer Academic Publishers Dordrecht-London-Boston
D.E. Goldberg (1989) Genetic Algorithms in Search, Optimization, and Machine Learning Addison-Wesley Reading, MA
T. Mavridou P. Pardalos L. Pitsoulis M. Resende (1998) ArticleTitleA GRASP for the biquadratic assignment problem European Journal of Operations Research 105 613–621 Occurrence Handle10.1016/S0377-2217(97)00083-0
Glover, F. Tabu search: improved solution alternatives, Mathematical Programming. State of the Art 1994, University of Michigan, MI, 64–92.
C. Floudas P. Pardalos C. Adjiman W. Esposito Z.H. Gu u. S. Harding J. Klepeis C. Meyer C. Schweiger (1999) Handbook of Test Problems in Local and Global Optimization Kluwer Academic Publishers Dordrecht-Boston-London
H. Cohn M. Fielding (1999) ArticleTitleSimulated annealing: searching for an optimal temperature schedule SIAM Journal Optimization 9 779–802 Occurrence Handle10.1137/S1052623497329683
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Mockus, J. A System for Distance Studies and Applications of Metaheuristics. J Glob Optim 35, 637–651 (2006). https://doi.org/10.1007/s10898-005-5369-0
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s10898-005-5369-0