Abstract
Generalized hill climbing algorithms provide a framework to describe and analyze metaheuristics for addressing intractable discrete optimization problems. The performance of such algorithms can be assessed asymptotically, either through convergence results or by comparison to other algorithms. This paper presents necessary and sufficient convergence conditions for generalized hill climbing algorithms. These conditions are shown to be equivalent to necessary and sufficient convergence conditions for simulated annealing when the generalized hill climbing algorithm is restricted to simulated annealing. Performance measures are also introduced that permit generalized hill climbing algorithms to be compared using random restart local search. These results identify a solution landscape parameter based on the basins of attraction for local optima that determines whether simulated annealing or random restart local search is more effective in visiting a global optimum. The implications and limitations of these results are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aarts, E. and J. Korst. (2002). "Selected Topics in Simulated Annealing." In P. Hansen and C.C. Ribeiro (Eds.), Essays and Surveys on Metaheuristics, Chapter 1, Norwell, MA, Kluwer Academic Publishers, pp. 1–37.
Armstrong, D.E. and S.H. Jacobson. (2003). "Studying the Complexity of Global Verification for NP-hard Discrete Optimization Problems." Journal of Global Optimization 27(1), 83–96.
Billingsley, P. (1979). Probability and Measure,New York, John Wiley and Sons.
Charon, I. and O. Hudry. (2001). "The Noising Method: A Generalization of Some Metaheuristics." European Journal of Operational Research 135(1), 86–101.
Cohn, H. and M. Fielding. (1999). "Simulated Annealing: Searching for an Optimal Temperature Schedule." SIAM Journal On Optimization 9(3), 779–802.
Cormen, T.H. C.E. Leiserson, and R.L. Rivest. (1997). Introduction to Algorithms,New York: McGraw-Hill.
Dueck, G. and T. Scheuer. (1990). "Threshold Accepting: A General Purpose Optimization Algorithm Appearing Superior to Simulated Annealing." European Journal of Operational Research 46, 271–281.
Ferreira, A.G. and J. Zerovnik. (1993). "Bounding the Probability of Success of Stochastic Methods for Global Optimization." Computers and Mathematics Applications 25(10/11) 1–8.
Fox, B.L. (1993). "Integrating and Accelerating Tabu Search, Simulated Annealing, and Genetic Algorithms." Annals of Operations Research 41, 47–67.
Fox, B.L. (1994). "Random Restarting versus Simulated Annealing." Computers and Mathematics Applications 27, 33–35.
Fox, B.L. (1995). "Faster Simulated Annealing." SIAM Journal of Optimization 5, 488–505.
Fox, B.L. (1999). Strategies for Quasi-Monte Carlo. Norwell, MA: Kluwer Academic Publishing.
Garey, M.R. and D.S. Johnson. (1979). Computers and Intractability: A Guide to the Theory of NP-Completeness San Francisco, CA: Freeman and Company.
Garnier, J. and L. Kallel. (2002). "Efficiency of Local Search with Multiple Local Optima." SIAM Journal of Discrete Mathematics 15(1), 122–141.
Glover, F. and M. Laguna. (1997). Tabu Search, Norwell, MA: Kluwer Academic Publishing.
Hajek, B. (1988). "Cooling Schedules for Optimal Annealing." Mathematics of Operations Research 13, 311–329.
Hart, W.E. (1999). "Sequential Stopping Rules for Random Optimization Methods with Applications to Multistart Local Search." SIAM Journal on Optimization 9(1), 270–290.
Henderson, D, S.H. Jacobson, and A.W. Johnson. (2003). "The Theory and Practice of Simulated Annealing." In F. Glover and G. Kochenberger (Eds.), Handbook on Metaheuristics, Chapter 10, Norwell, MA: Kluwer Academic Publishers, pp. 287–319.
Hoos, H.H. (1999). "On the Run-Time Behavior of Stochastic Local Search Algorithms for SAT." Proceedings of the AAAI-99 MIT Press, pp. 661–666.
Hoos, H.H. and T. Stutzle. (2000). "Local Search Algorithms for SAT: An Empirical Evaluation." Journal of Automated Reasoning 24, 421–481.
Jacobson, S.H. and K.A. Sullivan, and A.W. Johnson. (1998). "Discrete Manufacturing Process Design Optimiza-tion Using Computer Simulation and Generalized Hill Climbing Algorithms." Engineering Optimization 31, 247–260.
Jacobson, S.H. and D. Solow. (1993). "The Effectiveness of Finite Improvement Algorithms for Finding Global Optima." Zeitschrift fur Operations Research (ZOR)—Methods and Models of Operations Research 37(3), 257–272.
Jacobson, S.H. E. Yücesan. (2004). "Global Optimization Performance Measures for Generalized Hill Climbing Algorithms." Journal of Global Optimization 29(2), 177–193.
Johnson, A.W. and S.H. Jacobson. (2002a). "A Convergence Result for a Class of Generalized Hill Climbing Algorithms." Applied Mathematics and Computation 125(2/3), 359–373.
Johnson, A.W. and S.H. Jacobson. (2002b). "On the Convergence of Generalized Hill Climbing Algorithms." Discrete Applied Mathematics 119(1/2), 37–57.
Johnson, D.S., C.H. Papadimitriou and M. Yannakakis. (1988). "HowEasy is Local Search?" Journal of Computers and Systems Science 37(1), 79–100.
Kirkpatrick, S., C.D. Gelatt Jr., and M.P. Vecchi. (1983). "Optimizaton by Simulated Annealing." Science 220, 671–680.
Lin, S. and B.W. Kernighan. (1973). "An Effective Heuristic for the Traveling Salesman Problem." Operations Research 21, 498–516.
Mitra, D., F. Romeo, and A.L. Sangiovanni-Vincentelli. (1986). "Convergence and Finite-Time Behavior of Sim-ulated Annealing." Advances in Applied Probability 18, 747–771.
Sullivan, K.A. and S.H. Jacobson. (2001). "A Convergence Analysis of Generalized Hill Climbing Algorithms." IEEE Transactions on Automatic Control 46(8), 1288–1293.
Theodosopoulus, T.V. (1999). "Some Remarks on the Optimal Level of Randomization in Global Optimization." In P. Pardalos, S. Rajasekaran, and J. Rolim (Eds.), Randomization Methods in Algorithm Design in DIMACS Series in Discrete Mathematics and Theoretical Computer Science, American Mathematical Society, pp. 303–318.
Tovey, C.A. (1983). "On the Number of Iterations of Local Improvement Algorithms." Operations Research Letters 2, 231–238.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Jacobson, S.H., Yücesan, E. Analyzing the Performance of Generalized Hill Climbing Algorithms. Journal of Heuristics 10, 387–405 (2004). https://doi.org/10.1023/B:HEUR.0000034712.48917.a9
Issue Date:
DOI: https://doi.org/10.1023/B:HEUR.0000034712.48917.a9