On Simulated Annealing and Nested Annealing

Abstract

Simulated Annealing is a family of randomized algorithms used to solve many combinatorial optimization problems. In practice they have been applied to solve some presumably hard (e.g., NP-complete) problems. The level of performance obtained has been promising [2,5,6,14]. The success of this heuristic technique has motivated analysis of this algorithm from a theoretical point of view. In particular, people have looked at the convergence of this algorithm. They have shown (see e.g., [10]) that this algorithm converges in the limit to a globally optimal solution with probability 1. However few of these convergence results specify a time limit within which the algorithm is guaranteed to converge (with some high probability, say). We present, for the first time, a simple analysis of SA that will provide a time bound for convergence with overwhelming probability. The analysis will hold no matter what annealing schedule is used. Convergence of Simulated Annealing in the limit will follow as a corollary to our time convergence proof. In this paper we also look at optimization problems for which the cost function has some special properties. We prove that for these problems the convergence is much faster. In particular, we give a simpler and more general proof of convergence for Nested Annealing, a heuristic algorithm developed in [12]. Nested Annealing is based on defining a graph corresponding to the given optimization problem. If this graph is ‘small separable’, they [12] show that Nested Annealing will converge ‘faster’. For an arbitrary optimization problem, we may not have any knowledge about the ‘separability’ of its graph. In this paper we give tight bounds for the ‘separability’ of a random graph. We then use these bounds to analyze the expected behavior of Nested Annealing on an arbitrary optimization problem. The ‘separability’ bounds we derive in this paper are of independent interest and have the potential of finding other applications.