Determination of initial temperature in fast simulated annealing

Abstract

In this paper, we propose a method of determining the initial temperature for continuous fast simulated annealing from the perspective of state variation. While the conventional method utilizes fitness variation, the proposed method additionally considers genotype variation. The proposed scheme is based on the fact that the annealing temperature, which includes the initial temperature, not only appears in the acceptance probability but serves as the scale parameter of a state generating probability distribution. We theoretically derive an expression for the probability of generating states to cover the state space in conjunction with the convergence property of the fast simulated annealing. We then numerically solve the expression to determine the initial temperature. We empirically show that the proposed method outperforms the conventional one in optimizing various benchmarking functions.

Appendix

In the Appendix, we derive Eq. (9) from

$$ q = \sum_{j\ne i}^{N} \mathit{Prob} \bigl\{ \bigl|x^{\ast}_{j} - x_{i}^{c} \bigr| \le \alpha \bigr\} . $$

(30)

Suppose that a state x has the range R=U−L, where x∈[L, U], and we divide the range into N intervals with widths of 2α so that N=R/(2α). For convenience, rank the centers of N intervals, $x_{j}^{c}$ (j=1,2,…,N), in ascending order in their position. That is, we assume, without a loss of generality, that $x_{i}^{c}$’s are ordered in such a way that $x_{i}^{c} > x_{j}^{c}$ when i>j. With this convention, we can express the center of j-th interval as

$$ x_{j}^{c}=2\alpha j- \alpha + L , $$

(31)

where j=1,2,…,N, so that we have

$$ x_{1}^{c}=L+\alpha\quad \mbox{and}\quad x_{N}^{c}=U- \alpha . $$

(32)

Denote $x_{j}^{\ast}$ (j=1,2,…,N) as a sample state generated by a Cauchy distribution of the location parameter $x_{j}^{c}$. Using the probability density function of the Cauchy distribution of Eq. (6), we can express

$$ \mathit{Prob} \bigl\{ \bigl\vert x_{j}^{\ast} - x_{i}^{c} \bigr\vert \le \alpha \bigr\} = \frac{T}{\pi} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} . $$

(33)

Thus, Eq. (30) becomes

$$\begin{aligned} q = &\sum_{j\ne i}^{N} \mathit{Prob} \bigl\{ \bigl|x^{\ast}_{j} - x_{i}^{c} \bigr| \le \alpha \bigr\} \end{aligned}$$

(34)

$$\begin{aligned} = & \frac{T}{\pi} \sum_{j=1}^{i-1} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} + \frac{T}{\pi} \sum_{j=i+1}^{N} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} . \end{aligned}$$

(35)

To compute the integrals in Eq. (35), consider the following transformation:

$$ y=x-x_{j}^{c}+x_{i}^{c}=x+2(i-j) \alpha . $$

(36)

With the transformation, we have

$$ \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} = \int _{x_{i}^{c}+2(i-j)\alpha -\alpha}^{x_{i}^{c}+2(i-j)\alpha +\alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} . $$

(37)

Thus, the first integral on the right hand side of Eq. (35) can be rewritten as

$$ \sum_{j=1}^{i-1} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} = \int_{x_{i}^{c}+\alpha}^{x_{i}^{c}+ 2 i \alpha - \alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} . $$

(38)

Similarly, the second integral can be rewritten as

$$ \sum_{j=i+1}^{N} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dx}{ (x-x_{j}^{c} )^{2}+T^{2}} = \int_{x_{i}^{c}-(2N+1)\alpha + 2 i \alpha}^{x_{i}^{c}-\alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} . $$

(39)

With Eqs. (38) and (39), we can rewrite Eq. (30) as

$$\begin{aligned} q = & \frac{T}{\pi} \int_{x_{i}^{c}-(2N+1)\alpha + 2 i \alpha}^{x_{i}^{c}+ 2 i \alpha - \alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} - \frac{T}{\pi} \int_{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} \\ = & \frac{1}{\pi} \tan^{-1} \biggl( \frac{2 i \alpha -\alpha}{T} \biggr) + \frac{1}{\pi} \tan^{-1} \biggl( \frac{(2N+1)\alpha - 2i\alpha}{T} \biggr)-p \\ = & \frac{1}{\pi} \tan^{-1} \biggl( \frac{R}{T} ~ \beta_{i} \biggr) + \frac{1}{\pi} \tan^{-1} \biggl( \frac{R}{T} (1-\beta_{i}) \biggr) -p , \end{aligned}$$

(40)

where

$$ \beta=\frac{i-1/2}{N} , $$

(41)

and we have used

$$ \alpha=\frac{R}{2N}\quad \mbox{and}\quad p= \frac{T}{\pi} \int _{x_{i}^{c}-\alpha}^{x_{i}^{c}+\alpha} \frac{dy}{ (y-x_{i}^{c} )^{2}+T^{2}} . $$

(42)