A derivative-free variant called DFSA of Dekkers and Aarts’ continuous simulated annealing algorithm

Abstract

We propose a derivative-free implementation of Dekkers and Aarts’ continuous simulated annealing (SA). Essential in DFSA is the ’random direction’ local technique that we introduce. We study the convergence properties of the resulting algorithm and test its performance on a set of 50 problems. Numerical results are presented which show the robustness of the algorithm. Numerical comparisons with SA, two derivative-free simulated annealing algorithms and three population-based global optimization algorithms show that the derivative-free SA, DFSA, offers a reasonable alternative to some recent global optimization algorithms, especially for problems requiring a ‘direct search’ type algorithm.

Introduction

We consider the problem of finding the global minimum of the optimization problem

$$\text{minimize}f(x)\text{subject to}x\in \Omega \text{,}$$

where $f:\Omega \subset {\mathbb{R}}^n\to \mathbb{R}$ is a continuous real-valued function and $\Omega =\{(x_1\text{,}{x}_2\text{,}\dots \text{,}{x}_n)\in {\mathbb{R}}^n|l_i⩽x_i⩽u_i\text{,}{l}_i\text{,}{u}_i\in \mathbb{R}\}$. A point $x^{\ast }$ is said to be a global minimizer of f in $\Omega$ if $f^{\ast }=f(x^{\ast })⩽f(x)\text{,}\forall x\in \Omega$. In many applications, for example, in applied sciences and engineering, the function of interest may be non-linear, non-smooth or simulation-based. It is with this view in mind, that some search methods that do not require much information about the function were developed. These methods, popularly known as the direct search methods, are applicable to a wide range of problems. A number of population-based direct search methods for global optimization are well known and widely used in practice, see Price [1], Storn and Price [2] and Ali and Kaelo [3].

Simulated annealing, on the other hand, is a single point-based continuous global optimization algorithm. To date, several simulated annealing algorithms have been suggested in the literature, see Dekkers and Aarts [4], Alluffi-Pentini et al. [5] and Ali and Storey [6]. However, extensive numerical testing of these algorithms are not reported, let alone using small size problems, to judge the strengths and weaknesses of them. Such studies are also not carried out in practical applications of the simulated annealing algorithm [7], [8], [9].

In this paper, we are concerned with the SA algorithm of Dekkers and Aarts [4]. Both the implementation and convergence properties of the SA algorithm are based on derivative information. We propose a, user-friendly, derivative-free implementation of the SA algorithm of Dekkers and Aarts [4], and report numerical results on a large set of test problems. We denote the new implementation of SA by DFSA. We motivate the design of DFSA and study its theoretical properties.

A number of papers dealing with derivative-free simulated annealing have been presented in literature [10], [11], [12]. However, the structure of the DFSA algorithm is different from the ones presented in these papers. The algorithm proposed by Corna et al. [10] has a similarity with DFSA in that it generates trial points in the coordinate directions in a deterministic fashion. DFSA generates trial points using a bimodal probability distribution which ensures that 25% of the points are generated in the coordinate directions. However, results of only two problems are reported in Corna et al. [10]. Hence, we have compared performance of DFSA with SA of Dekkers and Aarts [4], two derivative-free simulated annealing algorithms [11], [12] and three recent population-based algorithms [3], [13], [14].

This paper is organized as follows. Section 2 briefly introduces the SA algorithm for continuous problems. Section 3 presents the derivative-free simulated annealing algorithm, DFSA. Results are presented in Section 4 and conclusions are made in Section 5.