Particle swarm optimization with FUSS and RWS for high dimensional functions

Abstract

High dimensional optimization problems play an important role in many complex engineering area. Though many variants of particle swarm optimization (PSO) have been proposed, however, most of them are tested and compared with dimension no larger than 300. Since numerical problem with high-dimension maintains a large linkage and correlation among different variables, and the number of local optimum increases significantly with different dimensions, this paper proposes a novel variant of PSO aiming to provide a balance between exploration and exploitation capability. Firstly, the fitness uniform selection strategy (FUSS) with a weak selection pressure is incorporated into the standard PSO. Secondly, “random walk strategy” (RWS) with four different form, is designed to further enhance the exploration capability to escaping from a local optimum. Finally, the proposed PSO combined with FUSS and RWS is applied to seven famous high dimensional benchmark with the dimension up to 3000. Simulation results demonstrate good performance of the new method in solving high dimensional multi-modal problems when compared with two other variants of the PSO.

Introduction

With the industrial and scientific development, many new optimization problems are needed to be solved. Several of them are complex multi-modal, high dimensional, non-differential problems. Therefore, some new optimization techniques have been designed, such as genetic algorithm [1], ant colony optimization [2], etc. However, due to the large linkage and correlation among different variables, these algorithms are easily trapped into a local optimum and failed to obtain the reasonable solution.

Particle swarm optimization (PSO) [3], [4] is a population-based, self-adaptive search optimization method motivated by the observation of simplified animal social behaviors such as fish schooling, bird flocking, etc. It is becoming very popular due to its simplicity of implementation and ability to quickly converge to a reasonably good solution [5], [6], [7].

In a PSO system, multiple candidate solutions coexist and collaborate simultaneously. Each solution called a “particle”, flies in the problem search space looking for the optimal position to land. A particle, as time passes through its quest, adjusts its position according to its own “experience” as well as the experience of neighboring particles. Tracking and memorizing the best position encountered build particle’s experience. For that reason, PSO possesses a memory (i.e. every particle remembers the best position it reached during the past). PSO system combines local search method (through self experience) with global search methods (through neighboring experience), attempting to balance exploration and exploitation.

A particle status on the search space is characterized by two factors: its position and velocity, which are updated by following equations:

$$\overrightarrow{v_j}(t+1)=w\overrightarrow{v_j}(t)+c_1{r}_1(\overrightarrow{p_j}(t)-\overrightarrow{x_j}(t))+c_2{r}_2(\overrightarrow{p_g}(t)-\overrightarrow{x_j}(t))\text{,}$$

$$\overrightarrow{x_j}(t+1)=\overrightarrow{x_j}(t)+\overrightarrow{v_j}(t+1)\text{,}$$

where $\overrightarrow{v_j}(t)$ and $\overrightarrow{x_j}(t)$ represent the velocity and position vectors of particle j at time t, respectively. $\overrightarrow{p_j}(t)$ means the best position vector which particle j had been found, as well as $\overrightarrow{p_g}(t)$ denotes the corresponding best position found by the whole swarm. Cognitive coefficient $c_1$ and social coefficient $c_2$ are constants known as acceleration coefficients, and $r_1$ and $r_2$ are two separately generated uniformly distributed random numbers in the range $[0\text{,}1]$.

The first part of (1) represents the previous velocity, which provides the necessary momentum for particles to roam across the search space. The second part, known as the “cognitive” component, represents the personal thinking of each particle. The cognitive component encourages the particles to move toward their own best positions found so far. The third part is known as the “social” component, which represents the collaborative effect of the particles, in finding the global optimal solution. The social component always pulls the particles toward the global best particle found so far.

Since particle swarm optimization is a new swarm intelligent technique, many researchers focus their attentions to this new area. One famous improvement is the introduction of the inertia weight [8], similarly with temperature schedule in the simulated annealing algorithm. Empirical results showed the linearly decreased setting of inertia weight can give a better performance, such as from 1.4 to 0 [8], and 0.9 to 0.4 [9], [10]. In 1999, Suganthan [11] proposed a time-varying acceleration coefficients automation strategy in which both $c_1$ and $c_2$ are linearly decreased during the course of run. Simulation results show the fixed acceleration coefficients at 2.0 generate better solutions. Following Suganthan’s method, Venter [12] found that the small cognitive coefficient and large social coefficient could improve the performance significantly. Further, Ratnaweera et al. [13] investigated a time-varying acceleration coefficients. In this automation strategy, the cognitive coefficient is linearly decreased during the course of run, however, the social coefficient is linearly increased inversely.

Hybrid with Kalman filter, Monson designed a new Kalman filter particle swarm optimization algorithm [14]. Similarly, Sun proposed a new quantum particle swarm optimization [15] in 2004. From the convergence point, Cui designed a global convergence algorithm – stochastic particle swarm optimization [16]. There are still many other modified methods, such as fast PSO [17], predicted PSO [18], etc. The details of these algorithms can be found in corresponding references.

The PSO algorithm has been empirically shown to perform well on many optimization problems. However, it may easily get trapped in a local optimum when solving high dimensional multi-modal problems. With respect to the PSO model, several papers have been written on the subject to deal with premature convergence, such as the addition of a queen particle [19], the alternation of the neighborhood topology [20], the introduction of subpopulation and giving the particles a physical extension [21], etc. But their results are always tested on some famous benchmark with a middle dimension less than 300, and the optimization results are not very well for the high dimensional case. In order to improve PSO’s performance on high dimensional multimodal problems, we present a new variant PSO combined with fitness uniform selection strategy (FUSS) and random walk strategy (RWS).

The rest of this paper is organized as follows: in Section 2, we summarize two significant previous developments to the standard PSO methodology. One method was used as the basis for our novel development, whereas the other one was selected as comparative measure of performance of the novel method proposed in this paper. In Section 3, we introduce the extension to PSO proposed in this paper. Experimental settings for the benchmarks and simulation strategies are explained in Section 4 and the results in comparison with the two previous developments are presented in Section 5.