Particle swarm optimization with neighborhood-based budget allocation

Abstract

The standard particle swarm optimization (PSO) algorithm allocates the total available budget of function evaluations equally and concurrently among the particles of the swarm. In the present work, we propose a new variant of PSO where each particle is dynamically assigned different computational budget based on the quality of its neighborhood. The main goal is to favor particles with high-quality neighborhoods by asynchronously providing them with more function evaluations than the rest. For this purpose, we define quality criteria to assess a neighborhood with respect to the information it possesses in terms of solutions’ quality and diversity. Established stochastic techniques are employed for the final selection among the particles. Different variants are proposed by combining various quality criteria in a single- or multi-objective manner. The proposed approach is assessed on widely used test suites as well as on a set of real-world problems. Experimental evidence reveals the efficiency of the proposed approach and its competitiveness against other PSO-based variants as well as different established algorithms.

Appendix: Test problems

1.1 Standard test suite

The standard test suite consists of the following problems:

Test Problem 0 (TP0—Sphere) [19]. This is a separable $n$-dimensional problem, defined as

$$\begin{aligned} f (x) = \sum _{i=1}^{n} x_{i}^{2}, \end{aligned}$$

(21)

and it has a single global minimizer, $x^{*} = (0,0,\ldots ,0)^{\top }$, with $f(x^{*}) = 0$.

Test Problem 1 (TP1—Generalized Rosenbrock) [19]. This is a non-separable $n$-dimensional problem, defined as

$$\begin{aligned} f(x) = \sum _{i=1}^{n-1} \left( 100 \left( x_{i+1}-x_{i}^{2} \right) ^{2} + \left( x_{i-1} \right) ^{2} \right) , \end{aligned}$$

(22)

and it has a global minimizer, $x^{*} = (1,1,\ldots ,1)^{\top }$, with $f(x^{*}) = 0$.

Test Problem 2 (TP2—Rastrigin) [19]. This is a separable $n$-dimensional problem, defined as

$$\begin{aligned} f(x) = 10 n + \sum _{i=1}^{n} \big ( x_{i}^{2} - 10 \cos (2\pi x_{i})\big ), \end{aligned}$$

(23)

and it has a global minimizer, $x^{*} = (0,0,\ldots ,0)^{\top }$, with $f(x^{*}) = 0$.

Test Problem 3 (TP3—Griewank) [19]. This is a non-separable $n$-dimensional problem, defined as

$$\begin{aligned} f(x) = \sum _{i=1}^{n} \frac{x_{i}^{2}}{4000} - \prod _{i=1}^{n} \cos \left( \frac{x_{i}}{\sqrt{i}}\right) + 1, \end{aligned}$$

(24)

and it has a global minimizer, $x^{*} = (0,0,\ldots ,0)^{\top }$, with $f(x^{*}) = 0$.

Test Problem 4 (TP4—Ackley) [19]. This is a non-separable $n$-dimensional problem, defined as

$$\begin{aligned} f(x)&= 20 + \exp (1) - 20\exp \left( -0.2\sqrt{\frac{1}{n}\sum _{i=1}^{n}x_{i}^{2}}\right) \nonumber \\&- \exp \left( \frac{1}{n}\sum _{i=1}^{n}\cos (2\pi x_{i})\right) , \end{aligned}$$

(25)

and it has a global minimizer, $x^{*} = (0,0,\ldots ,0)^{\top }$, with $f(x^{*}) = 0$.

1.2 Nonlinear systems

This test set consists of six real-application problems, which are modeled as systems of nonlinear equations. Computing a solution of a nonlinear system is a very challenging task and it has received the ongoing attention of the scientific community. A common methodology for solving such systems is their transformation to an equivalent global optimization problem, which allows the use of a wide range of optimization tools. The transformation produces a single objective function by aggregating all the system’s equations, such that the solutions of the original system are exactly the same with that of the derived optimization problem.

Consider the system of nonlinear equations:$\begin{aligned} \left\{ \begin{array}{ll} f_1(x)=0, \\ f_2(x)=0, \\ \qquad \vdots \\ f_m(x)=0, \end{array} \right. \end{aligned}$with $x \in S \subset \mathbb {R}^n$. Then, the objective function,

$$\begin{aligned} f(x) = \sum _{i=1}^{m} |f_i(x)|, \end{aligned}$$

(26)

defines an equivalent optimization problem. Obviously, if $x^*$ with $f(x^*) = 0$ is a global minimizer of the objective function, then $x^*$ is also a solution of the corresponding nonlinear system and vice versa.

In our experiments, we considered the following nonlinear systems, previously employed by Grosan and Abraham [9] to justify the usefulness of evolutionary approaches as efficient solvers of nonlinear systems:

Test Problem 5 (TP5—Interval Arithmetic Benchmark) [9]. This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} x_1 - 0.25428722 - 0.18324757\ x_4 x_3 x_9 = 0,\\ x_2 - 0.37842197 - 0.16275449\ x_1 x_{10} x_6 = 0,\\ x_3 - 0.27162577 - 0.16955071\ x_1 x_2 x_{10} = 0,\\ x_4 - 0.19807914 - 0.15585316\ x_7 x_1 x_6 = 0,\\ x_5 - 0.44166728 - 0.19950920\ x_7 x_6 x_3 = 0,\\ x_6 - 0.14654113 - 0.18922793\ x_8 x_5 x_{10} = 0,\\ x_7 - 0.42937161 - 0.21180486\ x_2 x_5 x_8 = 0,\\ x_8 - 0.07056438 - 0.17081208\ x_1 x_7 x_6 = 0,\\ x_9 - 0.34504906 - 0.19612740\ x_{10} x_6 x_8 = 0,\\ x_{10} - 0.42651102 - 0.21466544\ x_4 x_8 x_1 =0.\\ \end{array} \right. \end{aligned}$$

(27)

The resulting objective function defined by Eq. (26), is $10$-dimensional with global minimum $f(x^*) = 0$.

Test Problem 6 (TP6—Neurophysiology Application) [9] This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} x_1^2 + x_3^2 = 1,\\ x_2^2 + x_4^2 = 1,\\ x_5 x_3^3 + x_6 x_4^3 = c_1,\\ x_5 x_1^3 + x_6 x_2^3 = c_2,\\ x_5 x_1 x_3^2 + x_6 x_4^2 x_2 = c_3,\\ x_5 x_1^2 x_3 + x_6 x_2^2 x_4 = c_4,\\ \end{array} \right. \end{aligned}$$

(28)

where the constants, $c_i = 0$, $i=1,2,3,4$. The resulting objective function is $6$-dimensional with global minimum $f(x^*) = 0$.

Test Problem 7 (TP7—Chemical Equilibrium Application) [9] This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} x_1 x_2 + x_1 - 3 x_5 = 0,\\ 2 x_1 x_2 + x_1 + x_2 x_3^2 + R_8 x_2 - R x_5 + 2 R_{10} x_2^2 + R_7 x_2 x_3 + \\ R9 x_2 x_4 = 0,\\ 2 x_2 x_3^2 + 2 R_5 x_3^2 - 8 x_5 + R_6 x_3 + R_7 x_2 x_3 = 0,\\ R_9 x_2 x_4 + 2 x_4^2 - 4 R x_5 = 0, \\ x_1(x_2 + 1) + R_{10} x_2^2 + x_2 x_3^2 + R_8 x_2 + R_5 x_3^2 + x_4^2 - 1 \\ + R_6 x_3 + R_7 x_2 x_3 + R_9 x_2 x_4 = 0, \\ \end{array} \right. \end{aligned}$$

(29)

where

$$\begin{aligned} R = 10, \quad R_5 = 0.193, \quad R_6 = \frac{0.002597}{\sqrt{40}}, \quad R_7 = \frac{0.003448}{\sqrt{40}}, \end{aligned}$$

$$\begin{aligned} R_8 = \frac{0.00001799}{40}, \quad R_9 = \frac{0.0002155}{\sqrt{40}}, \quad R_{10} = \frac{0.00003846}{40}. \end{aligned}$$

The corresponding objective function is $5$-dimensional with global minimum $f(x^*) = 0$.

Test Problem 8 (TP8—Kinematic Application) [9] This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} x_i^2 + x_{i+1}^2 -1 = 0,\\ a_{1i} x_1 x_3 + a_{2i} x_1 x_4 + a_{3i} x_2 x_3 + a_{4i} x_2 x_4 + a_{5i} x_2 x_7 + \\ a_{6i} x_5 x_8 + a_{7i} x_6 x_7 + a_{8i} x_6 x_8 + a_{9i} x_1 + a_{10i} x_2 + a_{11i} x_3 + \\ a_{12 i} x_4 + a_{13 i} x_5 + a_{14 i} x_6 + a_{15 i} x_7 + a_{16 i} x_8 + a_{17 i} = 0,\\ \end{array} \right. \end{aligned}$$

(30)

with $a_{ki}$, $1 \leqslant k \leqslant 17$, $1 \leqslant i \leqslant 4$, is the corresponding element of the $k$-th row and $i$-th column of the matrix:

$$\begin{aligned} A = \left[ \begin{array}{llll} -0.249150680&{} 0.125016350&{} -0.635550077&{} 1.48947730 \\ 1.609135400&{} -0.686607360&{} -0.115719920&{} 0.23062341 \\ 0.279423430&{} -0.119228120&{} -0.666404480&{} 1.32810730 \\ 1.434801600&{} -0.719940470&{} 0.110362110&{} -0.25864503 \\ 0.000000000&{} -0.432419270&{} 0.290702030&{} 1.16517200 \\ 0.400263840&{} 0.000000000&{} 1.258776700&{} -0.26908494 \\ -0.800527680&{} 0.000000000&{} -0.629388360&{} 0.53816987 \\ 0.000000000&{} -0.864838550&{} 0.581404060&{} 0.58258598 \\ 0.074052388&{} -0.037157270&{} 0.195946620&{} -0.20816985 \\ -0.083050031&{} 0.035436896&{} -1.228034200&{} 2.68683200 \\ -0.386159610&{} 0.085383482&{} 0.000000000&{} -0.69910317 \\ -0.755266030&{} 0.000000000&{} -0.079034221&{} 0.35744413 \\ 0.504201680&{} -0.039251967&{} 0.026387877&{} 1.24991170 \\ -1.091628700&{} 0.000000000&{} -0.057131430&{} 1.46773600 \\ 0.000000000&{} -0.432419270&{} -1.162808100&{} 1.16517200 \\ 0.049207290&{} 0.000000000&{} 1.258776700&{} 1.07633970 \\ 0.049207290&{} 0.013873010&{} 2.162575000&{} -0.69686809 \\ \end{array} \right] \end{aligned}$$

The corresponding objective function is $8$-dimensional with global minimum $f(x^*) = 0$.

Test Problem 9 (TP9—Combustion Application) [9] This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} x_2 + 2 x_6 + x_9 + 2 x_{10} = 10^{-5}, \\ x_3 + x_8 = 3 \times 10^{-5},\\ x_1 + x_3 + 2 x_5 + 2 x_8 + x_9 + x_{10} = 5 \times 10^{-5},\\ x_4 + 2 x_7 = 10^{-5},\\ 0.5140437 \times 10^{-7} x_5 = x_1^2,\\ 0.1006932 \times 10^{-6} x_6 {}={} 2 x_2^2,\\ 0.7816278 \times 10^{-15} x_7 = x_4^2,\\ 0.1496236 \times 10^{-6} x_8 = x_1 x_3,\\ 0.6194411 \times 10^{-7} x_9 = x_1 x_2,\\ 0.2089296 \times 10^{-14} x_{10} = x_1 x_2^2.\\ \end{array} \right. \end{aligned}$$

(31)

The corresponding objective function is $10$-dimensional with global minimum $f(x^*) = 0$.

Test Problem 10 (TP10—Economics Modeling Application) [9] This problem consists of the following system:

$$\begin{aligned} \left\{ \begin{array}{l} \displaystyle \left( x_k + \sum _{i=1}^{n-k-1} x_i x_{i+k} \right) x_n - c_k = 0,\\ \displaystyle \sum _{l=1}^{n-1} x_l + 1 = 0, \\ \end{array} \right. \end{aligned}$$

(32)

where $1 \leqslant k \leqslant n-1$, and $c_i = 0$, $i=1,2,\ldots ,n$. The problem was considered in its $20$-dimensional instance. Thus, the corresponding objective function was also $20$-dimensional, with global minimum $f(x^*) = 0$.