An efficient differential evolution using speeded-up k-nearest neighbor estimator

Abstract

In evolutionary algorithm, one of the main issues is how to reduce the number of fitness evaluations required to obtain optimal solutions. Generally a large number of evaluations are needed to find optimal solutions, which leads to an increase of computational time. Expensive cost may have to be paid for fitness evaluation as well. Differential evolution (DE), which is widely used in many applications due to its simplicity and good performance, also cannot escape from this problem. In order to solve this problem a fitness approximation model has been proposed so far, replacing real fitness function for evaluation. In fitness approximation, an ability to estimate accurate value with compact structure is needed for good performance. Therefore in this paper we propose an efficient differential evolution using fitness estimator. We choose k-nearest neighbor (kNN) as fitness estimator because it does not need any training period or complex computation. However too many training samples in the estimator may cause computational complexity to be exponentially high. Accordingly, two schemes with regard to accuracy and efficiency are proposed to improve the estimator. Our proposed algorithm is tested with various benchmark functions and shown to find good optimal solutions with less fitness evaluation and more compact size, compared with DE and DE-kNN.

Appendices

Appendix A: Benchmark functions

f ₁: Sphere model

$$ f_{1}(x) = \sum_{i=1}^N x_i^2, -100 \le x_i \le 100 $$

$$ {\rm min}(f_1) =f_1 (0,\ldots, 0)=0 $$

f ₂: Schwefel’s problem 2.22

$$ f_2 (x) = \sum_{i=1}^N | x_i |+\prod_{i=1}^N |x_i |,\quad -10\le x_i \le 10 $$

$$ {\rm min}(f_2) =f_2 (0,\ldots, 0)=0 $$

f ₃: Schwefel’s problem 1.2

$$ f_3 (x)= \sum_{i=1}^N \left( \sum_j^i x_j \right)^2 , \quad -100 \le x_i \le 100 $$

$$ {\rm min}(f_3) =f_3 (0,\ldots, 0)=0 $$

f ₄: Schwefel’s problem 2.21

$$ f_4 (x)= \max_{i} \{|x_i |, 1\le i\le 30 \},\quad -100 \le x_i \le 100 $$

$$ {\rm min}(f_4) =f_4 (0,\ldots, 0)=0 $$

f ₅: Generalized Rosenbrock’s function

$$ f_5 (x)= \sum_{i=1}^{N-1} [100(x_{i+1} -x_i^2 )^2 + (x_i -1)^2 ],\quad -30\le x_i \le 30 $$

$$ {\rm min}(f_5) =f_5 (1,\ldots, 1)=0 $$

f ₆: Step function

$$ f_6 (x)= \sum_{i=1}^N ( \lfloor x_i +0.5 \rfloor )^2 ,\quad -100 \le x_i \le 100 $$

$$ {\rm min}(f_6) =f_6 (0,\ldots, 0)=0 $$

f ₇: Quartic function i.e. noise

$$ f_7 (x)= \sum_{i=1}^N ix_i^4 + {\mathrm{random}}[0,1),\quad -1.28 \le\; {x_i} \;\le 1.28 $$

$$ {\rm min}(f_7) =f_7 (0,\ldots, 0)=0 $$

f ₈: Generalized Schwefel’s problem 2.26

$$ f_8 (x)= -\sum_{i=1}^N \left(x_i \sin(\sqrt{|x_i |}) \right) ,\quad -500 \le x_i \le 500 $$

$$ {\rm min}(f_8) =f_8 (420.9687,\ldots, 420.9687)=-418.983N $$

f ₉: Generalized Rastrigin’s function

$$ f_9 (x)= \sum_{i=1}^N [x_i^2 - 10\cos(2\pi x_i )+10] ,\quad -5.12 \le x_i \le 5.12 $$

$$ {\rm min}(f_9) =f_9 (0,\ldots, 0)=0 $$

f ₁₀: Ackley’s function

$$ \begin{aligned} f_{10} &= -20{\mathrm{exp}}\left( -0.2 \sqrt{ \frac{1}{N} \sum_{i=1}^N x_i^2} \right)\\ &\quad -{\mathrm{exp}}\left( \frac{1}{N} \sum_{i=1}^N \cos 2\pi x_i \right)+20+e\\ \end{aligned} $$

$$ -32 \le x_i \le 32,\,{\mathrm{min}} (f_{10}) =f_{10} (0,\ldots, 0)=0 $$

f ₁₁: Generalized Griewank function

$$ f_{11} (x)= \frac{1}{4,000} \sum_{i=1}^N x_i^2 - \prod_{i=1}^N \cos\left( \frac{x_i}{\sqrt{i}} \right) +1 $$

$$ -600\le x_i \le 600 ,\,{\mathrm{min}}(f_{11}) =f_{11} (0,\ldots, 0)=0 $$

f ₁₂ , f ₁₃: Generalized penalized functions

$$ \begin{aligned} f_{12} &= \frac{\pi}{N} \left\{ 10\sin^2 (\pi y_1 ) + \sum_{i=1}^{N-1} (y_i -1)^2 \right. \\ &\quad \left.\cdot [1+10\sin^{2} (\pi y_{i+1} )] + (y_N -1)^2 \right \} \\ & \quad+ \sum_{i=1}^N u(x_i ,10,100,4) \end{aligned} $$

$$ -50 \le x_i \le 50,\,{\mathrm{min}}(f_{12}) =f_{12} (-1,\ldots, -1)=0 $$

$$ \begin{aligned} f_{13} =\,& 0.1 \biggl\{ \sin^2 (3\pi x_1 ) + \sum_{i=1}^{N-1} (x_i -1)^2 [1+\sin^2 (3\pi x_{i+1})]\biggr.\\ & \qquad\biggl.+(x_N -1)^2 [1+\sin^2 (2\pi x_N )] \biggr\} \\ & + \sum_{i=1}^N u(x_i , 5,100,4) \\ \end{aligned} $$

$$ -50 \le x_i \le 50,\,{\mathrm{min}}(f_{13}) =f_{13} (1,\ldots, 1)=0 $$

where

$$ \begin{aligned} u(x_i , a,k,m) &= \left\{\begin{array}{ll} k (x_i -a)^m , & x_i >a \\ 0, & -a \le x_i \le a \\ k (-x_i -a)^m , & x_i < -a \end{array}\right. \\ y_i &= 1+ \frac{1}{4} (x_i +1) \\ \end{aligned} $$

f ₁₄: Six-hump camel-back function

$$ f_{14} = 4x_1^2 -2.1x_1^4 + \frac{1}{3} x_1^6 +x_1 x_2 -4x_2^2+4x_2^4 $$

$$ -5 \le x_i \le 5 $$

$$ x_{\mathrm{min}} = (0.08983,\,-0.7126),\,(-0.08983,\,0.7126) $$

$$ {\mathrm{min}}(f_{14} )=-1.0316285 $$

f ₁₅: Branin function

$$ f_{15} = \left( x_2 - \frac{5.1}{4\pi^2} x_1^2 + \frac{5}{\pi} x_1 -6 \right)^2 + 10\left(1- \frac{1}{8\pi}\right)\cos(x_1 ) + 10 $$

$$ -5\le x_1 \le 10,\,0\le x_2 \le 15 $$

$$ x_{\mathrm{min}} =(-3.142,\,12.275),\,(3.142,\,2.275),\,(9.425,\,2.425) $$

$$ {\mathrm{min}}(f_{15} )=0.398 $$

f ₁₆: Goldstein-Price function

$$ \begin{aligned} f_{16} =& [1+(x_1 +x_2 +1)^2 (19-14x_1 +3x_1^2 -14x_2 \\ & +6x_1 x_2 +3x_2^2)]\times [30+(2x_1 -3x_2)^2 \\ & \times (18-32x_1 +12x_1^2 + 48x_2 -36x_1 x_2 +27x_2^2 )] \\ \end{aligned} $$

$$ -2\le x_i \le 2,\,{\mathrm{min}}(f_{16} ) = f_{16} (0, -1)=3 $$

f ₁₇: Bukin function

$$ f_{17} = 100 \times \sqrt{|x_2 - 0.01 \times x_1^2 |} + 0.01 \times |x_1 + 10| $$

$$ -15 \le x_1 \le -5,\,-3\le x_2 \le 3 $$

$$ {\mathrm{min}}(f_{17})=f_{17} (-10,1)=0 .$$

Appendix B

See Tables 7, 8, 9 and 10.

Table 7 Mean and standard deviation of optimal value found by DE

Table 8 Mean and standard deviation of optimal value found by DE-EkNN, DE-kNN, DE-wkNNAll, and DE-okNNSel

Table 9 Mean and standard deviation of the number of fitness evaluations for DE-EkNN, DE-kNN, DE-wkNNAll, and DE-okNNSel

Table 10 Mean and standard deviation of the size of archive containing training samples for DE-EkNN, DE-kNN, DE-wkNNAll, and DE-okNNSel