Skip to main content
SpringerLink
Log in

Learning-enhanced differential evolution for numerical optimization

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Differential evolution (DE) is a simple and powerful population-based search algorithm, successfully used in various scientific and engineering fields. However, DE is not free from the problems of stagnation and premature convergence. Hence, designing more effective search strategies to enhance the performance of DE is one of the most salient and active topics. This paper proposes a new method, called learning-enhanced DE (LeDE) that promotes individuals to exchange information systematically. Distinct from the existing DE variants, LeDE adopts a novel learning strategy, namely clustering-based learning strategy (CLS). In CLS, there are two levels of learning strategies, intra-cluster learning strategy and inter-cluster learning strategy. They are adopted for exchanging information within the same cluster and between different clusters, respectively. Experimental studies over 23 benchmark functions show that LeDE significantly outperforms the conventional DE. Compared with other clustering-based DE algorithms, LeDE can obtain better solutions. In addition, LeDE is also shown to be significantly better than or at least comparable to several state-of-art DE variants as well as some other evolutionary algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The upper bound of the number of clusters is set to \(\sqrt{{\rm NP}}, \) which is a rule of thumb used in many researches (Sheng et al. 2005).

  2. Since the CEC2005 test functions are defined up to D = 50 in Suganthan et al. (2005), they are tested only at D = 30 and 50 in this paper.

References

  • Alessandro P, Antonina S (2008) Particle swarm optimization for multimodal functions: a clustering approach. J Artif Evol Appl 2008:1–15

  • Brest J, Greiner S, Boskovic B, Mernik M, Zumer V (2006) Self-adapting control parameters in differential evolution: a comparative study on numerical benchmark problems. IEEE Trans Evol Comput 10(6):646–657

    Article  Google Scholar 

  • Cai Z, Gong W, Ling C, Zhang H (2011) A clustering-based differential evolution for global optimization. Appl Soft Comput 11(1):1363–1379

    Article  Google Scholar 

  • Damavandi N, Safavi-Naeini S (2005) A hybrid evolutionary programming method for circuit optimization. IEEE Trans Circuits Syst I Regul Pap 52(5):902–910

    Article  MathSciNet  Google Scholar 

  • Das S, Abraham A, Chakraborty U, Konar A (2009) Differential evolution using a neighborhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553

    Article  Google Scholar 

  • Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15(1):4–13

    Article  Google Scholar 

  • Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18

    Article  Google Scholar 

  • Emmendorfer L, Pozo A (2009) Effective linkage learning using low-order statistics and clustering. IEEE Trans Evol Comput 13(6):1233–1246

    Article  Google Scholar 

  • Fan H, Lampinen J (2003) A trigonometric mutation operation to differential evolution. J Glob Optim 27 (1):105–129

    Article  MATH  MathSciNet  Google Scholar 

  • García S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694

    MATH  Google Scholar 

  • García S, Fernández A, Luengo J, Herrera F (2009a) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput Fusion Found Methodol Appl 13 (10):959–977

    Google Scholar 

  • García S, Molina D, Lozano M, Herrera F (2009b) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’ 2005 special session on real parameter optimization. J Heuristics 15(6):617–644

    Article  MATH  Google Scholar 

  • García S, Fernández A, Luengo J, Herrera F (2010) Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power. Inf Sci 180 (10):2044–2064

    Article  Google Scholar 

  • Ilonen J, Kamarainen J, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17(1):93–105

    Article  Google Scholar 

  • Iorio A, Li X (2011) Improving the performance and scalability of differential evolution on problems exhibiting parameter interactions. Soft Comput Fusion Found Methodol Appl. doi:10.1007/s00500-010-0614-y

  • Jain A, Murty M, Flynn P (1999) Data clustering: a review. ACM Comput Surv (CSUR) 31(3):264–323

    Article  Google Scholar 

  • Joshi R, Sanderson A (1999) Minimal representation multisensor fusion using differential evolution. IEEE Trans Syst Man Cybern Part A Syst Hum 29(1):63–76

    Article  Google Scholar 

  • Kennedy J (2000) Stereotyping: improving particle swarm performance with cluster analysis. In: Proceedings of the IEEE congress on evolutionary computation, California, USA, pp 1507–1512

  • Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks (ICNN’95), pp 1942–1948

  • Lee C, Yao X (2004) Evolutionary programming using mutations based on the Lévy probability distribution. IEEE Trans Evol Comput 8(1):1–13

    Article  Google Scholar 

  • Leung Y, Wang Y (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Transa Evol Comput 5(1):41–53

    Article  Google Scholar 

  • Li M, Kou J (2008) Crowding with nearest neighbors replacement for multiple species niching and building blocks preservation in binary multimodal functions optimization. J Heuristics 14 (3):243–270

    Article  MATH  Google Scholar 

  • Liu J, Lampinen J (2005) A fuzzy adaptive differential evolution algorithm. Soft Comput Fusion Found Methodol Appl 9(6):448–462

    MATH  Google Scholar 

  • Lu Q, Yao X (2005) Clustering and learning Gaussian distribution for continuous optimization. IEEE Trans Syst Man Cybern Part C Appl Rev 35(2):195–204

    Article  Google Scholar 

  • Neri F, Tirronen V (2009) Scale factor local search in differential evolution. Memet Comput 1(2):153–171

    Article  Google Scholar 

  • Neri F, Tirronen V (2010) Recent advances in differential evolution: a survey and experimental analysis. Artif Intell Rev 33(1):61–106

    Google Scholar 

  • Noman N, Iba H (2008) Accelerating differential evolution using an adaptive local search. IEEE Trans Evol Comput 12(1):107–125

    Article  Google Scholar 

  • Parrott D, Li X (2006) Locating and tracking multiple dynamic optima by a particle swarm model using speciation. IEEE Trans Evol Comput 10(4):440–458

    Article  Google Scholar 

  • Pelikan M, Goldberg D (2000) Genetic algorithms, clustering, and the breaking of symmetry. In: the 6th international conference on parallel problem solving from nature, 2000. Springer, Berlin, pp 385–394

  • Plagianakos V, Tasoulis D, Vrahatis M (2008) A review of major application areas of differential evolution. In: Advances in differential evolution, vol 143, 2008. Springer, Berlin, pp 197–238

  • Price K, Storn R, Lampinen J (2005) Differential evolution: a practical approach to global optimization. Springer, New York

    MATH  Google Scholar 

  • Qin AK, Huang VL, Suganthan PN (2009) Differential evolution algorithm with strategy adaptation for global numerical optimization. IEEE Trans Evol Comput 13 (2):398–417

    Article  Google Scholar 

  • Rahnamayan S, Tizhoosh HR, Salama MMA (2008) Opposition-based differential evolution. IEEE Trans Evol Comput 12(1):64–79

    Article  Google Scholar 

  • Ray T, Liew K (2003) Society and civilization: an optimization algorithm based on the simulation of social behavior. IEEE Trans Evol Comput 7(4):386–396

    Article  Google Scholar 

  • Sheng W, Swift S, Zhang L, Liu X (2005) A weighted sum validity function for clustering with a hybrid niching genetic algorithm. IEEE Trans Syst Man Cybern Part B Cybern 35(6):1156–1167

    Article  Google Scholar 

  • Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11(4):341–359

    Article  MATH  MathSciNet  Google Scholar 

  • Storn R, Price K (2010) Differential evolution homepage. http://www.icsi.berkeley.edu/∼storn/code.htm

  • Suganthan P, Hansen N, Liang J, Deb K, Chen Y, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nanyang Technol Universiy, Singapore, pp 1–50

  • Sun J, Zhang Q, Tsang EPK (2005) DE/EDA: a new evolutionary algorithm for global optimization. Inf Sci 169 (3–4):249–262

    Article  MathSciNet  Google Scholar 

  • Tirronen V, Neri F, Kärkkäinen T, Majava K, Rossi T (2008) An enhanced memetic differential evolution in filter design for defect detection in paper production. Evol Comput 16 (4):529–555

    Article  Google Scholar 

  • Wang F, Jang H (2000) Parameter estimation of a bioreaction model by hybrid differential evolution. In: Proceedings of 2000 IEEE congress on evolutionary computation, 2000, pp 410–417

    Google Scholar 

  • Wang Y, Zhang J, Zhang G (2007) A dynamic clustering based differential evolution algorithm for global optimization. Eur J Oper Res 183(1):56–73

    Article  MATH  Google Scholar 

  • Wong K, Leung K, Wong M (2010) Effect of spatial locality on an evolutionary algorithm for multimodal optimization. In: Applications of evolutionary computation, vol 6024/2010. Springer, Berlin, pp 481–490

  • Wright A (1991) Genetic algorithms for real parameter optimization. In: Rawlins GJ (ed) Foundations of genetic algorithms, vol 1. Morgan Kaufmann, San Mateo, pp 205–218

  • Wu S, Chow T (2007) Self-organizing and self-evolving neurons: a new neural network for optimization. IEEE Trans Neural Netw 18(2):385–396

    Article  Google Scholar 

  • Yang S, Li C (2010) A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Trans Evol Comput 14(6):959–974

    Article  Google Scholar 

  • Yang Z, Yao X, He J (2008) Making a difference to differential evolution. In: Advances in metaheuristics for hard optimization. Springer, Berlin, pp 397–414

  • Yao X, Liu Y, Lin G (1999) Evolutionary programming made faster. IEEE Trans Evol Comput 3(2):82–102

    Article  Google Scholar 

  • Zhang J, Sanderson A (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958

    Article  Google Scholar 

  • Zhong W, Liu J, Xue M, Jiao L (2004) A multiagent genetic algorithm for global numerical optimization. IEEE Trans Syst Man Cybern Part B Cybern 34(2):1128–1141

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Dr. W. Gong, Prof. J. Brest and Prof. P.N. Suganthan for providing the source code of CDE_cai (Cai et al. 2011), jDE (Brest et al. 2006) and SaDE (Qin et al. 2009), respectively. This work was supported in part by the National Natural Science Foundation of China (60805026, 60905038, 61070076, 61033010), and the Fundamental Research Funds for the Central Universities (10lgpy32).

Author information

Authors and Affiliations

  1. Department of Computer Science, Sun Yat-sen University, Guangzhou Higher Education Mega Center, Guangzhou, 510006, China

    Yiqiao Cai & Jian Yin

  2. Sun Yat-sen University, Guangzhou Higher Education Mega Center, Guangzhou, 510006, China

    Jiahai Wang

Authors
  1. Yiqiao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiahai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiahai Wang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cai, Y., Wang, J. & Yin, J. Learning-enhanced differential evolution for numerical optimization. Soft Comput 16, 303–330 (2012). https://doi.org/10.1007/s00500-011-0744-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-011-0744-x

Keywords

Search

Navigation