Skip to main content
SpringerLink
Log in

Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The existence of the curse of dimensionality is well known, and its general effects are well acknowledged. However, and perhaps due to this colloquial understanding, specific measurements on the curse of dimensionality and its effects are not as extensive. In continuous domains, the volume of the search space grows exponentially with dimensionality. Conversely, the number of function evaluations budgeted to explore this search space usually grows only linearly. The divergence of these growth rates has important effects on the parameters used in particle swarm optimization and differential evolution as dimensionality increases. New experiments focus on the effects of population size and key changes to the search characteristics of these popular metaheuristics when population size is less than the dimensionality of the search space. Results show how design guidelines developed for low-dimensional implementations can become unsuitable for high-dimensional search spaces.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Kennedy J, Eberhart RC (1995) Particle swarm optimization. IEEE ICNN:1942–1948

  2. Storn R (1996) On the usage of differential evolution for function optimization. EEE NAFIPS:519–523

  3. Bellman RE (1957) Dynamic Programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  4. Bratton D, Kennedy J (2007) Defining a standard for particle swarm optimization. IEEE SIS:120–127

  5. Chu W, Gao X, Sorooshian S (2011) A new evolutionary search strategy for global optimization of high-dimensional problems. Inform Sci 181:4909–4927

    Article  Google Scholar 

  6. Grefenstette JJ (1986) Optimization of control parameters for genetic algorithms. IEEE SMC 16(1):122–128

    Google Scholar 

  7. Shi Y, Eberhart RC (1999) Empirical study of particle swarm optimization. IEEE CEC:1945–1950

  8. Zhang L-P, Yu H-J, Hu S-X (2005) Optimal choice of parameters for particle swarm optimization. J Zhejiang Univ Sci 2005 6A(6):528–534

    Article  Google Scholar 

  9. Mallipeddi R, Suganthan PN (2008) Empirical study on the effect of population size on differential evolution algorithm. IEEE CEC:3663–3670

  10. Dragoi E-N, Curteanu S, Leon F, Galaction A-I, Cascaval D (2011) Modeling of oxygen mass transfer in the presence of oxygen-vectors using neural networks developed by differential evolution algorithm. Eng Appl AI 24(7):1214–1226

    Google Scholar 

  11. Vafashoar R, Meybodi MR, Momeni Azandaryani AH (2012) CLA-DE: a hybrid model based on cellular learning automata for numerical optimization. Appl Intell 36(3):735–748

    Article  Google Scholar 

  12. Hansen N, Finck S, Ros R, Auger A (2009) Real-parameter black-box optimization benchmarking 2009: noiseless functions definitions, INRIA Technical Report RR-6829

  13. Liang JJ, Qu BY, Suganthan PN, Hernández-Díaz AG (2013) Problem definitions and evaluation criteria for the CEC 2013 special session on real-parameter optimization, Technical Report. Nanyang Technological University

  14. Engelbrecht A (2012) Particle swarm optimization: velocity initialization. IEEE CEC:70–77

  15. Helwig S, Branke J, Mostaghim S (2013) Experimental analysis of bound handling techniques in particle swarm optimization. IEEE TEC 17 (2):259–271

    Google Scholar 

  16. https://www.researchgate.net/publication/259643342_Source_code_for_an_implementation_of_Standard_Particle_Swarm_Optimization_--_revised?ev=prf_pub

  17. Storn R, Price K (1997) Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11:341–359

    Article  MATH  MathSciNet  Google Scholar 

  18. Montgomery J, Chen S (2010) An analysis of the operation of differential evolution at high and low crossover rates. IEEE CEC:881–888

  19. Montgomery J (2009 ) Differential evolution: Difference vectors and movement in solution space. IEEE CEC:2833–2840

  20. Clerc M (1999) The swarm and the queen: towards a deterministic and adaptive particle swarm optimization. IEEE CEC :1951–1957

  21. Jin’no K, Shindo T. (2010) Analysis of dynamical characteristic of canonical deterministic PSO. IEEE CEC:1105–1110

  22. Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313

    Article  MATH  Google Scholar 

  23. Singer S, Singer S (1999) Complexity analysis of Nelder-Mead search iterations. Proc Appl Math Comput:185–196

  24. Pham N, Wilamowski BM (2011) Improved Nelder Mead’s simplex method and applications. J Comput 3(3):55–63

    Google Scholar 

  25. Gao F., Han L (2012) Implementing the Nelder-Mead simplex algorithm with adaptive parameters. Comput Optim Appl 51(1):259–277

    Article  MATH  MathSciNet  Google Scholar 

  26. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671–680

    Article  MATH  MathSciNet  Google Scholar 

  27. Dueck G, Scheuer T (1990) Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. J Comp Phys 90(1):161–175

    Article  MATH  MathSciNet  Google Scholar 

  28. Talbi E-G (2009) Metaheuristics. Wiley Publishing, From Design to Implementation

    Book  MATH  Google Scholar 

  29. (2009) Particle swarm optimisation and high dimensional problem spaces. IEEE CEC:1988–1994

Download references

Author information

Authors and Affiliations

  1. School of Information Technology, York University, Toronto, Canada

    Stephen Chen

  2. School of Engineering and ICT, University of Tasmania, Hobart, Australia

    James Montgomery

  3. School of Mathematics and Computer Science, University of Havana, Havana, Cuba

    Antonio Bolufé-Röhler

Authors
  1. Stephen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James Montgomery
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Bolufé-Röhler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Montgomery, J. & Bolufé-Röhler, A. Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution. Appl Intell 42, 514–526 (2015). https://doi.org/10.1007/s10489-014-0613-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-014-0613-2

Keywords

Search

Navigation