Skip to main content
SpringerLink
Log in

Memory-based approaches for eliminating premature convergence in particle swarm optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Particle Swarm Optimization (PSO) is a computational method in which a group of particles moves in search space in search of an optimal solution. During this movement, each particle updates its position and velocity with its best previous position and best position found by the swarm. Though PSO is considered as a potential solution and applied in many areas, it suffers from premature convergence in which all the particles are converged too early, resulting in sub-optimal results. Although there are several techniques to address premature convergence, achieving a higher convergence rate while avoiding premature convergence is still challenging. In this paper, we present two new memory-based variants of PSO for preventing premature convergence. The first technique (PSOMR), augments memory by leveraging the concepts of the Ebbinghaus forgetting curve. The second technique (MS-PSOMR) divides swarm into multiple subswarms. Both techniques use memory to store promising historical values and use them later to avoid premature convergence. The proposed approaches are compared with existing algorithms belonging to a similar category and evaluations on CEC 2010 and CEC 2017 benchmark functions. The results show that both the approaches performed significantly better for the measured metrics and discouraged premature convergence.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. J Kennedy, R. Eberhart (1995) Particle swarm optimization. In Proc. of IEEE International Conference on Neural Networks: 1942–1948

  2. Pluhacek M., Senkerik R., Viktorin A., Kadavy T., Zelinka I. (2018) A review of real-world applications of particle swarm optimization algorithm. In: Duy V., Dao T., Zelinka I., Kim S., Phuong T. (eds) AETA 2017 - Recent Advances in Electrical Engineering and Related Sciences: Theory and Application. AETA 2017. Lecture notes in electrical engineering, vol 465. Springer, Cham.

  3. Wachowiak MP, Smoliková R, Zheng YF, Zurada JM, Elmaghraby AS (2004) An approach to multimodal biomedical image registration utilizing particle swarm optimization. IEEE Trans Evol Comput 8(3):289–301

    Article  Google Scholar 

  4. del Valle Y, Venayagamoorthy GK, Mohagheghi S, Hernandez JC, Harley RG (2008) Particle swarm optimization: basic concepts, variants and applications in power systems. IEEE Trans Evol Comput 12(2):171–195

    Article  Google Scholar 

  5. W Lin, X. Gu, Z Lian, Y Xu, B Jiao ( 2013) A self-government particle swarm optimization algorithm and its application.Texaco gasification. Journal of Software 8(2):472–479

  6. Chen WN, Zhang J, Lin Y, Chen N, Zhan ZH, Chung H, Li Y, Shi YH (2013) Particle swarm optimization with an aging leader and challengers. IEEE Trans Evol Comput 17(2):241–258

    Article  Google Scholar 

  7. Xu G (2013) An adaptive parameter tuning of particle swarm optimization algorithm. Appl Math Comput 219(9):4560–4569

    MathSciNet  MATH  Google Scholar 

  8. Rezaei F, Safavi HR (2020) GuASPSO: a new approach to hold a better exploration–exploitation balance in PSO algorithm. Soft Comput 24:4855–4875

    Article  Google Scholar 

  9. Nickabadi A, Ebadzadeh MM, Safabakhsh R (2011) A novel particle swarm optimization algorithm with adaptive inertia weight. Appl Soft Comput 11(4):3658–3670

    Article  Google Scholar 

  10. Ratnaweera A, Halgamuge S, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput 8(3):240–255

    Article  Google Scholar 

  11. Liu Y., Zhao Q., Shao Z., Shang Z., Sui C. (2009) Particle swarm optimizer based on dynamic neighborhood topology. In: Huang DS., Jo KH., Lee HH., Kang HJ., Bevilacqua V. (eds) Emerging Intelligent Computing Technology and Applications. With Aspects of Artificial Intelligence. ICIC 2009. Lecture notes in computer science, vol 5755. Springer, Berlin, Heidelberg

  12. Liu Z, Li H, Zhu P (2019) Diversity enhanced particle swarm optimization algorithm and its application in vehicle lightweight design. J Mech Sci Technol 33:695–709

    Article  Google Scholar 

  13. Zhang, Y., Gong, D., Sun, X (2014) Adaptive bare-bones particle swarm optimization algorithm and its convergence analysis in Soft Computing 18:1337–1352

  14. R Tang, Y Fang (2015) Modification of particle swarm optimization with human simulated property in Neurocomputing 153: 319–331

  15. Zhang Z, Ding XM (2011) A multi-swarm self-adaptive and cooperative particle swarm optimization. Eng Appl Artif Intell 24(6):958–967

    Article  Google Scholar 

  16. Yen GG, Leong WF (2009) Dynamic multiple swarms in multiobjective particle swarm optimization. IEEE Trans Syst Man Cybern Syst Hum 39(4):890–911

    Article  Google Scholar 

  17. Xua X, Tang Y, Li J, Hua C, Guan X (2015) Dynamic multi-swarm particle swarm optimizer with cooperative learning strategy. Appl Soft Comput 29:169–183

    Article  Google Scholar 

  18. Zhao SZ, Suganthan PN, Pan QK, Fatih Tasgetiren M (2011) Dynamic multi-swarm particle swarm optimizer with harmony search. Exp Syst Appl 38(4):3735–3742

    Article  Google Scholar 

  19. van den Bergh F (2001) An analysis of particle swarm optimizers. University of Pretoria, Pretoria

    Google Scholar 

  20. van den Bergh F, Engelbrecht AP (2010) A convergence proof for the particle swarm Optimiser. Fundam Inf 105(4):341–374

    MathSciNet  MATH  Google Scholar 

  21. Hu X, Eberhart RC, Shi Y (2003) Particle swarm with extended memory for multiobjective optimization. In: Proceedings of the IEEE swarm intelligence symposium (SIS). Indianapolis, IN, USA, pp 193–197

    Google Scholar 

  22. Kudělka M, Horák Z, Snášel V, Krömer P, Platoš J, Abraham A (2012) Social and swarm aspects of co-authorship network. Logic Journal of IGPL Advance Access 20:634–643

    Article  MathSciNet  Google Scholar 

  23. Bennett AG, Rebello NS (2012) Retention and learning. In: Seel NM (ed) Encyclopedia of the sciences of learning. Springer, Boston, MA

    Google Scholar 

  24. Bergh VF, Engelbrecht AP (2004) A cooperative approach to particle swarm optimization. IEEE Trans Evol Comput 8(3):225–239

    Article  Google Scholar 

  25. Mendes R, Kennedy J, Neves J (2004) The fully informed particle swarm: Simpler, maybe better. IEEE Transactions on Evolutionary Computation 8(3):204–210

    Article  Google Scholar 

  26. Liang J, Qin AK, Suganthan PN, Baskar S (2006) Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Trans Evol Comput 10(3):281–295

    Article  Google Scholar 

  27. Huang H, Lv L, Ye S (2019) Particle swarm optimization with convergence speed controller for large-scale numerical optimization. Soft Comput 23:4421–4437

    Article  Google Scholar 

  28. Li Y, Gui W, Yang C (2005) Improved PSO algorithm and its application. Journal of the Central South University of Technology 12:222–226

    Article  Google Scholar 

  29. Arani BO, Mirzabeygi P, Panahi MS (2013) An improved PSO algorithm with a territorial diversity-preserving scheme and enhanced exploration-exploitation balance. In Swarm and Evolutionary Computation 11:1–15

    Article  Google Scholar 

  30. C Coello, M Lechuga (2002) MOPSO: A proposal for multiple objective particle swarm optimization. In IEEE Congress on Evolutionary Computation. (CEC) IEEE Computer Society Washington, DC, USA : 1051–1056

  31. H Wang, D Wang, S Yang (2007) Triggered memory-based swarm optimization in dynamic environments. Applications of Evolutionary Computing. EvoWorkshops: 637–646

  32. Acan A, Gunay A (2005) Enhanced particle swarm optimization through external memory support. In: IEEE congress on evolutionary computation. Vancouver, Canada, pp 1875–1882

    Google Scholar 

  33. Acan, A Unveren (2009) A memory-based colonization scheme for particle swarm optimization. In IEEE Congress on Evolutionary Computation (CEC), Piscataway, NJ:1965–1972

  34. Shahriar Asta, A sima Uyar (2011) A novel particle swarm optimization algorithm. 10th international conference on Artificial Evolution

  35. Li J, Zhang J, Jiang C, Zhou M (2015) Composite particle swarm optimizer, with historical memory for function optimization. IEEE Transactions on Cybernetics 45(10):2168–2267

    Google Scholar 

  36. Acan A, Ünveren A A two-stage memory powered Great Deluge algorithm for global optimization. Soft Computing 19(9):2565–2585

  37. Li W (2018) Improving particle swarm optimization based on neighborhood and historical memory for training multi-layer perceptron. Information 9(16)

  38. Broderick I, Howley E (2014) Particle swarm optimisation with enhanced memory particles. In: Dorigo M. et al. (eds) Swarm Intelligence. ANTS 2014. Lecture notes in computer science, 8667. Springer, Cham

  39. S. Z. Zhao, J. J. Liang, P. N. Suganthan and M. F. Tasgetiren (2008) Dynamic multi-swarm particle swarm optimizer with local search for Large Scale Global Optimization. IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence), Hong Kong: 3845–3852,

  40. Dongping Tian, Zhongzhi Shi. MPSO (2018) Modified particle swarm optimization and its applications. Swarm and Evolutionary Computation. (41): 49–68

  41. Lynn N, Suganthan PN (2015) Heterogeneous comprehensive learning particle swarm optimization with enhanced exploration and exploitation. Swarm and Evolutionary Computation 24:11–24

    Article  Google Scholar 

  42. Nandar Lynn Ponnuthurai Nagaratnam Suganthan (2017) Ensemble particle swarm optimizer. Appl Soft Comput 55:533–548

    Article  Google Scholar 

  43. Song X, Zhang Y, Guo Y, Sun X, Wang Y (2020) Variable-size cooperative Coevolutionary particle swarm optimization for feature selection on high-dimensional data. IEEE Trans Evol Comput 24(5):882–895

    Article  Google Scholar 

  44. Zhang Y, Li H, Wang Q et al (2019) A filter-based bare-bone particle swarm optimization algorithm for unsupervised feature selection. Appl Intell 49:2889–2898

    Article  Google Scholar 

  45. Xia X, Tang Y, Wei B (2020) Dynamic multi-swarm global particle swarm optimization. Computing 102:1587–1626

    Article  MathSciNet  Google Scholar 

  46. Piotrowski AP, Napiorkowski JJ, Piotrowska AE (2020) Population size in particle swarm optimization. Swarm and Evolutionary Computation 58:1–18

    Article  Google Scholar 

  47. K. Tang, X.D. Li, P.N. Suganthan, Z.Y. Yang, T. Weise, Benchmark functions for the CEC'2010 special session and competition on large-scale global optimization, in Proceedings of the Nature Inspired Computation and Applications Laboratory,

  48. Wu G, Mallipeddi R, Suganthan PN (2016) Problem definitions and evaluation criteria for the CEC 2017 competition and special session on constrained single objective real-parameter optimization. Nanyang Technological University, Singapore, Technical Report

    Google Scholar 

  49. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  50. Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11:86–92

    Article  MathSciNet  Google Scholar 

  51. Iman R, Davenport J (1980) Approximations of the critical region of the Friedman statistic. Communications in Statistics 9:571–595

    Article  Google Scholar 

  52. Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32:674–701

    Article  Google Scholar 

  53. Quade D (1979) Using weighted rankings in the analysis of complete blocks with additive block effects. J Am Stat Assoc 74:680–683

    Article  MathSciNet  Google Scholar 

  54. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of non-parametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation 1:3–18

    Article  Google Scholar 

  55. Tangherloni A, Rundo L, Nobile MS (2017) Proactive particles in swarm optimization: a settings-free algorithm for real-parameter single objective optimization problems. Proc IEEE Congr Evol Comput:1940–1947

Download references

Author information

Authors and Affiliations

  1. Infosys Limited, Hyderabad, Telangana, India

    K. Chaitanya

  2. National Institute of Technology, Warangal, Telangana, India

    D. V. L. N Somayajulu & P. Radha Krishna

Authors
  1. K. Chaitanya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. D. V. L. N Somayajulu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. Radha Krishna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. Chaitanya.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chaitanya, K., Somayajulu, D.V.L.N. & Krishna, P.R. Memory-based approaches for eliminating premature convergence in particle swarm optimization. Appl Intell 51, 4575–4608 (2021). https://doi.org/10.1007/s10489-020-02045-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02045-z

Keywords

Search

Navigation