Abstract
In this article, an Improved Particle Swarm Optimization (IPSO) is proposed for solving global optimization and hyperparameter optimization. This improvement is proposed to reduce the probability of particles falling into local optimum and alleviate premature convergence and the imbalance between the exploitation and exploration of the Particle Swarm Optimization (PSO). The IPSO benefits from a new search policy named group-based update policy. The initial population of IPSO is grouped by the k-means to form a multisubpopulation, which increases the intragroup learning mechanism of particles and effectively enhances the balance between the exploitation and exploration. The performance of IPSO is evaluated on six representative test functions and one engineering problem. In all experiments, IPSO is compared with PSO and one other state-of-the-art metaheuristics. The results are also analyzed qualitatively and quantitatively. The experimental results show that IPSO is very competitive and often better than other algorithms in the experiments. The results of IPSO on the hyperparameter optimization problem demonstrate its efficiency and robustness.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
Enquiries about data availability should be directed to the authors.
References
Aaha B, Sm C, Hf D, Ia D, Mm E, Hc F (2019) Harris hawks optimization: algorithm and applications. Futur Gener Comput Syst 97:849–872
Bagley J (1967) The behavior of adaptive systems which employ genetic and correlation algorithms: technical report. University of Michigan
Baruah SK, Cohen NK, Plaxton CG, Varvel DA (1996) Proportionate progress: a notion of fairness in resource allocation. Algorithmica 15(6):600–625
D’Angelo G, Castiglione A, Palmieri F (2021) A cluster-based multidimensional approach for detecting attacks on connected vehicles. IEEE Internet Things J 8(16):12518–12527
D'Angelo G, Rampone S (2016) Diagnosis of aerospace structure defects by a HPC implemented soft computing algorithm. Physics
Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Trans Syst Man Cybern Part B 26(1):29–41. https://doi.org/10.1109/3477.484436
Fujita Y, Izui K, Nishiwaki S, Zhang Z, Yin Y (2022) Production planning method for seru production systems under demand uncertainty. Comput Indus Eng 163:107856
Glover F (1986) Future paths for integer programming and links to artificial intelligence. Comput Oper Res 13(5):533–549
Glover F (1989) Tabu search—part i. Orsa J Comput 1(1):89–98
Gupta J, Nijhawan P, Ganguli S (2022) Parameter estimation of different solar cells using a novel swarm intelligence technique. Soft Comput 26(12):5833–5863
Hannan S, Gallagher MA, Perrin AM (2015) Military active and reserve component mix: the grey space. Interfaces 45(4):283–292. https://doi.org/10.1287/inte.2015.0795
Kalimeris D, Kaplun G, Singer Y (2019) Robust influence maximization for hyperparametric models. Statistics
Karaboga D, Basturk B (2007) Artificial bee colony (abc) optimization algorithm for solving constrained optimization problems. Springer, Berlin
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on neural networks (ICNN 95), 1995
Kilbridge MD, O’Block RP, Teplitz PV (1969) A conceptual framework for urban planning models. Manag Sci 15(6):B-246-B−266. https://doi.org/10.1287/mnsc.15.6.B246
Liu XZ (2017) Application of swarm intelligence algorithm in machine learning parameter optimization. Beijing University of Posts and telecommunications
Liu W, Wang Z, Zeng N, Yuan Y, Alsaadi FE, Liu X (2020) A novel randomised particle swarm optimizer. Int J Mach Learn Cybern 12(2):529–540. https://doi.org/10.1007/s13042-020-01186-4
Mahajan S, Abualigah L, Pandit AK, Nasar MA, Alkhazaleh HA, Altalhi M (2022) Fusion of modern meta-heuristic optimization methods using arithmetic optimization algorithm for global optimization tasks. Soft Comput. https://doi.org/10.1007/s00500-022-07079-8
Mohammad N, Hamed M, Emad E, Ali L, Mahdiyeh E, Baseem K (2022) Golden search optimization algorithm. IEEE Access 10:37515–37532
Nadimi-Shahraki MH, Taghian S, Mirjalili S (2021) An improved grey wolf optimizer for solving engineering problems. Expert Syst Appl 166:113917. https://doi.org/10.1016/j.eswa.2020.113917
Parouha RP, Verma P (2021) Design and applications of an advanced hybrid meta-heuristic algorithm for optimization problems. Artif Intell Rev
Ruz JJ, Arevalo O, Cruz JMDL, Pajares G (2020) Using MILP for UAVs trajectory optimization under radar detection risk. In: IEEE Conference on Emerging Technologies & Factory Automation. IEEE
Shami TM, El-Saleh AA, Alswaitti M, Al-Tashi Q, Summakieh MA, Mirjalili S (2022) Particle swarm optimization: a comprehensive survey. IEEE Access 10:10031–10061
Tang J, Liu G, Pan Q (2021) A review on representative swarm intelligence algorithms for solving optimization problems: applications and trends. IEEE/CAA J Autom Sinic 8(10):1627–1643. https://doi.org/10.1109/JAS.2021.1004129
Funding
This work was supported by National Natural Science Foundation of China (Grant number 62073330), and National Basic Research Fund (Grant number 2021XXJJ081).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by JZ, HL and QP. The first draft of the manuscript was written by JZ and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhan, J., Tang, J., Pan, Q. et al. Improved particle swarm optimization algorithm based on grouping and its application in hyperparameter optimization. Soft Comput 27, 8807–8819 (2023). https://doi.org/10.1007/s00500-023-08039-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-08039-6