Abstract
Particle swarm optimization (PSO) is a metaheuristic commonly used for optimization problems. However, PSO does not scale well to large-scale optimization problems (LSOPs). A divide-and-conquer approach to PSO has shown to be effective when scaling up to LSOPs. Two cooperative PSO (CPSO) approaches, decomposition CPSO (DCPSO) and merging CPSO (MCPSO), were previously introduced, but are limited when it comes to exploring variable dependencies. An improvement to DCPSO and MCPSO incorporating a random grouping of decision variables at a fixed rate, referred to as RG-DCPSO and RG-MCPSO, was introduced recently in order to better explore variable dependencies. This work introduces two additional approaches to incorporating the random grouping of decision variables, denoted by \(\hbox {RG-DCPSO}_{\hbox {cv}}\), \(\hbox {RG-DCPSO}_{\hbox {sp}}\), \(\hbox {RG-MCPSO}_{\hbox {cv}}\), and \(\hbox {RG-MCPSO}_{\hbox {sp}}\). The various random grouping approaches were compared to five other decomposition-based PSO approaches found in the literature to determine their relative performance. The \(\hbox {RG-DCPSO}_{\hbox {cv}}\) approach introduced in this paper has competitive performance in the CEC’2010 large-scale global optimization benchmark set on environments with up to 1000 decision variables. Furthermore, \(\hbox {RG-DCPSO}_{\hbox {cv}}\) had the best performance out of all tested approaches on the CEC’2013 large-scale global optimization benchmark set.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and material
The datasets generated during and/or analysed during the course of the study are available from the corresponding author on reasonable request.
Notes
nx is the number of decision variables
References
Alirezanejad, M., Enayatifar, R., Motameni, H., et al. (2021). GSA-LA: gravitational search algorithm based on learning automata. Journal of Experimental & Theoretical Artificial Intelligence, 33(1), 109–125. https://doi.org/10.1080/0952813X.2020.1725650
Barry, W. (2012). Generating Aesthetically Pleasing Images in a Virtual Environment using Particle Swarm Optimization. Master’s thesis, Brock University, http://hdl.handle.net/10464/4137
Clark, M., Ombuki-Berman, B., Aksamit, N., et al. (2022). Cooperative particle swarm optimization decomposition methods for large-scale optimization. In: IEEE Symposium Series on Computational Intelligence (IEEE SSCI 2022).
Cleghorn, C.W., & Engelbrecht, A.P. (2014). Particle Swarm Convergence: An Empirical Investigation. In: 2014 IEEE Congress on Evolutionary Computation (CEC). IEEE, pp 2524–2530, https://doi.org/10.1109/CEC.2014.6900439
Douglas, J. (2019). Effcient Merging and Decomposition Variants of Cooperative Particle Swarm Optimization for Large Scale Problems. Master’s thesis, Brock University, http://hdl.handle.net/10464/13876
Douglas, J., Engelbrecht, A.P., & Ombuki-Berman, B.M. (2018). Merging and Decomposition Variants of Cooperative Particle Swarm Optimization: New Algorithms for Large Scale Optimization Problems. In: Proceedings of the 2nd International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence. ACM, pp 70–77, https://doi.org/10.1145/3206185.3206199
Erwin, K., & Engelbrecht, A. (2020). Set-based particle swarm optimization for portfolio optimization. In: Proceedings of the 12th International Swarm Intelligence Conference (ANTS), Lecture Notes in Computer Science, vol 12421. Springer, p 333–339, https://doi.org/10.1007/978-3-030-60376-2_28
Hajihassani, M., Armaghani, D. J., & Kalatehjari, R. (2018). Applications of particle swarm optimization in geotechnical engineering: A comprehensive review. Geotechnical and Geolocigal Engineering, 36, 705–722. https://doi.org/10.1007/s10706-017-0356-z
Helwig, S., & Wanka, R. (2008). Theoretical Analysis of Initial Particle Swarm Behavior. In: Parallel Problem Solving from Nature - PPSN X, pp 889–898, https://doi.org/10.1007/978-3-540-87700-4_88
Hereford, J.M. (2006). A Distributed Particle Swarm Optimization Algorithm for Swarm Robotic Applications. In: IEEE International Congress on Evolutionary Computation. IEEE, pp 1678–1685, https://doi.org/10.1109/CEC.2006.1688510
Kennedy, J., & Eberhart, R. (1995). Particle Swarm Optimization. In: Proceedings of International Conference on Neural Networks, pp 1942–1948, https://doi.org/10.1109/ICNN.1995.488968
Khare, A., & Rangnekar, S. (2013). A review of particle swarm optimization and its applications in solar photovoltaic system. Applied Soft Computing, 13(5), 2997–3006. https://doi.org/10.1016/j.asoc.2012.11.033
Komarudin, & Chandra, A. (2019). Optimization of Very Large Scale Capacitated Vehicle Routing Problems. In: Proceedings of the 2019 5th International Conference on Industrial and Business Engineering, pp 18–22, https://doi.org/10.1145/3364335.3364389
Kruskal, W. H., & Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260), 583–621. https://doi.org/10.1080/01621459.1952.10483441
Li, X., & Yao, X. (2012). Cooperatively coevolving particle swarms for large scale optimization. IEEE Transactions on Evolutionary Computation, 16(2), 210–224. https://doi.org/10.1109/TEVC.2011.2112662
Li, X., Tang, K., Omidvar, M.N. et al. (2013). Benchmark Functions for the CEC’2013 Special Session and Competition on Large-Scale Global Optimization.
Liu, B.n., Zhang, W.g., & Nie, R. (2012). An Improved Cooperative PSO Algorithm and its Application in the Flight Control System. In: International Conference on Automatic Control and Artificial Intelligence (ACAI 2012), pp 424–428, https://doi.org/10.1049/cp.2012.1007
Liu, L., Wang, Y., Xie, F., et al. (2018). Legendre cooperative PSO strategies for trajectory optimization. Complexity, 2018, 1–13. https://doi.org/10.1155/2018/5036791
Liu, Z., Shi, X., He, L., et al. (2020). A parameter-level parallel optimization algorithm for large-scale spatio-temporal data mining. Distributed and Parallel Databases, 38(3), 739–765. https://doi.org/10.1007/s10619-020-07287-x
Luna, F., Estébanez, C., León, C., et al. (2011). Optimization algorithms for large-scale real-world instances of the frequency assignment problem. Soft Computing, 15(5), 975–990. https://doi.org/10.1007/s00500-010-0653-4
Ma, K., Hu, S., Yang, J., et al. (2018). Appliances scheduling via cooperative multi-swarm PSO under day-ahead prices and photovoltaic generation. Applied Soft Computing, 62, 504–513. https://doi.org/10.1016/j.asoc.2017.09.021
Mann, H. B., & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50–60. https://doi.org/10.1214/aoms/1177730491
McNulty, A., Ombuki-Berman, B., & Engelbrecht, A. (2022). Decomposition and Merging Co-operative Particle Swarm Optimization with Random Grouping. In: Swarm Intelligence, pp 117–129, https://doi.org/10.1007/978-3-031-20176-9_10
Neethling, M., & Engelbrecht, A. (2006). Determining RNA Secondary Structure Using Set-Based Particle Swarm Optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp 1670–1677, https://doi.org/10.1109/CEC.2006.1688509
Oldewage, E.T. (2017). The Perils of Particle Swarm Optimization in High Dimensional Problem Spaces. Master’s thesis, University of Pretoria
Oldewage, E.T., Engelbrecht, A.P., Cleghorn, C.W. (2017). The Merits of Velocity Clamping Particle Swarm Optimisation in High Dimensional Spaces. In: Proceedings of the IEEE Symposium Series on Computational Intelligence, https://doi.org/10.1109/SSCI.2017.8280887
Oldewage, E.T., Engelbrecht, A.P., & Cleghorn, C.W. (2018). Boundary Constraint Handling Techniques for Particle Swarm Optimization in High Dimensional Problem Spaces. In: Swarm Intelligence. Cham: Springer International Publishing, p 333–341, https://doi.org/10.1007/978-3-030-00533-7_27
Oldewage, E. T., Engelbrecht, A. P., & Cleghorn, C. W. (2020). Movement patterns of a particle swarm in high dimensional spaces. Information Sciences, 512, 1043–1062. https://doi.org/10.1016/j.ins.2019.09.057
Omidvar, M.N., Li, X., Yang, Z., et al. (2010). Cooperative Co-evolution for Large Scale Optimization Through More Frequent Random Grouping. In: IEEE Congress on Evolutionary Computation, pp 1–8, https://doi.org/10.1109/CEC.2010.5586127
Pluhacek, M., Senkerik, R., Viktorin, A., et al. (2017). A Review of Real-World Applications of Particle Swarm Optimization Algorithm. In: Proceedings of the International Conference on Advanced Engineering Theory and Applications, pp 115–122, https://doi.org/10.1007/978-3-319-69814-4_11
Shi, Y., & Eberhart, R.C. (2005). Parameter Selection in Particle Swarm Optimization. In: Proceedings of Evolutionary Programming VII, pp 591–600, https://doi.org/10.1007/BFb0040810
Sopov, E., Vakhnin, A., Semenkin, E. (2018). On Tuning Group Sizes in the Random Adaptive Grouping Algorithm for Large-Scale Global Optimization Problems. In: Proceedings of the International Conference on Applied Mathematics Computational Science , pp 134–145, https://doi.org/10.1109/ICAMCS.NET46018.2018.00031
Sun, L., Yoshida, S., & Liang, Y. (2010). Cooperative Particle Swarm Optimization for Large Scale Numerical Optimization. SCIS & ISIS pp 892–897. https://doi.org/10.14864/softscis.2010.0.892.0
Sun, Y., Kirley, M., & Halgamuge, S. K. (2018). A recursive decomposition method for large scale continuous optimization. IEEE Transactions on Evolutionary Computation, 22(5), 647–661. https://doi.org/10.1109/TEVC.2017.2778089
Tang, K., Li, X., Suganthan, P.N., et al. (2010). Benchmark Functions for the CEC’2010 Special Session and Competition on Large-Scale Global Optimization
Van den Bergh, F., & Engelbrecht, A. P. (2004). A cooperative approach to particle swarm optimization. IEEE Transactions on Evolutionary Computation, 8(3), 225–239. https://doi.org/10.1109/TEVC.2004.826069
Van der Merwe, D., & Engelbrecht, A. (2003). Data Clustering Using Particle Swarm Optimization. In: Proceedings of IEEE Congress on Evolutionary Computation, pp 215–220, https://doi.org/10.1109/CEC.2003.1299577
Wang, Z., Wang, S., Yang, B., et al. (2021). A novel hybrid algorithm for large-scale composition optimization problems in cloud manufacturing. International Journal of Computer Integrated Manufacturing, 34(9), 898–919. https://doi.org/10.1080/0951192X.2021.1946852
Zhang, W., Ma, D., Wei, J., et al. (2014). A parameter selection strategy for particle swarm optimization based on particle positions. Expert Systems with Applications, 41(7), 3576–3584. https://doi.org/10.1016/j.eswa.2013.10.061
Acknowledgements
The authors would like to thank NSERC for funding this research.
Funding
This study was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) grant.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Alanna McNulty. The first draft of the manuscript was written by Alanna McNulty and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
Not applicable.
Ethics approval and Consent to participate
Not applicable.
Consent for publication
All authors agree with the content of this study and give their consent for it to be published.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix A
This appendix gives the best parameter values found for each random grouping condition. In the case of a tie, all parameter values are given.
“Sep.” refers to the separable functions, “Single-Group” refers to single-group partially separable functions, \(n_{x}/2m\) refers to \(n_x/2m\)-separable functions, \(n_{x}/m\) refers to \(n_x/m\)-separable functions, and “Non-Sep.” refers to non-separable functions. “Part. Additive Sep. Sub-component" refers to partially additively separable functions with a separable sub-component, “Part. Additive No Sep. Sub-component" refers to partially additively separable functions with no separable sub-component, and “Overlapping" refers to overlapping functions.
Appendix B
This appendix gives the rankings for all of the decomposition-based approaches. “Sep.” refers to the separable functions, “Single-Group” refers to single-group partially separable functions, \(n_{x}/2m\) refers to \(n_x/2m\)-separable functions, \(n_{x}/m\) refers to \(n_x/m\)-separable functions, and “Non-Sep.” refers to non-separable functions. “Part. Additive Sep. Sub-component" refers to partially additively separable functions with a separable sub-component, “Part. Additive No Sep. Sub-component" refers to partially additively separable functions with no separable sub-component, and “Overlapping" refers to overlapping functions.
“Total” sums up the points earned by each algorithm across all functions tested. The points earned by each algorithm are listed in the order “W/L/T \(\vert\) Rank”, where “W” indicates the number of wins, “L” indicates the number of losses, and “T” indicates the number of ties. “Rank” refers to the final ranking of each algorithm.
The ranks should be referred to column-wise to see which algorithm performed best for each benchmark type. The top-ranked algorithm has its rank indicated with a bold \({{\textbf {1}}}\) for easy identification.
Appendix C
This section gives graphs of the average final fitness values for each algorithm in each problem type. “Sep.” refers to the separable functions, “Single-Group” refers to single-group partially separable functions, \(n_{x}/2m\) refers to \(n_x/2m\)-separable functions, \(n_{x}/m\) refers to \(n_x/m\)-separable functions, and “Non-Sep.” refers to non-separable functions. “Part. Additive Sep. Sub-component" refers to partially additively separable functions with a separable sub-component, “Part. Additive No Sep. Sub-component" refers to partially additively separable functions with no separable sub-component, and “Overlapping" refers to overlapping functions (Figs. 3, 4, 5, 6, 7, 8).
The average final fitness values on the CEC 2010 benchmark functions with 30 dimensions
The average final fitness values on the CEC 2010 benchmark functions with 100 dimensions
The average final fitness values on the CEC 2010 benchmark functions with 500 dimensions
The average final fitness values on the CEC 2010 benchmark functions with 1000 dimensions
The average final fitness values on the CEC 2010 benchmark functions with 2000 dimensions
The average final fitness values on the CEC 2013 benchmark functions with 1000 dimensions
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
McNulty, A., Ombuki-Berman, B. & Engelbrecht, A. Decomposition and merging cooperative particle swarm optimization with random grouping for large-scale optimization problems. Swarm Intell 18, 141–166 (2024). https://doi.org/10.1007/s11721-023-00229-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11721-023-00229-0