Abstract
As an efficient search technique based on population, particle swarm optimizer (PSO) has been widely used to deal with practical optimization problems in different fields. To improve the generalization ability and accuracy of PSO, this paper proposes an automatic PSO based on reinforcement learning (RLAPSO). In RLAPSO, reinforcement learning is introduced to conduct the global search. By designing state, action, 3-dimensional Q table, and reward function to determine the generation strategy that is more suitable for the current process characteristics. Meanwhile, the parameters of optimizers are adjusted linearly in the process of optimization. To avoid prematurity, the global search in the later stage of the search is transformed into a local one to find a fine solution. Finally, the performance of RLAPSO is tested on five notable benchmark functions. The experimental results show that RLAPSO is competitive with the state-of-the-art PSO variants.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Park, H.J., Cho, S.W., Lee, C.: Particle swarm optimization algorithm with time buffer insertion for robust berth scheduling. Comput. Ind. Eng. 160, 107585 (2021)
Song, B., Wang, Z., Zou, L.: An improved PSO algorithm for smooth path planning of mobile robots using continuous high-degree Bezier curve. Appl. Soft Comput. 100(1), 106960 (2021)
Yang, Y., Pedersen, J.: A comparative study on feature selection in text categorization. In: Proceedings of International Conference on Machine Learning, pp. 412–420. Morgan Kaufmann Publishers (1997)
Chen, W., Zhang, J., Lin, Y., Chen, E.: Particle swarm optimization with an aging leader and challengers. IEEE Trans. Evol. Comput. 17(2), 241–258 (2013)
Cheng, R., Jin, Y.: A competitive swarm optimizer for large scale optimization. IEEE Trans. Cybern. 45(2), 191–204 (2015)
Liu, Y., Lu, H., Cheng, S., et al.: An adaptive online parameter control algorithm for particle swarm optimization based on reinforcement learning. In: IEEE Congress on Evolutionary Computation (CEC), pp. 815–822 (2019)
Lu, H., Liu, Y., Cheng, S., et al.: Adaptive online data-driven closed-loop parameter control strategy for swarm intelligence algorithm. Inf. Sci. 536, 25–52 (2020)
Cheng, R., Jin, Y.: A social learning particle swarm optimization algorithm for scalable optimization. Inf. Sci. 291, 43–60 (2015)
Lu, L., Zheng, H., Jie, J., Zhang, M., Dai, R.: Reinforcement learning-based particle swarm optimization for sewage treatment control. Complex Intell. Syst. 7(5), 2199–2210 (2021). https://doi.org/10.1007/s40747-021-00395-w
Liang, J., Suganthan, P.: Dynamic multi-swarm particle swarm optimizer. In: Proceedings of IEEE Swarm Intelligence Symposium, pp. 124–129. IEEE (2005)
Zhao, X., Liu, Z., Yang, X.: A multi-swarm cooperative multistage perturbation guiding particle swarm optimizer. Appl. Soft Comput. 22, 77–93 (2014)
Wang, S., Liu, G., et al.: Heterogeneous comprehensive learning and dynamic multi-swarm particle swarm optimizer with two mutation operators. Inf. Sci. 540, 175–201 (2020)
Wolpert, D., Macready, W.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Lynn, N., Suganthan, P.: Ensemble particle swarm optimizer. Appl. Soft Comput. 55, 533–548 (2017)
Olorunda, O., Engelbrecht, A.: An analysis of heterogeneous cooperative algorithms. In: IEEE Congress on Evolutionary Computation, pp. 1562–1569 (2009)
Potter, M.: The Design and Analysis of a Computational Model of Cooperative Coevolution. Citeseer (1997)
Liang, B., Zhao, Y., Li, Y.: A hybrid particle swarm optimization with crisscross learning strategy. Eng. Appl. Artif. Intell. 105, 104418 (2021)
Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micro Machine and Human Science, vol. 1, pp. 39–43 (1995)
Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 69–73 (1999)
Peram, T., Veeramachaneni, K., Mohan, C.: Fitness-distance-ratio based particle swarm optimization. In: Proceedings of the 2003 IEEE Swarm Intelligence Symposium, pp. 174–181 (2003)
Ratnaweera, A., Halgamuge, S., Watson, H.: Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans. Evol. Comput. 8, 240–255 (2004)
Qu, B., Suganthan, P., Das, S.: A distance-based locally informed particle swarm model for multimodal optimization. IEEE Trans. Evol. Comput. 17, 387–402 (2013)
Shahrabi, J., Adibi, M., Mahootchi, M.: A reinforcement learning approach to parameter estimation in dynamic job shop scheduling. Comput. Ind. Eng. 110, 75–82 (2017)
Huynh, T., Do, D., Lee, J.: Q-Learning-based parameter control in differential evolution for structural optimization. Appl. Soft Comput. 107(11), 107464 (2021)
Acknowledgment
This work was supported in part by Pinghu Science and Technology Plan Project (No. GY202112) and Zhejiang Public Welfare Technology Application Research Project (No. LGG19F030005).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Dai, R., Zheng, H., Jie, J., Wu, X. (2022). Automatic Particle Swarm Optimizer Based on Reinforcement Learning. In: Pan, L., Cui, Z., Cai, J., Li, L. (eds) Bio-Inspired Computing: Theories and Applications. BIC-TA 2021. Communications in Computer and Information Science, vol 1565. Springer, Singapore. https://doi.org/10.1007/978-981-19-1256-6_24
Download citation
DOI: https://doi.org/10.1007/978-981-19-1256-6_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-1255-9
Online ISBN: 978-981-19-1256-6
eBook Packages: Computer ScienceComputer Science (R0)