Abstract
An online learning differential evolution algorithm (OLDE) integrated with deep reinforcement learning is proposed to solve complex optimization problems. First, a neural network model maintained by a double deep Q network algorithm is introduced to select the proper parameter adaptation method and control the mutation and crossover of the population. The history information generated by the search process is collected as the training data of the model. The adaptive ability of OLDE is enhanced due to the online learning method. Second, a long-term strategy is proposed to reduce computational complexity and boost learning efficiency. Finally, an adaptive optimization operator is designed to select a suitable mutation strategy for the different search processes. The experimental results reveal that the proposed algorithm is superior to comparison algorithms on CEC 2017 real-parameter numerical optimization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yi, W., Chen, Y., Pei, Z., Lu, J.: Adaptive differential evolution with ensembling operators for continuous optimization problems. Swarm Evol. Comput. 69, 100994 (2022)
Ahmad, M.F., Isa, N.A.M., Lim, W.H., Ang, K.M.: Differential evolution: a recent review based on state-of-the-art works. Alex. Eng. J. 61, 3831–3872 (2022)
Stanovov, V., Akhmedova, S., Semenkin, E.: The automatic design of parameter adaptation techniques for differential evolution with genetic programming. Knowl.-Based Syst. 239, 108070 (2022)
Deng, W., Ni, H., Liu, Y., Chen, H., Zhao, H.: An adaptive differential evolution algorithm based on belief space and generalized opposition-based learning for resource allocation. Appl. Soft Comput. 127, 109419 (2022)
Sun, J., Liu, X., Bäck, T., Xu, Z.: Learning adaptive differential evolution algorithm from optimization experiences by policy gradient. IEEE Trans. Evol. Comput. 25, 666–680 (2021)
Jingqiao, Z., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13, 945–958 (2009)
Tanabe, R., Fukunaga, A.: Reviewing and benchmarking parameter control methods in differential evolution. IEEE Trans. Cybern. 50, 1170–1184 (2019)
Wang, L., Pan, Z., Wang, J.: A review of reinforcement learning based intelligent optimization for manufacturing scheduling. Complex Syst. Model. Simul. 1, 257–270 (2021)
Zhang, Z., Shao, Z., Shao, W., Chen, J., Pi, D.: MRLM: A meta-reinforcement learning-based metaheuristic for hybrid flow-shop scheduling problem with learning and forgetting effects. Swarm Evol. Comput. 85, 101479 (2024)
Zhao, F., Hu, X., Wang, L., Zhao, J., Tang, J., Jonrinaldi: A reinforcement learning brain storm optimization algorithm (BSO) with learning mechanism. Knowl.-Based Syst. 235, 107645 (2022)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence (Year)
Zhao, F., Zhou, G., Wang, L.: A cooperative scatter search with reinforcement learning mechanism for the distributed permutation flowshop scheduling problem with sequence-dependent setup times. IEEE Trans. Syst. Man Cybern. Syst. 53, 4899–4911 (2023)
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)
Tanabe, R., Fukunaga, A.: Success-history based parameter adaptation for differential evolution. In: 2013 IEEE Congress on Evolutionary Computation, pp. 71–78. IEEE (2013)
Zhan, Z.H., Wang, Z.J., Jin, H., Zhang, J.: Adaptive distributed differential evolution. IEEE Trans. Cybern. 50, 4633–4647 (2019)
Ma, L., Cheng, S., Shi, Y.: Enhancing learning efficiency of brain storm optimization via orthogonal learning design. IEEE Trans. Syst. Man Cybern. Syst. 51, 6723–6742 (2020)
GarcÃa, S., Molina, D., Lozano, M., Herrera, F.: A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J. Heuristics 15, 617–644 (2009)
Acknowledgments
This work was financially supported by the National Natural Science Foundation of China under grant 62063021. It was also supported by the Key Program of National Natural Science Foundation of Gansu Province under Grant 23JRRA784, the High-level Foreign Experts Project of Gansu Province under Grant 22JR10KA007, and Lanzhou Science Bureau project (2018-rc-98), respectively.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhao, F., Yang, M. (2024). A Double Deep Q Network Guided Online Learning Differential Evolution Algorithm. In: Huang, DS., Zhang, X., Chen, W. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14862. Springer, Singapore. https://doi.org/10.1007/978-981-97-5578-3_16
Download citation
DOI: https://doi.org/10.1007/978-981-97-5578-3_16
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5577-6
Online ISBN: 978-981-97-5578-3
eBook Packages: Computer ScienceComputer Science (R0)