Skip to main content
SpringerLink
Log in

An efficient evolutionary algorithm based on deep reinforcement learning for large-scale sparse multiobjective optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Large-scale sparse multiobjective optimization problems (SMOPs) widely exist in academic research and engineering applications. The curse of dimensionality and the fact that most decision variables take zero values make optimization very difficult. Sparse features are common to many practical complex problems currently, and using sparse features as a breakthrough point can enable many large-scale complex problems to be solved. We propose an efficient evolutionary algorithm based on deep reinforcement learning to solve large-scale SMOPs. Deep reinforcement learning networks are used for mining sparse variables to reduce the problem dimensionality, which is a challenge for large-scale multiobjective optimization. Then the three-way decision concept is used to optimize decision variables. The emphasis is on optimizing deterministic nonzero variables and continuously mining uncertain decision variables. Experimental results on sparse benchmark problems and real-world application problems show that the proposed algorithm performs well on SMOPs while being highly efficient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Fig. 3
Algorithm 3
Algorithm 4
Algorithm 5
Algorithm 6
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability statement

The datasets generated during and analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Ye Tian, Chang L u, Zhang Xingyi, Cheng Fan, Jin Yaochu (2020) A pattern mining-based evolutionary algorithm for large-scale sparse multiobjective optimization problems. IEEE Trans Cybern, pp 1–14

  2. Sarker IH (2021) Machine learning: Algorithms, real-world applications and research directions. SN Comput Sci 2(3):1–21

    MathSciNet  Google Scholar 

  3. Cope B, Kalantzis M (2022) The cybernetics of learning

  4. Gong C, Ren T, Ye M, Liu Q (2021) Maxup: Lightweight adversarial training with data augmentation improves neural network training. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp 2474–2483

  5. Zhang Q, Ma W, Li G, Ding J, Xie M (2022) Fault diagnosis of power grid based on variational mode decomposition and convolutional neural network. Electr Power Syst Res 208:107871

    Google Scholar 

  6. Tan Z, Wang H, Liu S (2021) Multi-stage dimension reduction for expensive sparse multi-objective optimization problems. Neurocomputing 440:159–174

    Google Scholar 

  7. Song X-F, Zhang Y, Gong D-W, Sun X-Y (2021) Feature selection using bare-bones particle swarm optimization with mutual information. Pattern Recogn 112:107804

    Google Scholar 

  8. Narkhede MV, Bartakke PP, Sutaone MS (2022) A review on weight initialization strategies for neural networks. Artif Intell Rev 55(1):291–322

    Google Scholar 

  9. Fan Z, Hu G, Sun X, Wang G, Dong J, Su C (2022) Self-attention neural architecture search for semantic image segmentation. Knowl-Based Syst 239:107968

    Google Scholar 

  10. Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2022) Ensemble of feature selection algorithms: a multi-criteria decision-making approach. Int J Mach Learn Cybern 13(1):49–69

    Google Scholar 

  11. Alhenawi E, Al-Sayyed R, Hudaib A, Mirjalili S (2022) Feature selection methods on gene expression microarray data for cancer classification: a systematic review. Comput Biol Med 140:105051

    Google Scholar 

  12. Shafiullah Md, Abido MA, Al-Mohammed AH (2022) Intelligent fault diagnosis for distribution grid considering renewable energy intermittency. Neural Comput Applic, pp 1–20

  13. Zhang X, Tian Y, Cheng R, Jin Y (2018) A decision variable clustering-based evolutionary algorithm for large-scale many-objective optimization. IEEE Trans Evol Comput 22(1):97–112

    Google Scholar 

  14. Tian Y, Lu C, Zhang X, Tan KC, Jin Y (2020) Solving large-scale multi-objective optimization problems with sparse optimal solutions via unsupervised neural networks. IEEE Transactions on Cybernetics PP(99)

  15. Tian Y, Zheng X, Zhang X, Jin Y (2019) Efficient large-scale multi-objective optimization based on a competitive swarm optimizer. IEEE Trans Cybern, pp 1–13

  16. Tian Y, Si L, Zhang X, Cheng R, Jin Y (2021) Evolutionary large-scale multi-objective optimization: A survey. ACM Computing Surveys

  17. Antonio LM, Coello CAC (2016) Indicator-based cooperative coevolution for multi-objective optimization. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp 991–998

  18. Omidvar MN, Yang M, Yi Mei, Li X, Yao X (2017) Dg2: a faster and more accurate differential grouping for large-scale black-box optimization. IEEE Trans Evol Comput 21(6):929–942

    Google Scholar 

  19. Sun Y, Yue H (2022) An improved decomposition method for large-scale global optimization: bidirectional-detection differential grouping. Appl Intell 52(10):11569–11591

    Google Scholar 

  20. Ma X, Liu F, Qi Y, Wang X, Li L, Jiao L, Yin M, Gong M (2016) A multiobjective evolutionary algorithm based on decision variable analyses for multiobjective optimization problems with large-scale variables. IEEE Trans Evol Comput 20(2):275–298

    Google Scholar 

  21. He C, Li L, Tian Y, Zhang X, Cheng R, Jin Y, Yao X (2019) Accelerating large-scale multiobjective optimization via problem reformulation. IEEE Trans Evol Comput 23(6):949–961

    Google Scholar 

  22. Chen H, Ran C, Wen J, Li H, Jian W (2018) Solving large-scale many-objective optimization problems by covariance matrix adaptation evolution strategy with scalable small subpopulations. Inf Sci, p 509

  23. Ding Z, Chen L, Sun D, Zhang X (2022) A multi-stage knowledge-guided evolutionary algorithm for large-scale sparse multi-objective optimization problems. Swarm Evol Comput 73:101119

    Google Scholar 

  24. Tian Y, Zhang X, Wang C, Jin Y (2020) An evolutionary algorithm for large-scale sparse multiobjective optimization problems. IEEE Trans Evol Comput 24(2):380–393

    Google Scholar 

  25. Fournier-Viger P, Lin JC-W, Kiran RU, Koh YS, Thomas R (2017) A survey of sequential pattern mining. Data Sci Pattern Recognit 1(1):54–77

    Google Scholar 

  26. Alsahaf A, Petkov N, Shenoy V, Azzopardi George (2022) A framework for feature selection through boosting. Expert Syst Appl 187:115895

    Google Scholar 

  27. Shetty RD, Bhattacharjee S, Dutta A, Namtirtha A (2022) Gsi: An influential node detection approach in heterogeneous network using covid-19 as use case. IEEE Trans Comput Soc Syst

  28. Zhang X, Duan F, Lei Z, Fan C, Jin Y, Ke T (2017) Pattern recommendation in task-oriented applications: a multi-objective perspective [application notes]. IEEE Comput Intell Mag 12(3):43–53

    Google Scholar 

  29. Zhang Y, Tian Y, Zhang X (2021) Improved sparseea for sparse large-scale multi-objective optimization problems. Complex Intell Syst, p 10

  30. Liu CH, Chen Z, Tang J, Xu J, Piao C (2018) Energy-efficient uav control for effective and fair communication coverage: a deep reinforcement learning approach. IEEE J Sel Areas Commun 36 (9):2059–2070

    Google Scholar 

  31. Chen L, Jiang S, Liu J, Wang C, Zhang S, Xie C, Liang J, Xiao Y, Song R (2022) Rule mining over knowledge graphs via reinforcement learning. Knowl-Based Syst 242:108371

    Google Scholar 

  32. Fan T-H, Wang Y (2022) Soft actor-critic with integer actions. In: 2022 American Control Conference (ACC). IEEE, pp 2611–2616

  33. Yuan Y, Lei L, Vu TX, Chatzinotas S, Sun S, Ottersten B (2021) Energy minimization in uav-aided networks: Actor-critic learning for constrained scheduling optimization. IEEE Trans Veh Technol 70 (5):5028–5042

    Google Scholar 

  34. Wei Y, Yu FR, Song M, Han Z (2019) Joint optimization of caching, computing, and radio resources for fog-enabled iot using natural actor-critic deep reinforcement learning. IEEE Int Things J 6(2):2061–2073

    Google Scholar 

  35. Liu C-L, Chang C-C, Tseng C-J (2020) Actor-critic deep reinforcement learning for solving job shop scheduling problems. IEEE Access 8:71752–71762

    Google Scholar 

  36. Vamvoudakis KG, Lewis FL (2010) Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46(5):878–888

    MathSciNet  MATH  Google Scholar 

  37. Kiumarsi B, Vamvoudakis KG, Modares H, Lewis FL (2017) Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans Neural Netw Learn Syst 29(6):2042–2062

    MathSciNet  Google Scholar 

  38. Gao M, Feng X, Yu H, Zheng Z (2022) Multi-granularity competition-cooperation optimization algorithm with adaptive parameter configuration. Appl Intell, pp 1–30

  39. Schweighofer N, Doya K (2003) Meta-learning in reinforcement learning. Neural Netw 16 (1):5–9

    Google Scholar 

  40. Peng B, Li X, Gao J, Liu J, Chen Y-N, Wong K-F (2018) Adversarial advantage actor-critic model for task-completion dialogue policy learning. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 6149–6153

  41. Zheng Y, Li X, Xu L (2020) Balance control for the first-order inverted pendulum based on the advantage actor-critic algorithm. Inter J Control Auto Syst 18(12):3093–3100

    Google Scholar 

  42. Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. In: International conference on machine learning. PMLR, pp 1928–1937

  43. Shang K, Ishibuchi H, He L, Pang LM (2021) A survey on the hypervolume indicator in evolutionary multiobjective optimization. IEEE Trans Evol Comput 25(1):1–20

    Google Scholar 

  44. Chen H, Dai X, Cai H, Zhang W, Yu Y (2019) Large-scale interactive recommendation with tree-structured policy gradient. In: Proceedings of the AAAI Conference on artificial intelligence, vol 33, pp 3312–3320

  45. Zhao S, Liu R, Bo C, Zhao D (2022) Classification-labeled continuousization and multi-domain spatio-temporal fusion for fine-grained urban crime prediction. IEEE Trans Knowl Data Eng, pp 1–14

  46. Yang S, Bo Y, Wong H-S, Kang Z (2019) Cooperative traffic signal control using multi-step return and off-policy asynchronous advantage actor-critic graph algorithm. Knowl-Based Syst 183:104855

    Google Scholar 

  47. Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017) Mastering the game of go without human knowledge. Nature 550 (7676):354–359

    Google Scholar 

  48. Zhang B, Hu W, Cao D, Li T, Zhang Z, Chen Z, Blaabjerg F (2021) Soft actor-critic–based multi-objective optimized energy conversion and management strategy for integrated energy systems with renewable energy. Energy Convers Manag 243:114381

    Google Scholar 

  49. Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

    Google Scholar 

  50. Memarian F, Goo W, Lioutikov R, Niekum S, Topcu U (2021) Self-supervised online reward shaping in sparse-reward environments. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 2369–2375

  51. Zhan J, Ye J, Ding W, Liu P (2021) A novel three-way decision model based on utility theory in incomplete fuzzy decision systems. IEEE Trans Fuzzy Syst

  52. Yao Y (2021) The geometry of three-way decision. Appl Intell 51(9):6298–6325

    Google Scholar 

  53. Bo Y, Li J (2020) Complex network analysis of three-way decision researches. Int J Mach Learn Cybern 11(5):973–987

    Google Scholar 

  54. Yang X, Li T, Tan A (2020) Three-way decisions in fuzzy incomplete information systems. Int J Mach Learn Cybern 11(3):667–674

    Google Scholar 

  55. Li H, Zhang L, Huang B, Zhou X (2016) Sequential three-way decision and granulation for cost-sensitive face recognition. Knowl-Based Syst 91:241–251

    Google Scholar 

  56. Zhang Q, Pang G, Wang G (2020) A novel sequential three-way decisions model based on penalty function. Knowl-Based Syst 192:105350

    Google Scholar 

  57. Ma Y, Bai Y (2020) A multi-population differential evolution with best-random mutation strategy for large-scale global optimization. Appl Intell 50(5):1510–1526

    Google Scholar 

  58. Wang H, Jiao L, Yao X (2015) Twoarch2: an improved two-archive algorithm for many-objective optimization. IEEE Trans Evol Comput 19(4):524–541

    Google Scholar 

  59. Tian Y, Cheng R, Zhang X, Jin Y (2017) Platemo: a matlab platform for evolutionary multi-objective optimization [educational forum]. IEEE Comput Intell Mag 12(4):73–87

    Google Scholar 

  60. Ishibuchi H, Imada R, Setoguchi Y, Nojima Y (2018) Reference point specification in inverted generational distance for triangular linear pareto front. IEEE Trans Evol Comput 22(6):961–975

    Google Scholar 

  61. Said R, Bechikh S, Louati A, Aldaej A, Said LB (2020) Solving combinatorial multi-objective bi-level optimization problems using multiple populations and migration schemes. IEEE Access 8:141674–141695

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No.62276097), Key Program of National Natural Science Foundation of China (No.62136003), National Key Research and Development Program of China (No. 2020YFB1711700), Special Fund for Information Development of Shanghai Economic and Information Commission (No.XX-XXFZ-02-20-2463) and Scientific Research Program of Shanghai Science and Technology Commission (No.21002411000).

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Mengqi Gao, Xiang Feng & Huiqun Yu

  2. Shanghai Engineering Research Center of Smart Energy, Shanghai, China

    Mengqi Gao, Xiang Feng & Huiqun Yu

  3. Chinese Academy of Science and Technology for Development, Beijing, 100038, China

    Xiuquan Li

Authors
  1. Mengqi Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huiqun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuquan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiang Feng.

Ethics declarations

Ethics approval

This paper does not contain any studies with human participants or animals performed by any of the authors.

Conflict of Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, M., Feng, X., Yu, H. et al. An efficient evolutionary algorithm based on deep reinforcement learning for large-scale sparse multiobjective optimization. Appl Intell 53, 21116–21139 (2023). https://doi.org/10.1007/s10489-023-04574-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04574-9

Keywords

Search

Navigation