Skip to main content

Advertisement

Springer Nature Link
Log in

Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel two-stage progressive search approach with unsupervised feature learning and Q-learning (TSLL) to enhance surrogate-assisted evolutionary optimization for medium-scale expensive problems. The method attempts to address the challenges posed by multi-polar and multi-variable coupling properties in such problems. During the iteration, TSLL splits the optimization process into two distinct stages. First, two unsupervised feature learning techniques: principal component analysis (PCA) and Autoencoder, are utilized to improve the representation of potential optimal samples in the solution space. PCA is used for feature reduction, extracting the most important features. On the other hand, Autoencoder focuses on reconstructing features within the medium-scale solution space. To ensure comprehensive exploration of the entire solution space, TSLL employs two distinct surrogate modeling approaches along with Q-learning in the second stage to dynamically select the mutation strategy for the differential evolution operator. Numerous experiments demonstrate the superiority of TSLL over five state-of-the-art surrogate-assisted approaches and two sophisticated evolutionary algorithms including the winner of CEC 2017 on medium-scale benchmark problems and a wind farm layout problem.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

Data will be made available on reasonable request from the corresponding author (H. Yu).

References

  1. Wang X, Wang G, Song B, Wang P, Wang Y (2019) A novel evolutionary sampling assisted optimization method for high-dimensional expensive problems. IEEE Trans Evol Comput 23(5):815–827

    Article  MathSciNet  Google Scholar 

  2. Tong H, Huang C, Minku L, Yao X (2021) Surrogate models in evolutionary single-objective optimization: a new taxonomy and experimental study. Inf Sci 562:414–437

    Article  MathSciNet  Google Scholar 

  3. Park S, Na J, Kim M, Lee J (2018) Multi-objective Bayesian optimization of chemical reactor design using computational fluid dynamics. Comput Chem Eng 119:25–37

    Article  Google Scholar 

  4. Liu Y, Sun Y, Xue B, Zhang M, Yen G, Tan K (2023) A survey on evolutionary neural architecture search. IEEE Trans Neural Netw Learn Syst 34(2):550–570

    Article  MathSciNet  Google Scholar 

  5. Manc A, Pappalardo C (2020) Topology optimization procedure of aircraft mechanical components based on computer-aided design, multibody dynamics, and finite element analysis. Springer International Publishing, pp 159–168

    Google Scholar 

  6. Slowik A, Kwasnicka H (2020) Evolutionary algorithms and their applications to engineering problems. Neural Comput Appl 32:12363–12379

    Article  Google Scholar 

  7. Loshchilo I, Schoenauer M, Sebag M (2010) Comparison-based optimizers need comparison-based surrogates. In: PPSN XI, 2010

  8. Liu B, Zhang Q, Gielen GGE (2014) A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192

    Article  Google Scholar 

  9. Tian J, Tan Y, Zeng J, Sun C, Jin Y (2019) Multiobjective infill criterion driven Gaussian process-assisted particle swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 23(3):459–472

    Article  Google Scholar 

  10. Krithikaa M, Mallipeddi R (2016) Differential evolution with an ensemble of low-quality surrogates for expensive optimization problems. In: IEEE CEC 2016, Vancouver, BC, Canada, 2016, pp 78–85

  11. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45:50–79

    Article  Google Scholar 

  12. Buche D, Schraudolph NN, Koumoutsakos P (2005) Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Trans Syst Man Cybern 35:183–194

    Article  Google Scholar 

  13. Chugh T, Jin Y, Miettinen K, Hakanen J, Sindhya K (2018) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142

    Article  Google Scholar 

  14. Regis RG (2014) Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans Evol Comput 18(3):326–347

    Article  Google Scholar 

  15. Li F, Cai X, Gao L (2019) Ensemble of surrogates assisted particle swarm optimization of medium scale expensive problems. Appl Soft Comput 74:291–305

    Article  Google Scholar 

  16. Zhou XG, Zhang GJ (2017) Abstract convex underestimation assisted multistage differential evolution. IEEE Trans Cybern 47:2730–2741

    Article  MathSciNet  Google Scholar 

  17. Zhen H, Gong W, Wang L, Ming F, Liao Z (2023) Two-stage data-driven evolutionary optimization for high-dimensional expensive problems. IEEE Trans Cybern 53:2368–2379

    Article  Google Scholar 

  18. Yu H, Tan Y, Zeng J, Sun C, Jin Y (2018) Surrogate-assisted hierarchical particle swarm optimization. Inf Sci 454–455:59–72

    Article  MathSciNet  Google Scholar 

  19. Wang X, Gao L, Li X (2022) Multiple surrogates and offspring-assisted differential evolution for high-dimensional expensive problems. Inf Sci 592:174–191

    Article  Google Scholar 

  20. Skiena SS, Revilla MA (2003) Dynamic programming, in programming challenges: the programming contest training manual. SIGACT News 34:245–267

    Article  Google Scholar 

  21. Lever J, Krzywinski M, Altman NJNM (2017) Points of significance: principal component analysis. Nat Methods 14:641–642

    Article  Google Scholar 

  22. Balamurali M, Melkumyan A (2016) t-SNE based visualisation and clustering of geological domain. In: NeurIPS, pp 565–572

  23. Li X, Li S (2021) An adaptive surrogate-assisted particle swarm optimization for expensive problems. Soft Comput 25:15051–15065

    Article  Google Scholar 

  24. Yang Z, Qiu H, Gao L, Jiang C, Zhang J (2019) Two-layer adaptive surrogate-assisted evolutionary algorithm for high-dimensional computationally expensive problems. J Glob Optim 74:327–359

    Article  MathSciNet  Google Scholar 

  25. Zhao M, Zhang K, Chen G, Zhao X, Yao C, Sun H, Huang Z, Yao J (2020) A surrogate-assisted multi-objective evolutionary algorithm with dimension-reduction for production optimization. J Pet Sci Eng 192:0920–4105

    Article  Google Scholar 

  26. Gu H, Wang H, Jin Y (2022) Surrogate-assisted differential evolution with adaptive multi-Subspace search for large-scale expensive optimization. IEEE Trans Evol Comput. https://doi.org/10.1109/TEVC.2022.3226837

    Article  Google Scholar 

  27. Hao G, Lim M, Ong Y, Huang H, Wang G (2019) Domination landscape in evolutionary algorithms and its applications. Soft Comput 23:3563–3570

    Article  Google Scholar 

  28. Cui M, Li L, Zhou M, Li J, Abusorrah A, Sedraoui K (2022) A bi-population cooperative optimization algorithm assisted by an Autoencoder for medium-scale expensive problems. IEEE/CAA J Autom Sinica 9(11):1952–1966

    Article  Google Scholar 

  29. Das S, Suganthan PN (2011) Differential evolution: a survey of the state-of-the-art. IEEE Trans Evol Comput 15:4–31

    Article  Google Scholar 

  30. Li Z, Shi L, Yue C, Shang Z, Qu B (2019) Differential evolution based on reinforcement learning with fitness ranking for solving multimodal multiobjective problems. Swarm Evol 49:234–244

    Article  Google Scholar 

  31. Fister I, Fister D (2022) Reinforcement learning-based differential evolution for global optimization. Springer Nature, Singapore, pp 43–75

    Google Scholar 

  32. Huynh T, Do D, Lee J (2021) Q-Learning-based parameter control in differential evolution for structural optimization. Appl Soft Comput 107:107464

    Article  Google Scholar 

  33. Peng L, Yuan Z, Dai G, Wang M, Tang Z (2023) Reinforcement learning-based hybrid differential evolution for global optimization of interplanetary trajectory design. Swarm Evol 81:101351

    Article  Google Scholar 

  34. Liu X, Sun J, Zhang Q, Wang Z, Xu Z (2023) Learning to learn evolutionary algorithm: A learnable differential evolution. TETCI. https://doi.org/10.1109/TETCI.2023.3251441

    Article  Google Scholar 

  35. Tatsis VA, Parsopoulos KE (2023) Reinforcement learning for enhanced online gradient-based parameter adaptation in metaheuristics. Swarm Evol 83:101371

    Article  Google Scholar 

  36. Deng L, Liu S (2023) Incorporating Q-learning and gradient search scheme into JAYA algorithm for global optimization. Artif Intell Rev 56:3705–3748

    Article  Google Scholar 

  37. Zhao S, Wu Y, Tan S, Wu J, Cui Z, Wang Y (2023) QQLMPA: A quasi-opposition learning and Q-learning based marine predators algorithm. Expert Syst Appl 213:0957–4174

    Article  Google Scholar 

  38. Al-Hmouz R, Pedrycz W, Balamash W, Morfeq A (2022) Logic-oriented Autoencoders and granular logic Autoencoders: developing interpretable data representation. IEEE Trans Fuzzy Syst 30(3):869–877

    Article  Google Scholar 

  39. Bengio Y, Courville AC, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828

    Article  Google Scholar 

  40. Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279–292

    Article  Google Scholar 

  41. Helto JC, Davis FJ (2003) Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliab Eng Syst Saf 81:23–69

    Article  Google Scholar 

  42. Yu H, Gong Y, Kang L, Sun C, Zeng J (2023) Dual-drive collaboration surrogate-assisted evolutionary algorithm by coupling feature reduction and reconstruction. Complex Intell Syst. https://doi.org/10.1007/s40747-023-01168-3

    Article  Google Scholar 

  43. Díaz-Manríquez A, Toscano G, Coello CA (2016) Comparison of metamodeling techniques in evolutionary algorithms. Soft Comput 21(19):5647–5663. https://doi.org/10.1007/s00500-016-2140-z

    Article  Google Scholar 

  44. Wu G, Mallipeddi R, Suganthan P (2017) Problem definitions and evaluation criteria for the CEC 2017 competition and special session on constrained single objective real-parameter optimization. Nanyang Technol. Univ., Singapore, Jordan Univ. Sci. Technol., Ar-Ramtha, Jordan, Zhengzhou Univ., Zhengzhou, China

  45. Li F, Cai X, Gao L, Shen W (2021) A surrogate-assisted multiswarm optimization algorithm for high-dimensional computationally expensive problems. IEEE Trans Cybern 51(3):1390–1402

    Article  Google Scholar 

  46. Liu Y, Liu J, Jin Y (2022) Surrogate-assisted multipopulation particle swarm optimizer for high-dimensional expensive optimization. IEEE Trans Syst Man Cybern 52(7):4671–4684

    Article  Google Scholar 

  47. Brest J, Maučec M S, Bošković B (2017) Single objective real-parameter optimization: algorithm jSO. In: 2017 IEEE CEC, Donostia, Spain, pp 1311–1318

  48. Hellwig M, Beyer HG (2020) A modified matrix adaptation evolution strategy with restarts for constrained real-world problems. In: 2020 IEEE CEC, Glasgow, UK, pp 1–8

  49. Kumar A, Wu G, Ali MZ, Mallipeddi R, Suganthan PN, Das S (2020) A test-suite of non-convex constrained optimization problems from the real-world and some baseline results. Swarm Evol Comput 56:100693

    Article  Google Scholar 

  50. Wang Y, Liu H, Long H, Zhang Z, Yang S (2018) Differential evolution with a new encoding mechanism for optimizing wind farm layout. IEEE Trans Ind Inf 14(3):1040–1054

    Article  Google Scholar 

  51. Sugantha PN, Hanse N, Lian JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Nat Comput 341–357

  52. Hellwig M, Beyer HG (2018) A matrix adaptation evolution strategy for constrained real-parameter optimization. In: 2018 IEEE CEC. Rio de Janeiro, Brazil. IEEE, pp 1–8

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant No. 62106237, the Joint Funds of the National Natural Science Foundation of China under Grant No. U21A20524, and the Shanxi Province Science Foundation for Youths under Grant No. 202203021222057.

Author information

Authors and Affiliations

  1. School of Electrical and Control Engineering, North University of China, Taiyuan, 030051, China

    Yiyun Gong & Jianchao Zeng

  2. School of Computer Science and Technology, North University of China, Taiyuan, 030051, China

    Haibo Yu & Jianchao Zeng

  3. School of Environment and Safety Engineering, North University of China, Taiyuan, 030051, China

    Li Kang

  4. School of Computer Science and Technology, Taiyuan University of Science and Technology, Taiyuan, 030024, China

    Chaoli Sun

Authors
  1. Yiyun Gong
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Haibo Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Li Kang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Chaoli Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jianchao Zeng
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Yiyun Gong helped in formal analysis, methodology, software, and writing—original draft. Haibo Yu helped in conceptualization, methodology, and writing—review & editing. Li Kang helped in formal analysis and investigation. Chaoli Sun contributed to writing—review & editing and investigation. Jianchao Zeng worked in supervision.

Corresponding authors

Correspondence to Haibo Yu or Jianchao Zeng.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, Y., Yu, H., Kang, L. et al. Enhancing surrogate-assisted evolutionary optimization for medium-scale expensive problems: a two-stage approach with unsupervised feature learning and Q-learning. Neural Comput & Applic 36, 15545–15565 (2024). https://doi.org/10.1007/s00521-024-09903-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09903-8

Keywords