Skip to main content
SpringerLink
Log in

Offline data-driven optimization based on dual-scale surrogate ensemble

  • Regular research paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

During offline optimization, real fitness evaluations cannot be obtained. Therefore, building high-quality surrogates based on the limited amount of offline data is crucial. Many ensemble surrogates are built by combining a number of models that are trained based on different subsampled data, and these weak models without characteristics lack interpretation. In this paper, we propose an adaptive differential evolution based on a dual-scale surrogate ensemble model, termed DSEDE. The ensemble model consists of a fine scale surrogate model and a global scale surrogate model, which approximates fine landscape information and the overall trend landscape, respectively. Adaptive differential evolution is used to search for the global optimum of the offline surrogate model. Comparing four state-of-the-art offline data-driven optimization algorithms, the results show that the proposed algorithm is efficient and effective in solving offline optimization problems. DSEDE costs the least time in optimization, especially for high-dimensional problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Brownlee AE, McCall JA, Zhang Q (2013) Fitness Modeling with Markov Networks. IEEE Transact Evolutionary Computa 17(6):862–879

    Article  Google Scholar 

  2. Cai X, Gao L, Li X (2020) Efficient generalized surrogate-assisted evolutionary algorithm for high-dimensional expensive problems. IEEE Transact Evolutionary Comput 24(2):365–379

    Article  Google Scholar 

  3. Cai X, Qiu H, Gao L, Jiang C, Shao X (2019) An efficient surrogate-assisted particle swarm optimization algorithm for high-dimensional expensive problems. Knowledge-Based Syst 184:104901

    Article  Google Scholar 

  4. Chen G, Li Y, Zhang K, Xue X, Wang J, Luo Q, Yao C, Yao J (2021) Efficient hierarchical surrogate-assisted differential evolution for high-dimensional expensive optimization. Inform Sci 542:228–246

    Article  MathSciNet  MATH  Google Scholar 

  5. Cheng R, Jin Y (2015) A social learning particle swarm optimization algorithm for scalable optimization. Inform Sci 291(C):43–60

    Article  MathSciNet  MATH  Google Scholar 

  6. Chugh T, Chakraborti N, Sindhya K, Jin Y (2017) A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem. Mater Manufact Process 32(10):1172–1178

    Article  Google Scholar 

  7. Dong H, Dong Z (2020) Surrogate-assisted grey wolf optimization for high-dimensional, computationally expensive black-box problems. Swarm Evolutionary Comput 57:100713

    Article  Google Scholar 

  8. Emmerich MT, Giannakoglou KC, Naujoks B (2006) Single-and multiobjective evolutionary optimization assisted by gaussian random field metamodels. IEEE Transact Evolutionary Comput 10(4):421–439

    Article  Google Scholar 

  9. Gaspar-Cunha A, Vieira A (2004) A hybrid multi-objective evolutionary algorithm using an inverse neural network. In: Hybrid Metaheuristics. pp. 25–30

  10. Gaspar-Cunha A, Vieira A (2005) A multi-objective evolutionary algorithm using neural networks to approximate fitness evaluations. Int. J. Comput. Syst. Signals 6(1):18–36

    Google Scholar 

  11. Gong W, Cai Z, Wang Y (2014) Repairing the crossover rate in adaptive differential evolution. Appl Soft Comput 15:149–168

    Article  Google Scholar 

  12. Gong W, Liao Z, Mi X, Wang L, Guo Y (2021) Nonlinear equations solving with intelligent optimization algorithms: A survey. Complex Syst Modeling Simulat 1(1):15–32

    Article  Google Scholar 

  13. Guo D, Chai T, Ding J, Jin Y (2016) Small data driven evolutionary multi-objective optimization of fused magnesium furnaces. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI). pp. 1–8

  14. Han L, Wang H (2021) A random forest assisted evolutionary algorithm using competitive neighborhood search for expensive constrained combinatorial optimization. Memetic Comput 13(1):19–30

    Article  MathSciNet  Google Scholar 

  15. Huang P, Wang H, Jin Y (2021) Offline data-driven evolutionary optimization based on tri-training. Swarm Evolutionary Comput 60:100800

    Article  Google Scholar 

  16. Jin Y, Olhofer M, Sendhoff B (2002) A framework for evolutionary optimization with approximate fitness functions. IEEE Transact evolutionary comput 6(5):481–494

    Article  Google Scholar 

  17. Jin Y, Wang H, Chugh T, Guo D, Miettinen K (2018) Data-driven evolutionary optimization: an overview and case studies. IEEE Transact Evolutionary Comput 23(3):442–458

    Article  Google Scholar 

  18. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-international conference on neural networks. Vol. 4. IEEE, pp. 1942–1948

  19. Lessmann S, Stahlbock R, Crone SF (2006) Genetic algorithms for support vector machine model selection. In: The 2006 IEEE International Joint Conference on Neural Network Proceedings. IEEE, pp. 3063–3069

  20. Li J-Y, Zhan Z-H, Wang C, Jin H, Zhang J (2020) Boosting data-driven evolutionary algorithm with localized data generation. IEEE Transact Evolutionary Comput 24(5):923–937

    Article  Google Scholar 

  21. Li J-Y, Zhan Z-H, Wang H, Zhang J (2020b) Data-driven evolutionary algorithm with perturbation-based ensemble surrogates. IEEE Transactions on Cybernetics, 1–13

  22. Li J-Y, Zhan Z-H, Zhang J (2022) Evolutionary computation for expensive optimization: A survey. Mach Intelligence Res 19(1):3–23

    Article  Google Scholar 

  23. Liu B, Zhang Q, Gielen GG (2013) A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Transact Evolutionary Comput 18(2):180–192

    Article  Google Scholar 

  24. Loshchilov I, Schoenauer M, Sebag M (2010) Comparison-based optimizers need comparison-based surrogates. In: International Conference on Parallel Problem Solving from Nature. Springer, pp. 364–373

  25. Nguyen HB, Xue B, Andreae P (2018) Pso with surrogate models for feature selection: static and dynamic clustering-based methods. Memetic Comput 10(3):291–300

    Article  Google Scholar 

  26. Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural comput 3(2):246–257

    Article  Google Scholar 

  27. Regis RG (2013) An initialization strategy for high-dimensional surrogate-based expensive black-box optimization. In: Modeling and optimization: theory and applications. Springer, pp. 51–85

  28. Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J global optimi 11(4):341–359

    Article  MathSciNet  MATH  Google Scholar 

  29. Suganthan PN, Hansen N, Liang JJ, Deb K, Tiwari S (2005) Problem definitions and evaluation criteria for the cec 2005 special session on real-parameter optimization. Natural Computing 341-357

  30. Sun C, Ding J, Zeng J, Jin Y (2018) A fitness approximation assisted competitive swarm optimizer for large scale expensive optimization problems. Memetic Comput 10(2):123–134

    Article  Google Scholar 

  31. Tanabe R, Fukunaga AS (2014) Improving the search performance of shade using linear population size reduction. In: 2014 IEEE congress on evolutionary computation (CEC). IEEE, pp. 1658–1665

  32. Tian J, Tan Y, Zeng J, Sun C, Jin Y (2019) Multiobjective infill criterion driven gaussian process-assisted particle swarm optimization of high-dimensional expensive problems. IEEE Transact Evolutionary Comput 23(3):459–472

    Article  Google Scholar 

  33. Tong H, Huang C, Minku LL, Yao X (2021) Surrogate models in evolutionary single-objective optimization: A new taxonomy and experimental study. Inform Sci 562:414–437

    Article  MathSciNet  Google Scholar 

  34. Wang H, Jin Y (2020) A random forest-assisted evolutionary algorithm for data-driven constrained multiobjective combinatorial optimization of trauma systems. IEEE Transact Cybernetics 50(2):536–549

    Article  Google Scholar 

  35. Wang H, Jin Y, Jansen JO (2016) Data-driven surrogate-assisted multiobjective evolutionary optimization of a trauma system. IEEE Transact Evolutionary Comput 20(6):939–952

    Article  Google Scholar 

  36. Wang H, Jin Y, Sun C, Doherty J (2019) Offline data-driven evolutionary optimization using selective surrogate ensembles. IEEE Transact Evolutionary Comput 23(2):203–216

    Article  Google Scholar 

  37. Wang X, Wang GG, Song B, Wang P, Wang Y (2019) A novel evolutionary sampling assisted optimization method for high-dimensional expensive problems. IEEE Transact Evolutionary Comput 23(5):815–827

    Article  MathSciNet  Google Scholar 

  38. Wei F, Chen W, Yang Q, Deng J, Luo X, Jin H, Zhang J (2020) A classifier-assisted level-based learning swarm optimizer for expensive optimization. IEEE Transactions on Evolutionary Computation, 1

  39. Willmes L, Back T, Jin Y, Sendhoff B (2003) Comparing neural networks and kriging for fitness approximation in evolutionary optimization. In: The 2003 Congress on Evolutionary Computation, 2003. CEC’03. Vol. 1. IEEE, pp. 663–670

  40. Wu G, Mallipeddi R, Suganthan PN (2017) Problem definitions and evaluation criteria for the cec 2017 competition on constrained real-parameter optimization. National University of Defense Technology, Changsha, Hunan, PR China and Kyungpook National University, Daegu, South Korea and Nanyang Technological University, Singapore, Technical Report

  41. Yu H, Tan Y, Zeng J, Sun C, Jin Y (2018) Surrogate-assisted hierarchical particle swarm optimization. Infom Sci 454:59–72

    Article  MathSciNet  Google Scholar 

  42. Zhang J, Sanderson AC (2009) Jade: adaptive differential evolution with optional external archive. IEEE Transactions evolutionary comput 13(5):945–958

    Article  Google Scholar 

  43. Zhang Q, Liu W, Tsang E, Virginas B (2009) Expensive multiobjective optimization by moea/d with gaussian process model. IEEE Transact Evolutionary Comput 14(3):456–474

    Article  Google Scholar 

  44. Zhao Y, Sun C, Zeng J, Tan Y, Zhang G (2021) A surrogate-ensemble assisted expensive many-objective optimization. Knowled-Based Syst 211:106520

    Article  Google Scholar 

  45. Zhen H, Gong W, Wang L (2022a) Evolutionary sampling agent for expensive problems. IEEE Transactions on Evolutionary Computation, 1

  46. Zhen H, Gong W, Wang L (2022) Offline data-driven evolutionary optimization based on model selection. Swarm and Evolutionary Computation 71:101080

    Article  Google Scholar 

  47. Zhen H, Gong W, Wang L, Ming F, Liao Z (2021) Two-stage data-driven evolutionary optimization for high-dimensional expensive problems. IEEE Transactions on Cybernetics, 1–12

  48. Zhou Z, Ong YS, Nguyen MH, Lim D (2005) A study on polynomial regression and gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm. In: 2005 IEEE congress on evolutionary computation. Vol. 3. IEEE, pp. 2832–2839

Download references

Author information

Authors and Affiliations

  1. School of Science, Qiongtai Normal University, Haikou, 571127, China

    Junhua Ku

  2. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Huixiang Zhen & Wenyin Gong

Authors
  1. Junhua Ku
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huixiang Zhen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenyin Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Huixiang Zhen or Wenyin Gong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ku, J., Zhen, H. & Gong, W. Offline data-driven optimization based on dual-scale surrogate ensemble. Memetic Comp. 15, 139–154 (2023). https://doi.org/10.1007/s12293-022-00380-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-022-00380-x

Keywords

Search

Navigation