Skip to main content
SpringerLink
Log in

A comparative study of pre-screening strategies within a surrogate-assisted multi-objective algorithm framework for computationally expensive problems

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The multi-offspring method has been recognized as an efficient approach to enhance the performance of multi-objective evolutionary algorithms. However, some pre-screening strategies should be used when a multi-offspring-assisted multi-objective evolutionary algorithm is used to solve computationally expensive problems. So far, there is no any reported comprehensive study that compares the effects of different pre-screening strategies on the performance of the multi-offspring-assisted multi-objective evolutionary algorithms. In this paper, four pre-screening strategies (convergence-based, maximin distance-based expected improvement matrix (EIM-based), diversity-based and random-based strategies) for the multi-offspring-assisted multi-objective evolutionary algorithm are compared. The convergence-based strategy gives more priority to non-dominated solutions, and it is vital for exploiting the current promising areas. The diversity-based strategy gives more priority to solutions with greater uncertainties, and it is important for exploring the sparse areas. The EIM-based strategy considers the exploration and exploitation simultaneously, and the random-based strategy gives no priority to any solution. A series of benchmark problems whose dimensions vary from 8 to 30 and a reactive power optimization problem are used to test the multi-offspring-assisted multi-objective evolutionary algorithm under the four pre-screening strategies. The experimental results show that the convergence-based strategy performs best on most of the simple problems, while the EIM-based strategy performs best on most of the complex problems. The diversity-based strategy can produce positive effects on some problems, while the random-based strategy cannot improve the performance of its basic algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Tawhid MA, Savsani V (2017) Multi-objective sine-cosine algorithm (MO-SCA) for multi-objective engineering design problems. Neural Comput Appl 31(S2):915–929. https://doi.org/10.1007/s00521-017-3049-x

    Article  Google Scholar 

  2. Surender Reddy S, Bijwe PR (2017) Differential evolution-based efficient multi-objective optimal power flow. Neural Comput Appl 31(S1):509–522. https://doi.org/10.1007/s00521-017-3009-5

    Article  Google Scholar 

  3. Su Y-x, Chi R (2015) Multi-objective particle swarm-differential evolution algorithm. Neural Comput Appl 28(2):407–418. https://doi.org/10.1007/s00521-015-2073-y

    Article  Google Scholar 

  4. Basseur M, Zeng R-Q, Hao J-K (2011) Hypervolume-based multi-objective local search. Neural Comput Appl 21(8):1917–1929. https://doi.org/10.1007/s00521-011-0588-4

    Article  Google Scholar 

  5. Zhou A, Qu B-Y, Li H, Zhao S-Z, Suganthan PN, Zhang Q (2011) Multiobjective evolutionary algorithms: a survey of the state of the art. Swarm Evolut Comput 1(1):32–49. https://doi.org/10.1016/j.swevo.2011.03.001

    Article  Google Scholar 

  6. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2):182–197. https://doi.org/10.1109/4235.996017

    Article  Google Scholar 

  7. Zitzler E, Laumanns M, Thiele L (2001) SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-report 103. https://doi.org/10.3929/ethz-a-004284029

  8. Zhang QF, Li H (2007) MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans Evol Comput 11(6):712–731. https://doi.org/10.1109/Tevc.2007.892759

    Article  Google Scholar 

  9. Zitzler E, Kunzli S (2004) Indicator-based selection in multiobjective search. Paper presented at the International Conference on Parallel Problem Solving from Nature, Birmingham, United Kingdom

  10. Allmendinger R, Emmerich MTM, Hakanen J, Jin Y, Rigoni E (2017) Surrogate-assisted multicriteria optimization: complexities, prospective solutions, and business case. J Multi-Criteria Decis Anal 24(1–2):5–24. https://doi.org/10.1002/mcda.1605

    Article  Google Scholar 

  11. Kourakos G, Mantoglou A (2013) Development of a multi-objective optimization algorithm using surrogate models for coastal aquifer management. J Hydrol 479:13–23. https://doi.org/10.1016/j.jhydrol.2012.10.050

    Article  Google Scholar 

  12. Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13(4):455–492. https://doi.org/10.1023/A:1008306431147

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhang JH, Xiao M, Gao L, Fu JJ (2018) A novel projection outline based active learning method and its combination with Kriging metamodel for hybrid reliability analysis with random and interval variables. Comput Methods Appl Mech Eng 341:32–52. https://doi.org/10.1016/j.cma.2018.06.032

    Article  MathSciNet  MATH  Google Scholar 

  14. Zhang J, Xiao M, Gao L, Pan Q (2018) Queuing search algorithm: a novel metaheuristic algorithm for solving engineering optimization problems. Appl Math Model 63:464–490. https://doi.org/10.1016/j.apm.2018.06.036

    Article  MathSciNet  MATH  Google Scholar 

  15. Hardy RL (1971) Multiquadric equations of topography and other irregular surfaces. J Geophys Res 76(8):1905–1915. https://doi.org/10.1029/JB076i008p01905

    Article  Google Scholar 

  16. Horng S-C, Lin S-Y (2013) Evolutionary algorithm assisted by surrogate model in the framework of ordinal optimization and optimal computing budget allocation. Inf Sci 233:214–229. https://doi.org/10.1016/j.ins.2013.01.024

    Article  MathSciNet  Google Scholar 

  17. Liu A, Li P, Sun W, Deng X, Li W, Zhao Y, Liu B (2019) Prediction of mechanical properties of micro-alloyed steels via neural networks learned by water wave optimization. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04149-1

    Article  Google Scholar 

  18. Yadav N, Yadav A, Kumar M, Kim JH (2015) An efficient algorithm based on artificial neural networks and particle swarm optimization for solution of nonlinear Troesch’s problem. Neural Comput Appl 28(1):171–178. https://doi.org/10.1007/s00521-015-2046-1

    Article  Google Scholar 

  19. Goel T, Hafkta RT, Shyy W (2008) Comparing error estimation measures for polynomial and kriging approximation of noise-free functions. Struct Multidiscip Optim 38(5):429–442. https://doi.org/10.1007/s00158-008-0290-z

    Article  Google Scholar 

  20. Clarke SM, Griebsch JH, Simpson TW (2005) Analysis of support vector regression for approximation of complex engineering analyses. J Mech Des 127(6):1077–1087. https://doi.org/10.1115/1.1897403

    Article  Google Scholar 

  21. Zhou J, Duan B, Huang J, Cao H (2013) Data-driven modeling and optimization for cavity filters using linear programming support vector regression. Neural Comput Appl 24(7–8):1771–1783. https://doi.org/10.1007/s00521-013-1418-7

    Article  Google Scholar 

  22. Ciccazzo A, Pillo GD, Latorre V (2013) Support vector machines for surrogate modeling of electronic circuits. Neural Comput Appl 24(1):69–76. https://doi.org/10.1007/s00521-013-1509-5

    Article  Google Scholar 

  23. Goel T, Haftka RT, Shyy W, Queipo NV (2006) Ensemble of surrogates. Struct Multidiscip Optim 33(3):199–216. https://doi.org/10.1007/s00158-006-0051-9

    Article  Google Scholar 

  24. Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evolut Comput 1(2):61–70. https://doi.org/10.1016/j.swevo.2011.05.001

    Article  Google Scholar 

  25. Li F, Cai X, Gao L, Shen W (2020) A surrogate-assisted multiswarm optimization algorithm for high-dimensional computationally expensive problems. IEEE Trans Cybernet. https://doi.org/10.1109/tcyb.2020.2967553

    Article  Google Scholar 

  26. Li F, Cai X, Gao L (2019) Ensemble of surrogates assisted particle swarm optimization of medium scale expensive problems. Appl Soft Comput 74:291–305. https://doi.org/10.1016/j.asoc.2018.10.037

    Article  Google Scholar 

  27. Li F, Shen WM, Cai XW, Gao L, Wang GG (2020) A fast surrogate-assisted particle swarm optimization algorithm for computationally expensive problems. Appl Soft Comput. https://doi.org/10.1016/j.asoc.2020.106303

    Article  Google Scholar 

  28. Knowles J (2006) ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Trans Evol Comput 10(1):50–66. https://doi.org/10.1109/Tevc.2005.851274

    Article  Google Scholar 

  29. Luo JP, Gupta A, Ong YS, Wang ZK (2019) Evolutionary optimization of expensive multiobjective problems with co-sub-pareto front gaussian process surrogates. IEEE Trans Cybernet 49(5):1708–1721. https://doi.org/10.1109/Tcyb.2018.2811761

    Article  Google Scholar 

  30. Zhang QF, Liu WD, Tsang E, Virginas B (2010) Expensive multiobjective optimization by MOEA/D with Gaussian process model. IEEE Trans Evol Comput 14(3):456–474. https://doi.org/10.1109/Tevc.2009.2033671

    Article  Google Scholar 

  31. Habib A, Singh HK, Chugh T, Ray T, Miettinen K (2019) A multiple surrogate assisted decomposition based evolutionary algorithm for expensive multi/many-objective optimization. IEEE Trans Evol Comput 23(6):1000–1014. https://doi.org/10.1109/TEVC.2019.2899030

    Article  Google Scholar 

  32. Chugh T, Jin YC, Miettinen K, Hakanen J, Sindhya K (2018) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142. https://doi.org/10.1109/Tevc.2016.2622301

    Article  Google Scholar 

  33. Ponweiser W, Wagner T, Biermann D, Vincze M (2008) Multiobjective optimization on a limited budget of evaluations using model-assisted S-metric selection. Lect Notes Comput Sc 5199:784–794. https://doi.org/10.1007/978-3-540-87700-4_78

    Article  Google Scholar 

  34. Tabatabaei M, Hakanen J, Hartikainen M, Miettinen K, Sindhya K (2015) A survey on handling computationally expensive multiobjective optimization problems using surrogates: non-nature inspired methods. Struct Multidiscip Optim 52(1):1–25. https://doi.org/10.1007/s00158-015-1226-z

    Article  MathSciNet  Google Scholar 

  35. Chugh T, Sindhya K, Hakanen J, Miettinen K (2019) A survey on handling computationally expensive multiobjective optimization problems with evolutionary algorithms. Soft Comput 23(9):3137–3166. https://doi.org/10.1007/s00500-017-2965-0

    Article  Google Scholar 

  36. Diaz-Manriquez A, Toscano G, Barron-Zambrano JH, Tello-Leal E (2016) A review of surrogate assisted multiobjective evolutionary algorithms. Comput Intell Neurosci 2016:1–14. https://doi.org/10.1155/2016/9420460

    Article  Google Scholar 

  37. Cheng R, He C, Jin Y, Yao X (2018) Model-based evolutionary algorithms: a short survey. Complex Intell Syst 4(4):283–292. https://doi.org/10.1007/s40747-018-0080-1

    Article  Google Scholar 

  38. Berveglieri N, Derbel B, Liefooghe A, Aguirre H, Tanaka K (2019) Surrogate-assisted multiobjective optimization based on decomposition. Paper presented at the Genetic and Evolutionary Computation Conference 2019, Prague, Czech Republic

  39. Wang J, Ersoy OK, He M, Wang F (2016) Multi-offspring genetic algorithm and its application to the traveling salesman problem. Appl Soft Comput 43:415–423. https://doi.org/10.1016/j.asoc.2016.02.021

    Article  Google Scholar 

  40. Syberfeldt A, Grimm H, Ng A, John RI (2008) A parallel surrogate-assisted multi-objective evolutionary algorithm for computationally expensive optimization problems. In: 2008 IEEE congress on evolutionary computation (IEEE world congress on computational intelligence). IEEE, pp 3177–3184. https://doi.org/10.1109/cec.2008.4631228

  41. Datta R, Regis RG (2016) A surrogate-assisted evolution strategy for constrained multi-objective optimization. Expert Syst Appl 57:270–284. https://doi.org/10.1016/j.eswa.2016.03.044

    Article  Google Scholar 

  42. Regis RG (2014) Particle swarm with radial basis function surrogates for expensive black-box optimization. J Comput Sci 5(1):12–23. https://doi.org/10.1016/j.jocs.2013.07.004

    Article  MathSciNet  Google Scholar 

  43. Regis RG (2018) Surrogate-assisted particle swarm with local search for expensive constrained optimization. Paper presented at the international conference on bioinspired methods and their applications, Paris, France

  44. Regis RG (2016) Multi-objective constrained black-box optimization using radial basis function surrogates. J Comput Sci 16:140–155. https://doi.org/10.1016/j.jocs.2016.05.013

    Article  MathSciNet  Google Scholar 

  45. Forrester AIJ, Keane AJ (2009) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79. https://doi.org/10.1016/j.paerosci.2008.11.001

    Article  Google Scholar 

  46. Jeong S, Obayashi S. Efficient global optimization (EGO) for multi-objective problem and data mining. Paper presented at the 2005 IEEE congress on evolutionary computation, Edinburgh, Scotland, UK

  47. Zhan DW, Cheng YS, Liu J (2017) Expected improvement matrix-based infill criteria for expensive multiobjective optimization. IEEE Trans Evol Comput 21(6):956–975. https://doi.org/10.1109/Tevc.2017.2697503

    Article  Google Scholar 

  48. Keane AJ (2006) Statistical improvement criteria for use in multiobjective design optimization. AIAA J 44(4):879–891. https://doi.org/10.2514/1.16875

    Article  Google Scholar 

  49. Svenson J, Santner T (2016) Multiobjective optimization of expensive-to-evaluate deterministic computer simulator models. Comput Stat Data Anal 94:250–264. https://doi.org/10.1016/j.csda.2015.08.011

    Article  MathSciNet  MATH  Google Scholar 

  50. Emmerich MTM, Giannakoglou KC, Naujoks B (2006) Single-objective and multiobjective evolutionary optimization assisted by Gaussian random field metamodels. IEEE Trans Evol Comput 10(4):421–439. https://doi.org/10.1109/Tevc.2005.859463

    Article  Google Scholar 

  51. Couckuyt I, Deschrijver D, Dhaene T (2013) Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization. J Global Optim 60(3):575–594. https://doi.org/10.1007/s10898-013-0118-2

    Article  MathSciNet  MATH  Google Scholar 

  52. Hupkens I , André Deutz, Yang K , et al. Faster exact algorithms for computing expected hypervolume improvement. Paper presented at the international conference on evolutionary multi-criterion optimization 2015, Guimarães, Portugal

  53. Deb K, Jain H (2013) An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans Evol Comput 18(4):577–601. https://doi.org/10.1109/TEVC.2013.2281535

    Article  Google Scholar 

  54. Regis RG, Shoemaker CA (2013) Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng Optim 45(5):529–555. https://doi.org/10.1080/0305215x.2012.687731

    Article  MathSciNet  Google Scholar 

  55. Tian J, Tan Y, Zeng J, Sun C, Jin Y (2018) Multi-objective infill criterion driven gaussian process assisted particle swarm optimization of high-dimensional expensive problems. IEEE Trans Evol Comput 23(3):459–472. https://doi.org/10.1109/TEVC.2018.2869247

    Article  Google Scholar 

  56. Liu B, Zhang Q, Gielen GGE (2014) A Gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192. https://doi.org/10.1109/tevc.2013.2248012

    Article  Google Scholar 

  57. Wang H, Jin Y, Doherty J (2017) Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems. IEEE Trans Cybernet 47(9):2664–2677. https://doi.org/10.1109/TCYB.2017.2710978

    Article  Google Scholar 

  58. Lim D, Yaochu J, Yew-Soon O, Sendhoff B (2010) Generalizing surrogate-assisted evolutionary computation. IEEE Trans Evol Comput 14(3):329–355. https://doi.org/10.1109/tevc.2009.2027359

    Article  Google Scholar 

  59. Deb K (1999) Multi-objective genetic algorithms: problem difficulties and construction of test problems. Evol Comput 7(3):205–230. https://doi.org/10.1162/evco.1999.7.3.205

    Article  Google Scholar 

  60. Deb K, Thiele L, Laumanns M, Zitzler E (2002) Scalable multi-objective optimization test problems. Paper presented at the Proceedings of the 2002 congress on evolutionary computation. CEC’02 (Cat. No. 02TH8600), Honolulu, HI, USA

  61. Czyzżak P, Jaszkiewicz A (1998) Pareto simulated annealing—a metaheuristic technique for multiple-objective combinatorial optimization. J Multi-Criteria Decis Anal 7(1):34–47. https://doi.org/10.1002/(SICI)1099-1360(199801)7:1%3c34:AID-MCDA161%3e3.0.CO;2-6

    Article  MATH  Google Scholar 

  62. Chen Z, Zhou Y, He X (2019) Handling expensive multi-objective optimization problems with a cluster-based neighborhood regression model. Appl Soft Comput 80:211–225. https://doi.org/10.1016/j.asoc.2019.03.049

    Article  Google Scholar 

  63. Ye T, Ran C, Zhang X, Jin Y (2017) PlatEMO: a MATLAB platform for evolutionary multi-objective optimization [educational forum]. IEEE Comput Intell Mag 12(4):73–87. https://doi.org/10.1109/MCI.2017.2742868

    Article  Google Scholar 

  64. He C, Tian Y, Wang H, Jin Y (2019) A repository of real-world datasets for data-driven evolutionary multiobjective optimization. Complex Intell Syst 6:189–197. https://doi.org/10.1007/s40747-019-00126-2

    Article  Google Scholar 

  65. Zhan D, Cheng Y, Liu J (2017) Expected improvement matrix-based infill criteria for expensive multiobjective optimization. IEEE Trans Evol Comput 21(6):956–975

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation for Distinguished Young Scholars of China [Grant Number 51825502], the 111 Project [Grant Number B16019] and Program for HUST Academic Frontier Youth Team [Grant Number 2017QYTD04].

Author information

Authors and Affiliations

  1. State Key Laboratory of Digital Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Fan Li, Liang Gao, Akhil Garg & Weiming Shen

  2. National CNC Engineering Technology Research Center, School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Shifeng Huang

Authors
  1. Fan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Akhil Garg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weiming Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shifeng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiming Shen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, F., Gao, L., Garg, A. et al. A comparative study of pre-screening strategies within a surrogate-assisted multi-objective algorithm framework for computationally expensive problems. Neural Comput & Applic 33, 4387–4416 (2021). https://doi.org/10.1007/s00521-020-05258-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05258-y

Keywords

Search

Navigation