Skip to main content

Advertisement

SpringerLink
Log in

A random forest assisted evolutionary algorithm using competitive neighborhood search for expensive constrained combinatorial optimization

  • Regular Research Paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

Many real-world combinatorial optimization problems have both expensive objective and constraint functions. Although surrogate models for the discrete decision variables can be trained to replace the expensive fitness evaluations in evolutionary algorithms, the approximation errors of the surrogate models for the constraint function easily misguide the search. The classic genetic algorithm, which is often applied straightforwardly to the combinatorial optimization problems, gradually exposes its inefficiency in the search process. Therefore, we proposed a random forest assisted evolutionary algorithm using a new competitive neighborhood search, where random forest is used as the surrogate models to approximate both the objective and constraint functions and the competitive neighborhood search is to improve the search efficiency. Moreover, competitive neighborhood search shows a natural adaptability to the surrogate model, which helps to reduce the impact of approximation errors. The proposed algorithm is tested on 01 knapsack problems and quadratic knapsack problems with various dimensions and constraints. The experimental results demonstrate that the proposed algorithm is able to solve the expensive constrained combinatorial optimization problems efficiently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Alyahya K, Rowe JE (2019) Landscape analysis of a class of np-hard binary packing problems. Evol Comput 27(1):47–73

    Article  Google Scholar 

  2. Cheng P, Chen M, Stojanovic V, He S (2021) Asynchronous fault detection filtering for piecewise homogenous markov jump linear systems via a dual hidden markov model. Mech Syst Signal Process 151:107353

    Article  Google Scholar 

  3. Bartz-Beielstein T, Zaefferer M (2017) Model-based methods for continuous and discrete global optimization. Appl Soft Comput 55:154–167

    Article  Google Scholar 

  4. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Statal Soc Ser B Methodol 57(1):289–300

    MathSciNet  MATH  Google Scholar 

  5. Bernardino HS, Fonseca LG, Barbosa HJ (2009) Surrogate-assisted artificial immune systems for expensive optimization problems. Evol Comput 179–198

  6. Branke J, Schmidt C (2005) Faster convergence by means of fitness estimation. Soft Comput 9(1):13–20

    Article  Google Scholar 

  7. Caprara A, Pisinger D, Toth P (1999) Exact solution of the quadratic knapsack problem. INFORMS J Comput 11(2):125–137

    Article  MathSciNet  MATH  Google Scholar 

  8. Ce F, Hui W (2010) The solving strategy for the real-world vehicle routing problem. In: 2010 3rd international congress on image and signal processing

  9. Chugh T, Jin Y, Miettinen K, Hakanen J, Sindhya K (2016) A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization. IEEE Trans Evol Comput 22(1):129–142

    Article  Google Scholar 

  10. Coello Coello CA (2000) Constraint-handling using an evolutionary multiobjective optimization technique. Civ Eng Syst 17(4):319–346

    Article  Google Scholar 

  11. Coit DW, Smith AE, Tate DM (1996) Adaptive penalty methods for genetic optimization of constrained combinatorial problems. INFORMS J Comput 8(2):173–182

    Article  MATH  Google Scholar 

  12. Couckuyt I, Declercq F, Dhaene T, Rogier H, Knockaert L (2010) Surrogate-based infill optimization applied to electromagnetic problems. Int J RF Microwave Comput-Aided Eng 20(5):492–501

    Article  Google Scholar 

  13. Dasgupta D, Michalewicz Z (2013) Evolutionary algorithms in engineering applications. Springer, Berlin

    MATH  Google Scholar 

  14. Deb K (2000) An efficient constraint handling method for genetic algorithms. Comput Methods Appl Mech Eng 186(2–4):311–338

    Article  MATH  Google Scholar 

  15. Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol Comput 1(1):3–18

    Article  Google Scholar 

  16. Dong X, He S, Stojanovic V (2020) Robust fault detection filter design for a class of discrete-time conic-type nonlinear markov jump systems with jump fault signals. IET Control Theory Appl 14:1912–1919

    Article  Google Scholar 

  17. Ehrgott M, Gandibleux X (2000) A survey and annotated bibliography of multiobjective combinatorial optimization. OR-Spektrum 22(4):425–460

    Article  MathSciNet  MATH  Google Scholar 

  18. Eiben AE, Smith JE et al (2003) Introduction to evolutionary computing, vol 53. Springer, Berlin

    Book  MATH  Google Scholar 

  19. Emmerich M, Giotis A, Özdemir M, Bäck T, Giannakoglou K (2002) Metamodel—assisted evolution strategies. In: International conference on parallel problem solving from nature, pp 361–370. Springer

  20. Emmerich MTM, Giannakoglou KC, Naujoks B (2006) Single- and multiobjective evolutionary optimization assisted by gaussian random field metamodels. IEEE Trans Evol Comput 10(4):421–439

    Article  Google Scholar 

  21. Fleming PJ, Purshouse RC (2002) Evolutionary algorithms in control systems engineering: a survey. Control Eng Pract 10(11):1223–1241

    Article  Google Scholar 

  22. Florios K, Mavrotas G, Diakoulaki D (2010) Solving multiobjective, multiconstraint knapsack problems using mathematical programming and evolutionary algorithms. Eur J Oper Res 203(1):14–21

    Article  MathSciNet  MATH  Google Scholar 

  23. Fonseca L, Barbosa HJ, Lemonge AC (2009) A similarity-based surrogate model for expensive evolutionary optimization with fixed budget of simulations. In: 2009 IEEE congress on evolutionary computation, pp 867–874. IEEE

  24. Gallo G, Hammer PL, Simeone B (1980) Quadratic knapsack problems. In: Combinatorial optimization, pp 132–149. Springer

  25. Goldberg DE (1989) Genetic algorithms in search. Optim Mach Learn

  26. Goldberg DE, Deb K (1991) A comparative analysis of selection schemes used in genetic algorithms. In: Foundations of genetic algorithms, vol 1, pp 69–93. Elsevier

  27. Handl J, Hart E, Lewis PR, López-Ibáñez M, Ochoa G, Paechter B (2016) Parallel problem solving from nature–PPSN XIV: 14th International Conference, Edinburgh, UK, September 17–21, 2016, Proceedings, vol 9921. Springer

  28. Hansen P, Mladenović N, Pérez JAM (2010) Variable neighbourhood search: methods and applications. Ann Oper Res 175(1):367–407

    Article  MathSciNet  MATH  Google Scholar 

  29. He C, Tian Y, Wang H, Jin Y (2019) A repository of real-world datasets for data-driven evolutionary multiobjective optimization. Complex Intell Syst 1–9

  30. Hildebrandt T, Branke J (2015) On using surrogates with genetic programming. Evol Comput 23(3):343–367

    Article  Google Scholar 

  31. Hoffman KL (2000) Combinatorial optimization: current successes and directions for the future. J Comput Appl Math 124(1–2):341–360

    Article  MathSciNet  MATH  Google Scholar 

  32. Hüsken M, Jin Y, Sendhoff B (2005) Structure optimization of neural networks for evolutionary design optimization. Soft Comput 9(1):21–28

    Article  Google Scholar 

  33. Hutter F (2009) Automated configuration of algorithms for solving hard computational problems. Ph.D. thesis, University of British Columbia

  34. Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12

    Article  Google Scholar 

  35. Jin Y (2011) Surrogate-assisted evolutionary computation: recent advances and future challenges. Swarm Evol Comput 1(2):61–70

    Article  Google Scholar 

  36. Jin Y, Olhofer M, Sendhoff B (2000) On evolutionary optimization with approximate fitness functions. In: Proceedings of the 2nd annual conference on genetic and evolutionary computation, pp 786–793

  37. Jin Y, Olhofer M, Sendhoff B (2002) A framework for evolutionary optimization with approximate fitness functions. IEEE Trans Evol Comput 6(5):481–494

    Article  Google Scholar 

  38. Jin Y, Wang H, Chugh T, Guo D, Miettinen K (2018) Data-driven evolutionary optimization: an overview and case studies. IEEE Trans Evol Comput 23(3):442–458

    Article  Google Scholar 

  39. Lin S, Kernighan BW (1973) An effective heuristic algorithm for the traveling-salesman problem. Oper Res 21(2):498–516

    Article  MathSciNet  MATH  Google Scholar 

  40. Liu B, Zhang Q, Gielen GGE (2014) A gaussian process surrogate model assisted evolutionary algorithm for medium scale expensive optimization problems. IEEE Trans Evol Comput 18(2):180–192

    Article  Google Scholar 

  41. Miller BL, Goldberg DE et al (1995) Genetic algorithms, tournament selection, and the effects of noise. Complex Syst 9(3):193–212

    MathSciNet  Google Scholar 

  42. Mladenović N, Hansen P (1997) Variable neighborhood search. Comput Oper Res 24(11):1097–1100

    Article  MathSciNet  MATH  Google Scholar 

  43. Moraglio A, Kattan A (2011) Geometric generalisation of surrogate model based optimisation to combinatorial spaces. In: European conference on evolutionary computation in combinatorial optimization, pp 142–154. Springer

  44. Myers RH, Montgomery DC, Anderson-Cook CM (2016) Response surfacemethodology: process and product optimization using designed experiments. Wiley, Hoboken

    MATH  Google Scholar 

  45. Nair P, Keane A, Shimpi R (1912) Combining approximation concepts with genetic algorithm-based structural optimization procedures. In: 39th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference and exhibit, p 1912

  46. Namura N, Shimoyama K, Obayashi S (2017) Expected improvement of penalty-based boundary intersection for expensive multiobjective optimization. IEEE Trans Evol Comput 898–913

  47. Nguyen K, Nguyen D, Trieu K, Tran N (2010) Automating a real-world university timetabling problem with tabu search algorithm. In: 2010 IEEE RIVF international conference on computing & communication technologies, research, innovation, and vision for the future (RIVF), pp 1–6. IEEE

  48. Nguyen S, Zhang M, Johnston M, Tan KC (2014) Selection schemes in surrogate-assisted genetic programming for job shop scheduling. In: Asia-pacific conference on simulated evolution and learning, pp 656–667. Springer

  49. Nguyen S, Zhang M, Tan KC (2016) Surrogate-assisted genetic programming with simplified models for automated design of dispatching rules. IEEE Trans Cybern 47(9):2951–2965

    Article  Google Scholar 

  50. Papadimitriou CH, Steiglitz K (1998) Combinatorial optimization: algorithms and complexity. Courier Corporation

  51. Parr JM, Forrester AI, Keane AJ, Holden CM (2012) Enhancing infill sampling criteria for surrogate-based constrained optimization. J Comput Methods Sci Eng 12(1–2):25–45

    MathSciNet  MATH  Google Scholar 

  52. Ponweiser W, Wagner T, Vincze M (2008) Clustered multiple generalized expected improvement: a novel infill sampling criterion for surrogate models. In: 2008 IEEE congress on evolutionary computation (IEEE World Congress on Computational Intelligence), pp 3515–3522. IEEE

  53. Regis RG (2013) Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans Evol Comput 18(3):326–347

    Article  Google Scholar 

  54. Runarsson TP (2004) Constrained evolutionary optimization by approximate ranking and surrogate models. In: International conference on parallel problem solving from nature, pp 401–410. Springer

  55. Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294

    Article  Google Scholar 

  56. Sahni S (1975) Approximate algorithms for the 0/1 knapsack problem. J ACM (JACM) 22(1):115–124

    Article  MathSciNet  MATH  Google Scholar 

  57. Shen W, Xu B, Huang JP (2011) An improved genetic algorithm for 0-1 knapsack problems. In: International conference on networking and distributed computing

  58. Singh HK, Ray T, Smith W (2010) Surrogate assisted simulated annealing (sasa) for constrained multi-objective optimization. In: IEEE congress on evolutionary computation, pp 1–8. IEEE

  59. Stojanovic V, He S, Zhang B (2020) State and parameter joint estimation of linear stochastic systems in presence of faults and non-gaussian noises. Int J Robust Nonlinear Control (4)

  60. Tao H, Wang P, Chen Y, Stojanovic V, Yang H (2020) An unsupervised fault diagnosis method for rolling bearing using stft and generative neural networks. J Franklin Inst 357(11)

  61. Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013) Auto-weka: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, pp 847–855

  62. Verel S, Derbel B, Liefooghe A, Aguirre H, Tanaka K (2018) A surrogate model based on walsh decomposition for pseudo-boolean functions. In: International conference on parallel problem solving from nature, pp 181–193. Springer

  63. Voutchkov I, Keane A, Bhaskar A, Olsen TM (2005) Weld sequence optimization: the use of surrogate models for solving sequential combinatorial problems. Comput Methods Appl Mech Eng 194(30–33):3535–3551

    Article  MATH  Google Scholar 

  64. Wang H, Jin Y (2018) A random forest-assisted evolutionary algorithm for data-driven constrained multiobjective combinatorial optimization of trauma systems. IEEE Trans Cybern 50(2):536–549

    Article  Google Scholar 

  65. Wang H, Jin Y, Doherty J (2017) Committee-based active learning for surrogate-assisted particle swarm optimization of expensive problems. IEEE Trans Cybern 47(9):2664–2677

    Article  Google Scholar 

  66. Wang H, Jin Y, Jansen JO (2016) Data-driven surrogate-assisted multiobjective evolutionary optimization of a trauma system. IEEE Trans Evol Comput 20(6):939–952

    Article  Google Scholar 

  67. Wang H, Jin Y, Sun C, Doherty J (2018) Offline data-driven evolutionary optimization using selective surrogate ensembles. IEEE Trans Evol Comput 23(2):203–216

    Article  Google Scholar 

  68. Wang Y, Yin DQ, Yang S, Sun G (2018) Global and local surrogate-assisted differential evolution for expensive constrained optimization problems with inequality constraints. IEEE Trans Cybern 49(5):1642–1656

    Article  Google Scholar 

  69. Zaefferer M (2018) Surrogate models for discrete optimization problems. Ph.D. thesis

  70. Zaefferer M, Stork J, Friese M, Fischbach A, Naujoks B, Bartz-Beielstein T (2014) Efficient global optimization for combinatorial problems. In: Proceedings of the 2014 annual conference on genetic and evolutionary computation, pp 871–878

  71. Zapotecas Martínez S, Coello Coello CA (2013) Moea/d assisted by rbf networks for expensive multi-objective optimization problems. In: Proceedings of the 15th annual conference on Genetic and evolutionary computation, pp 1405–1412

  72. Zhou Z, Ong YS, Nguyen MH, Lim D (2005) A study on polynomial regression and gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm. In: 2005 IEEE congress on evolutionary computation, vol 3, pp 2832–2839. IEEE

Download references

Author information

Authors and Affiliations

  1. School of Artificial Intelligence, Xidian University, Xi’an, 710071, Shaanxi, China

    Lei Han & Handing Wang

Authors
  1. Lei Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Handing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Handing Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China (No. 61976165).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Han, L., Wang, H. A random forest assisted evolutionary algorithm using competitive neighborhood search for expensive constrained combinatorial optimization. Memetic Comp. 13, 19–30 (2021). https://doi.org/10.1007/s12293-021-00326-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-021-00326-9

Keywords

Search

Navigation