Skip to main content

Advertisement

SpringerLink
Log in

Classification- and Regression-Assisted Differential Evolution for Computationally Expensive Problems

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Differential Evolution (DE) has been well accepted as an effective evolutionary optimization technique. However, it usually involves a large number of fitness evaluations to obtain a satisfactory solution. This disadvantage severely restricts its application to computationally expensive problems, for which a single fitness evaluation can be highly time-consuming. In the past decade, a lot of investigations have been conducted to incorporate a surrogate model into an evolutionary algorithm (EA) to alleviate its computational burden in this scenario. However, only limited work was devoted to DE. More importantly, although various types of surrogate models, such as regression, ranking, and classification models, have been investigated separately, none of them consistently outperforms others. In this paper, we propose to construct a surrogate model by combining both regression and classification techniques. It is shown that due to the specific selection strategy of DE, a synergy can be established between these two types of models, and leads to a surrogate model that is more appropriate for DE. A novel surrogate model-assisted DE, named Classification- and Regression-Assisted DE (CRADE) is proposed on this basis. Experimental studies are carried out on a set of 16 benchmark functions, and CRADE has shown significant superiority over DE-assisted with only regression or classification models. Further comparison to three state-of-the-art DE variants, i.e., DE with global and local neighborhoods (DEGL), JADE, and composite DE (CoDE), also demonstrates the superiority of CRADE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Storn R, Price K. Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 1997, 11(4): 341–359.

    Article  MathSciNet  MATH  Google Scholar 

  2. Price K, Storn R M, Lampinen J A. Differential Evolution: A Practical Approach to Global Optimization. New York, USA: Springer-Verlag, 2005.

  3. Jin Y, Olhofer M, Sendhoff B. Managing approximate models in evolutionary aerodynamic design optimization. In Proc. the 2001 IEEE Congress on Evolutionary Computation (CEC2001), Seoul, Korea, May 2001, pp.592–599.

  4. Ong Y S, Nair P B, Keane A J. Evolutionary optimization of computationally expensive problems via surrogate modeling. AIAA Journal, 2003, 41(4): 687–696.

    Article  Google Scholar 

  5. Zhang P, Yao X, Jia L, Sendhoff B, Schnier T. Target shape design optimization by evolving splines. In Proc. CEC2007, Singapore, Sept. 2007, pp.2009–2016.

  6. Farina M, Sykulski J. Comparative study of evolution strategies combined with approximation techniques for practical electromagnetic optimization problems. IEEE Transactions on Magnetics, 2002, 37(5): 3216–3220.

    Article  Google Scholar 

  7. Hajela P, Lee J. Genetic algorithms in multidisciplinary rotor blade design. In Proc. the 36th Conference on Structures, Structural Dynamics, and Materials, New Orleans, USA, April 1995, pp.2187–2197.

  8. Shi L, Rasheed K. A survey of fitness approximation methods applied in evolutionary algorithms. Computational Intelligence in Expensive Optimization Problems, 2010, 2(1): 3–28.

    Article  Google Scholar 

  9. Jin Y, Olhofer M, Sendhoff B. A framework for evolutionary optimization with approximate fitness functions. IEEE Transactions on Evolutionary Computation, 2002, 6(5): 481–494.

    Article  Google Scholar 

  10. Jin Y. A comprehensive survey of fitness approximation in evolutionary computation. Soft Computing, 2005, 9(1): 3–12.

    Article  Google Scholar 

  11. Jin Y, Olhofer M, Sendhoff B. On evolutionary optimization with approximate fitness functions. In Proc. Genetic and Evolutionary Computation Conference, Las Vegas, Nevada, USA, July 2000, pp.786–793.

  12. Buche D, Schraudolph N N, Koumoutsakos P. Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, 2005, 35(2): 183–194.

    Article  Google Scholar 

  13. Runarsson T. Ordinal regression in evolutionary computation. In Proc. the 9th Int. Conf. Parallel Problem Solving from Nature-PPSN IX, Reykjavik, Iceland, Sept. 2006, pp.1048–1057.

  14. Loshchilov I, Schoenauer M, Sebag M. Comparison-based optimizers need comparison-based surrogates. In Proc. the 11th Int. Conf. Parallel Problem Solving from Nature-PPSN XI, Krakov, Poland, September 2010, pp.364–373.

  15. Handoko S D, Kwoh C K, Ong Y S. Using classification for constrained memetic algorithm: A new paradigm. In Proc. IEEE International Conference on Systems, Man and Cybernetics. Suntec, Singapore, Oct. 2008, pp.547–552.

  16. Handoko S D, Kwoh C K, Ong Y S. Classification-assisted memetic algorithms for equality-constrained optimization problems. In Proc. the 22nd AI, Melbourne, Australia, May 2009, pp.391–400.

  17. Lim D, Ong Y S, Setiawan R, Idris M. Classifier-assisted constrained evolutionary optimization for automated geometry selection of orthodontic retraction spring. In Proc. the 2010 IEEE Congress on Evolutionary Computation (CEC2010), Barcelona, Spain, July 2010, pp.1–8.

  18. Handoko S D, Kwoh C K, Ong Y S. Feasibility structure modeling: An effective chaperone for constrained memetic algorithms. IEEE Transactions on Evolutionary Computation, 2010, 14(5): 740–758.

    Article  Google Scholar 

  19. Yang Z, Tang K, Yao X. Self-adaptive differential evolution with neighborhood search. In Proc. the 2008 IEEE Congress on Evolutionary Computation (CEC2008), Hong kong, China, June 2008, pp. 1110–1116.

  20. Das S, Abraham A, Chakraborty U K, Konar A. Differential evolution using a neighborhood-based mutation operator. IEEE Transactions on Evolutionary Computation, 2009, 13(3): 526–553.

    Article  Google Scholar 

  21. Zhang J, Sanderson A C. JADE: Adaptive differential evolution with optional external archive. IEEE Transactions on Evolutionary Computation, 2009, 13(5): 945–958.

    Article  Google Scholar 

  22. Yang Z, Tang K, Yao X. Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft Computing, 2011, 15(11): 2141–2155.

    Article  Google Scholar 

  23. Wang Y, Cai Z, Zhang Q. Differential evolution with composite trial vector generation strategies and control parameters. IEEE Transactions on Evolutionary Computation, 2011, 15(1): 55–66.

    Article  MathSciNet  Google Scholar 

  24. Wang Y, Cai Z, Zhang Q. Enhancing the search ability of differential evolution through orthogonal crossover. Information Sciences, 2012, 185(1): 153–177.

    Article  MathSciNet  Google Scholar 

  25. Zhang J, Sanderson A. DE-AEC: A differential evolution algorithm based on adaptive evolution control. In Proc. the 2007 IEEE Congress on Evolutionary Computation (CEC2007), Singapore, Sept. 2007, pp.3824–3830.

  26. Wang Y, Shi Y, Yue B, Teng H. An efficient differential evolution algorithm with approximate fitness functions using neural networks. In Proc. the 2010 Int. Conf. Artificial Intelligence and Computational Intelligence, Part 2, Oct. 2010, pp.334–341.

  27. Lu X, Tang K, Yao, X. Classification-assisted differential evolution for computationally expensive problems. In Proc. the 2011 IEEE Congress on Evolutionary Computation (CEC2011), New Orleans, USA, June 2011, pp.1986–1993.

  28. Lim D, Jin Y, Ong Y S, Sendhoff B. Generalizing surrogateassisted evolutionary computation. IEEE Transactions on Evolutionary Computation, 2010, 14(3): 329–355.

    Article  Google Scholar 

  29. Suganthan P N, Hansen N, Liang J J, Deb K, Chen Y P, Auger A, Tiwari S. Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical Report, Nanyang Technological University, Singapore, and KanGAL Report, Kanpur Genetic Algorithms Laboratory, IITkanpur, 2005.

  30. Tang K, Yao X, Suganthan P N, MacNish C, Chen Y P, Chen C M, Yang Z. Benchmark functions for the CEC2008 special session and competition on large scale global optimization. Technical Report, Nature Inspired Comput. Applicat. Lab., USTC, China, 2007. http://nical.ustc.edu.cn/cec08ss.php

  31. Duin R, Juszczak P, Paclik P, Pekalska E, Ridder D de, Tax D M J, Verzakov S. PRTools 4.1, a matlab toolbox for pattern recognition. Delft University of Technology, 2007.

  32. Yu H, Kim S. SVM tutorial: Classification, regression, and ranking. In Handbook of Natural Computing, Rozenderg G, Bäck T, Kok J (eds.), Springer 2009.

  33. Gunn S. Support vector machines for classification and regression. Technical Report, University of Southampton, 1998.

Download references

Author information

Authors and Affiliations

  1. Nature Inspired Computation and Applications Laboratory, School of Computer Science and Technology University of Science and Technology of China, Hefei, 230027, China

    Xiao-Fen Lu & Ke Tang (Member, IEEE)

Authors
  1. Xiao-Fen Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ke Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiao-Fen Lu.

Additional information

This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 61028009, U0835002, and 61175065, the Natural Science Foundation of Anhui Province of China under Grant No. 1108085J16, and the Open Research Fund of State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing of China under Grant No. 10R04.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(DOC 15.8 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, XF., Tang, K. Classification- and Regression-Assisted Differential Evolution for Computationally Expensive Problems. J. Comput. Sci. Technol. 27, 1024–1034 (2012). https://doi.org/10.1007/s11390-012-1282-4

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-012-1282-4

Keywords

Search

Navigation