Skip to main content
SpringerLink
Log in

Reinforcement learning aided parameter control in multi-objective evolutionary algorithm based on decomposition

  • Regular Paper
  • Published:
Progress in Artificial Intelligence Aims and scope Submit manuscript

Abstract

Multi-objective evolutionary algorithm based on decomposition (MOEA/D) has been successfully applied in solving multi-objective optimization problems. However, the performance of MOEA/D could be severely influenced by its parameter settings. In this paper, we introduce reinforcement learning into MOEA/D as a generic parameter controller. The resulting algorithm, reinforcement learning enhanced MOEA/D (RL-MOEA/D), is used to adaptively control the neighborhood size T and the differential evolutionary operators used in MOEA/D. RL-MOEA/D is first compared with MOEA/D with a random parameter control mechanism and MOEA/Ds with some fixed parameter settings on ten widely used multi-objective test instances. Then, RL-MOEA/D is compared with FRRMAB to show the effectiveness of the proposed algorithm. The experimental results indicate that RL-MOEA/D is very competitive. Finally, the characteristics of RL-MOEA/D are studied.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Bosman, P.A.N., Thierens, D.: The balance between proximity and diversity in multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 7(2), 174–188 (2003). https://doi.org/10.1109/TEVC.2003.810761

    Article  Google Scholar 

  2. Consoli, P.A., Mei, Y., Minku, L.L., Yao, X.: Dynamic selection of evolutionary operators based on online learning and fitness landscape analysis. Soft. Comput. 20(10), 3889–3914 (2016)

    Article  Google Scholar 

  3. Eiben, A.E., Horvath, M., Kowalczyk, W., Schut, M.C.: Reinforcement learning for online control of evolutionary algorithms. In: International Workshop on Engineering Self-Organising Applications, pp. 151–160 (2006)

  4. Ginley, B.M., Maher, J., O’Riordan, C., Morgan, F.: Maintaining healthy population diversity using adaptive crossover, mutation, and selection. IEEE Trans. Evol. Comput. 15(5), 692–714 (2011). https://doi.org/10.1109/tevc.2010.2046173

    Article  Google Scholar 

  5. Goncalves, R.A., Almeida, C.P., Pozo, A.: Upper confidence bound (UCB) algorithms for adaptive operator selection in MOEA/D. In: Evolutionary Multi-criterion Optimization, Lecture Notes in Computer Science, pp. 411–425. Springer, Cham (2015)

    Google Scholar 

  6. Karafotias, G., Eiben, A.E., Hoogendoorn, M.: Generic parameter control with reinforcement learning. In: Conference on Genetic and Evolutionary Computation, pp. 1319–1326 (2014)

  7. Karafotias, G., Hoogendoorn, M., Eiben, A.: Parameter control in evolutionary algorithms: trends and challenges. IEEE Trans. Evol. Comput. 19(2), 167–187 (2015). https://doi.org/10.1109/TEVC.2014.2308294

    Article  Google Scholar 

  8. Karafotias, G., Hoogendoorn, M., Eiben, A.E.: Evaluating reward definitions for parameter control. In: European Conference on the Applications of Evolutionary Computation, pp. 667–680 (2015)

    Google Scholar 

  9. Li, K., Fialho, A., Kwong, S., Zhang, Q.: Adaptive operator selection with bandits for a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 18(1), 114–130 (2014). https://doi.org/10.1109/TEVC.2013.2239648

    Article  Google Scholar 

  10. Lin, Q., Liu, Z., Yan, Q., Du, Z., Coello, C.A.C., Liang, Z., Wang, W., Chen, J.: Adaptive composite operator selection and parameter control for multiobjective evolutionary algorithm. Inf. Sci. 339, 332–352 (2016). https://doi.org/10.1016/j.ins.2015.12.022

    Article  Google Scholar 

  11. Muller, S.D., Schraudolph, N.N., Koumoutsakos, P.D.: Step size adaptation in evolution strategies using reinforcement learning. In: Proceedings of the World on Congress on Computational Intelligence, vol. 1, pp. 151–156. IEEE Computer Society, Los Alamitos, CA, USA (2002). https://doi.org/10.1109/CEC.2002.1006225

  12. Rost, A., Petrova, I., Buzdalova, A.: Adaptive parameter selection in evolutionary algorithms by reinforcement learning with dynamic discretization of parameter range. In: Genetic and Evolutionary Computation Conference Companion, pp. 141–142 (2016)

  13. Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  14. Sutton, R.S.: Learning to predict by the methods of temporal differences. Mach. Learn. 3(1), 9–44 (1988). https://doi.org/10.1023/A:1022633531479

    Article  Google Scholar 

  15. Venske, S.M., Gonçalves, R.A., Delgado, M.R.: ADEMO/D: multiobjective optimization by an adaptive differential evolution algorithm. Neurocomputing 127(127), 65–77 (2014)

    Article  Google Scholar 

  16. Wong, Y.Y., Lee, K.H., Leung, K.S., Ho, C.W.: A novel approach in parameter adaptation and diversity maintenance for genetic algorithms. Soft. Comput. 7(8), 506–515 (2003)

    Article  Google Scholar 

  17. Zhang, H., Lu, J.: Adaptive evolutionary programming based on reinforcement learning. Inf. Sci. 178(4), 971–984 (2008). https://doi.org/10.1016/j.ins.2007.09.026

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhang, J., Sanderson, A.C.: JADE: adaptive differential evolution with optional external archive. IEEE Trans. Evol. Comput. 13(5), 945–958 (2009)

    Article  Google Scholar 

  19. Zhang, Q., Li, H.: MOEA/D: a multiobjective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007). https://doi.org/10.1109/TEVC.2007.892759

    Article  Google Scholar 

  20. Zhang, Q., Liu, W., Li, H.: The performance of a new version of MOEA/D on CEC09 unconstrained MOP test instances. In: IEEE Congress on Evolutionary Computation, 2009. CEC ’09, pp. 203–208 (2009). https://doi.org/10.1109/CEC.2009.4982949

  21. Zhang, Q., Zhou, A., Zhao, S., Suganthan, P.N., Liu, W., Tiwari, S.: Multiobjective optimization test instances for the CEC 2009 special session and competition. University of Essex, Colchester, UK and Nanyang Technological University, Singapore, Special Session on Performance Assessment of Multi-Objective Optimization Algorithms, Technical Report (2008)

  22. Zhao, S.Z., Suganthan, P., Zhang, Q.: Decomposition-based multiobjective evolutionary algorithm with an ensemble of neighborhood sizes. IEEE Trans. Evol. Comput. 16(3), 442–446 (2012). https://doi.org/10.1109/TEVC.2011.2166159

    Article  Google Scholar 

  23. Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999). https://doi.org/10.1109/4235.797969

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under grant number 61571346. The authors would like to thank the reviewers and the editor for their comments for improving this paper.

Author information

Authors and Affiliations

  1. School of Aerospace Science and Technology, Xidian University, Xi’an, 710071, China

    Weikang Ning, Baolong Guo, Xinxing Guo, Cheng Li & Yunyi Yan

Authors
  1. Weikang Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Baolong Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinxing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yunyi Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weikang Ning.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ning, W., Guo, B., Guo, X. et al. Reinforcement learning aided parameter control in multi-objective evolutionary algorithm based on decomposition. Prog Artif Intell 7, 385–398 (2018). https://doi.org/10.1007/s13748-018-0155-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13748-018-0155-7

Keywords

Search

Navigation