Abstract
In this paper, we combine multiagent reinforcement learning (MARL) with grid-based Pareto local search for combinatorial multiobjective optimization problems (CMOPs). In the multiagent system, each agent (grid) maintains at most one solution after the MARL-guided selection for local search. MARL adaptively adjusts the selection strategy for conducting better collaborative Pareto local search. In the experimental studies, the MARL-guided grid Pareto local search (MARL-GPLS) is compared with the Pareto local search (PLS), two decomposition-based multiobjective local search approaches, a grid-based approach (\(\epsilon \)-MOEA), and one state-of-the-art hybrid approach on benchmark CMOPs. The results show that the MARL-GPLS outperforms the other six algorithms on most instances.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Afanasyeva, A., Buzdalov, M.: Choosing best fitness function with reinforcement learning. In: 2011 10th International Conference on Machine Learning and Applications and Workshops, vol. 2, pp. 354–357, December 2011. https://doi.org/10.1109/ICMLA.2011.163
Bhowmik, P., Rakshit, P., Konar, A., Kim, E., Nagar, A.K.: DE-TDQL: an adaptive memetic algorithm. In: 2012 IEEE Congress on Evolutionary Computation, pp. 1–8, June 2012. https://doi.org/10.1109/CEC.2012.6256573
Busoniu, L., Babuska, R., De Schutter, B.: A comprehensive survey of multiagent reinforcement learning. IEEE Trans. Syst. Man Cybern. Part C (App. Rev.) 38(2), 156–172 (2008). https://doi.org/10.1109/TSMCC.2007.913919
Cai, X., Sun, H., Zhang, Q., Huang, Y.: A grid weighted sum pareto local search for combinatorial multi and many-objective optimization. IEEE Trans. Cybern. 49(9), 3586–3598 (2019). https://doi.org/10.1109/TCYB.2018.2849403
Hisao, I., Tadahiko, M.: Multi-objective genetic local search algorithm. In: Fukuda, T., Furuhashi, T. (eds.) Proceedings of the 1996 International Conference on Evolutionary Computation, Nagoya, Japan, pp. 119–124. IEEE (1996)
Jaakkola, T., Jordan, M.I., Singh, S.P.: On the convergence of stochastic iterative dynamic programming algorithms. Neural Comput. 6(6), 1185–1201 (2014)
Jaszkiewicz, A.: On the performance of multiple-objective genetic local search on the 0/1 knapsack problem - a comparative experiment. IEEE Trans. Evol. Comput. 6(4), 402–412 (2002). https://doi.org/10.1109/TEVC.2002.802873
Juan, J.D., Zhang, Y., Enrique, A., Mark, H., Antonio, J.N.: A study of the bi-objective next release problem. Empirical Softw. Eng. 16(1), 29–60 (2011)
Karafotias, G., Eiben, A.E., Hoogendoorn, M.: Generic parameter control with reinforcement learning. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1319–1326 (2014)
Ke, L., Zhang, Q., Battiti, R.: Hybridization of decomposition and local search for multiobjective optimization. IEEE Trans. Cybern. 44(10), 1808–1820 (2014). https://doi.org/10.1109/TCYB.2013.2295886
Liu, H., Gu, F., Zhang, Q.: Decomposition of a multiobjective optimization problem into a number of simple multiobjective subproblems. IEEE Trans. Evol. Comput. 18(3), 450–455 (2014). https://doi.org/10.1109/TEVC.2013.2281533
Lust, T., Teghem, J.: Two-phase pareto local search for the biobjective traveling salesman problem. J. Heuristics 16(3), 475–510 (2010). https://doi.org/10.1007/s10732-009-9103-910.1007/s10732-009-9103-9
Matignon, L., Laurent, G.J., Fort-Piat, N.L.: Independent reinforcement learners in cooperative Markov games: a survey regarding coordination problems. Knowl. Eng. Rev. 27(01), 1–31 (2012)
Sierra, M.R., Coello, C.A.: A new multi-objective particle swarm optimizer with improved selection and diversity mechanisms (2004)
Muller, S.D., Schraudolph, N.N., Koumoutsakos, P.D.: Step size adaptation in evolution strategies using reinforcement learning. In: Proceedings of the 2002 Congress on Evolutionary Computation, CEC 2002 (Cat. No. 02TH8600), vol. 1, pp. 151–156, May 2002. https://doi.org/10.1109/CEC.2002.1006225
Papadimitriou, C.H., Steiglitz, K.: Combinatorial Optimization: Algorithms and Complexity. Dover, New York (1998)
Paquete, L.F.: Stochastic Local Search Algorithms for Multiobjective Combinatorial Optimizations: Methods and Analysis. IOS Press, Inc., Amsterdam (2006)
Reinelt, G.: TSPLIB: traveling salesman problem library. Orsa J. Comput. 3(1), 376–384 (1991)
Shim, V.A., Tan, K.C., Cheong, C.Y.: A hybrid estimation of distribution algorithm with decomposition for solving the multiobjective multiple traveling salesman problem. IEEE Trans. Syst. Man Cybern. Part C 42(5), 682–691 (2012)
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, vol. 135. MIT Press, Cambridge (1998)
Yang, S., Li, M., Liu, X., Zheng, J.: A grid-based evolutionary algorithm for many-objective optimization. IEEE Trans. Evol. Comput. 17(5), 721–736 (2013)
Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999). https://doi.org/10.1109/4235.797969
Acknowledgment
This work was supported in part by the Aeronautical Science Foundation of China under grant 20175552042, by the National Natural Science Foundation of China (NSFC) under grant 61300159, by the Natural Science Foundation of Jiangsu Province of China under grant BK20181288 and by China Postdoctoral Science Foundation under grant 2015M571751.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Gu, Y., Sun, Q., Cai, X. (2020). Multiagent Reinforcement Learning for Combinatorial Optimization. In: Zhang, H., Zhang, Z., Wu, Z., Hao, T. (eds) Neural Computing for Advanced Applications. NCAA 2020. Communications in Computer and Information Science, vol 1265. Springer, Singapore. https://doi.org/10.1007/978-981-15-7670-6_3
Download citation
DOI: https://doi.org/10.1007/978-981-15-7670-6_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7669-0
Online ISBN: 978-981-15-7670-6
eBook Packages: Computer ScienceComputer Science (R0)