Abstract
One of the key features in software-defined networking (SDN) with a multi-controller environment is the controller placement problem (CPP) that aims to find the number of controllers, controller placements and controller assignment. Solving the CPP in software-defined multihop wireless networking (SDMWN) has a significant impact on the generated control overhead. In SDMWN, devices use unreliable and shared multihop wireless communications without the help of any infrastructure, such as a base station or an access point. Various algorithms have been proposed to find near-optimal solutions for the CPP. Deep reinforcement learning (DRL) becomes popular in various fields and a few studies have also investigated using DRL for the CPP in wired networks and infrastructure-based wireless networks. However, DRL has not been researched for the CPP in SDMWN. Hence, in this paper, the potential of using DRL to the CPP in SDMWN for a given number of controllers is investigated to minimize the generated control overhead referred to as the network cost. The results show that the adapted DRL is able to find controller placements and assign the controllers to devices such that the obtained network cost and the average number of hops among devices and their assigned controllers, as well as the average number of hops among different controllers in the network, are close to those obtained from the optimal solutions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Heller, B., Sherwood, R., McKeown, N.: The controller placement problem. In: Proceedings of the 1st Workshop on Hot topics in Software Defined Networks, pp. 7–12 (2012)
Das, T., Sridharan, V., Gurusamy, M.: A survey on controller placement in SDN. IEEE Commun. Surv. Tutor. 22(1), 472–503 (2020)
Isong, B., Molose, R.R.S., Abu-Mahfouz, A.M., Dladlu, N.: Comprehensive review of SDN controller placement strategies. IEEE Access 8, 170070–170092 (2020)
Yao, G., Bi, J., Li, Y., Guo, L.: On the capacitated controller placement problem in software defined networks. IEEE Commun. Lett. 18,(8), 1339–1342 (2014)
Zahmatkesh, A., Kunz, T., Lung, C.-H.: Cost-effective controller placement problem for software defined multihop wireless networks. In: Foschini, L., El Kamili, M. (eds.) ADHOCNETS 2020. LNICSSITE, vol. 345, pp. 130–146. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67369-7_10
Sallahi, A., St-Hilaire, M.: Expansion model for the controller placement problem in software defined networks. IEEE Commun. Lett. 21(2), 274–277 (2017)
Zahmatkesh, A., Kunz, T.: Software defined multihop wireless networks: promises and challenges. J. Commun. Netw. 19(6), 546–554 (2017)
Kumari, A., Sairam, A.S.: Controller placement problem in software-defined networking: a survey. Networks, 1–29 (2021)
Roscher, R., Bohn, B., Duarte, M.F., Garcke, J.: Explainable machine learning for scientific insights and discoveries. IEEE Access 8, 42200–42216 (2020)
Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34, 26–38 (2017)
Fran cois-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An Introduction to Deep Reinforcement Learning. Now Publishing Inc., Boston (2018)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Mostafaei, H., Menth, M., Obaidat, M.S.: A learning automaton-based controller placement algorithm for software-defined networks. In: 2018 IEEE Global Communications Conference (GLOBECOM), pp. 1–6 (2018)
Wu, Y., Zhou, S., Wei, Y., Leng, S.: Deep reinforcement learning for controller placement in software defined network. In: IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp. 1254–1259 (2020)
Yuan, T., Neto, W.d.R., Rothenberg, C.E., Obraczka, K., Barakat, C., Turletti, T.: Dynamic controller assignment in software defined internet of vehicles through multi-agent deep reinforcement learning. In: IEEE Transactions on Network and Service Management, vol. 18, no. 1, pp. 585–596 (2021)
Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Wang, G., Zhao, Y., Huang, J., Duan, Q., Li, J.: A K-means-based network partition algorithm for controller placement in software defined network. In: 2016 IEEE International Conference on Communications (ICC), pp. 1–6 (2016)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agentactor-critic for mixed cooperative-competitive environments. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, Red Hook, NY, USA, pp. 6382–6393, Curran Associates Inc. (2017)
Fourer, R., Gay, D.M., Kernighan, B.: Algorithms and model formulations in mathematical programming. AMPL: A Mathematical Programming Language, pp. 150–151. Springer, Heidelberg (1989). https://doi.org/10.1007/978-3-642-83724-1
Gropp, W., More, J.J.: Optimization environments and the NEOS server. In: Buhmann, M.D., Iserles, A. (eds.) Approximation Theory and Optimization, pp. 167–182. Cambridge University Press, Cambridge (1997)
CPLEX: IBM’s Linear Programming Solver. http://www.ilog.com/product/cplex/. Accessed May 2021
NEOS Server for CPLEX/AMPL. https://neos-server.org/neos/solvers/lp:CPLEX/AMPL.html. Accessed May 2021
Acknowledgment
The authors acknowledge the support from the Natural Sciences and Engineering Research Council of Canada (NSERC) through the Discovery Grant program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Zahmatkesh, A., Lung, CH. (2022). Deep Reinforcement Learning for Cost-Effective Controller Placement in Software-Defined Multihop Wireless Networking. In: Renault, É., Boumerdassi, S., Mühlethaler, P. (eds) Machine Learning for Networking. MLN 2021. Lecture Notes in Computer Science, vol 13175. Springer, Cham. https://doi.org/10.1007/978-3-030-98978-1_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-98978-1_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98977-4
Online ISBN: 978-3-030-98978-1
eBook Packages: Computer ScienceComputer Science (R0)