Abstract
In this paper, we propose a reinforcement learning based Q-learning routing mechanism for unicast routing in Software-defined Networks (SDN). The main objective is to minimize the delay experienced by unicast traffic as it traverses the network. We consider unicast traffic arriving with a Poisson arrival rate at each switch with exponentially distributed service times. We consider M/M/1 system for the forwarding tables in network switches (OpenFlow switches), and model the delay function. Q-learning mechanism is adopted for dynamically updating the routing paths, based on the derived delay function. Efficacy of the proposed routing algorithm has been evaluated using a component-based framework i.e., OMNET++ simulator. The proposed routing scheme was compared to the legacy shortest path routing mechanism (Dijkstra’s algorithm). The proposed scheme effectively reduces the delay for unicast traffic. We further proceed by exploiting system parameters, and observe network behavior under the proposed scheme.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kreutz, D., Ramos, F.M., Verissimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig, S.: Software-defined networking: a comprehensive survey. Proc. IEEE 103(1), 14–76 (2015). https://doi.org/10.1109/JPROC.2014.2371999
Open Networking Foundation (ONF). https://www.opennetworking.org/. Accessed 16 Nov 2018
Huang, M., Liang, W., Xu, Z., Xu, W., Guo, S., Xu, Y.: Online unicasting and multicasting in software-defined networks. Comput. Netw. 132, 26–39 (2018)
Shafiee, M., Ghaderi, J.: A simple congestion-aware algorithm for load balancing in datacenter networks. IEEE/ACM Trans. Netw. 25(6), 3670–3682 (2017). https://doi.org/10.1109/TNET.2017.2751251
Agarwal, S., Kodialam, M., Lakshman, T.V.: Traffic engineering in software defined networks. In: 2013 IEEE Proceedings of International Conference on Computer Communications, IEEE INFOCOM, Turin, Italy, pp. 2211–2219. IEEE, April 2013
Forster, A., Murphy, A.L.: FROMS: feedback routing for optimizing multiple sinks in WSN with reinforcement learning. In: 3rd International Conference on Intelligent Sensors, Sensor Networks and Information, ISSNIP, Melbourne, Australia, pp. 371–376. IEEE, December 2007
Haeri, S.: Applications of reinforcement learning to routing and virtualization in computer networks. Doctoral dissertation, Applied Sciences: School of Engineering Science, Fraser University, Burnaby, Canada (2016)
Martins, J.L., Campos, N.: Short-sighted routing, or when less is more. IEEE Commun. Mag. 54(10), 82–88 (2016)
Watkins, C.J.C.H.: Learning from delayed rewards. Doctoral dissertation, King’s College, Cambridge (1989)
Riedmiller, M.: Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method. In: European Conference on Machine Learning, pp. 317–328. Springer, Heidelberg, October 2005
Cohen, R., Lewin-Eytan, L., Naor, J.S., Raz, D.: On the effect of forwarding table size on SDN network utilization. In: 2014 IEEE Proceedings of International Conference on Computer Communications, IEEE INFOCOM, , Toronto, Canada, pp. 1734–1742. IEEE (2014)
Karlin, S.: A First Course in Stochastic Processes. Academic Press, Cambridge (2014)
Kleinrock, L.: Queueing Systems, Volume 2: Computer Applications. Wiley, New York (1976)
Acknowledgment
This work was supported by Institute for Information & communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (B0190-18-2013, Development of Access Technology Agnostic Next-Generation Networking Technology for Wired-Wireless Converged Networks).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Mahboob, T., Jung, Y.R., Chung, M.Y. (2019). Optimized Routing in Software Defined Networks – A Reinforcement Learning Approach. In: Lee, S., Ismail, R., Choo, H. (eds) Proceedings of the 13th International Conference on Ubiquitous Information Management and Communication (IMCOM) 2019. IMCOM 2019. Advances in Intelligent Systems and Computing, vol 935. Springer, Cham. https://doi.org/10.1007/978-3-030-19063-7_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-19063-7_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-19062-0
Online ISBN: 978-3-030-19063-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)