Abstract
As Internet usage increases, new, smarter, networking methods are required to enhance or maintain Quality of Service (QoS). One method, Software-Defined Networking (SDN) offers many advantages by separating the Forwarding and Control Planes. However, heuristic routing algorithms employed by SDN, such as Shortest Path, are not always suited for QoS-based pathfinding. This paper introduces a new Q-Routing algorithm that separates training and pathfinding, utilising two network metrics - latency and bandwidth - instead of latency alone. Two versions of this algorithm are employed, a static and a dynamic version where additional re-training is undertaken to allow Q-Routing to adapt to changing network environments. Both are tested on different size mesh topologies. The results show that static and dynamic Q-Routing are faster at pathfinding compared to K-Shortest Path and on average, find equally good routes.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Cisco: VNI Complete Forecast Highlights (2021). https://www.cisco.com/c/m/en_us/solutions/service-provider/vni-forecast-highlights.html
Teh, Y.-C.: Lockdown Leads to Surge in TV Screen Time and Streaming (2020). https://www.ofcom.org.uk/about-ofcom/latest/features-and-news/lockdown-leads-to-surge-in-tv-screen-time-and-streaming
Bellman, R.: On a Routing Problem. Q. Appl. Maths. 16, 87–90 (1958)
Ford, L.: Network flow theory. Technical report. Santa Monica, USA (1956)
Dijkstra, E.: A note on two problems in connections with graphs. Numer. Math. (1959)
Watkins, C.J.C.H.: Learning from delayed rewards. Ph.D. thesis, Cambridge, UK (1989). http://www.cs.rhul.ac.uk/~chrisw/new_thesis.pdf
While, R.: Artificial intelligence: reinforcement learning (2019). http://teaching.csse.uwa.edu.au/courses/CITS4211/Lectures/wk7.pdf
Huang, Y., Tan, T., Wang, N., Chen, Y., Li, Y.: Resource allocation For D2D communications with a novel distributed Q-learning algorithm in heterogeneous networks. In: International Conference on Machine Learning and Cybernetics (ICMLC), pp. 533–537. IEEE (2018). https://doi.org/10.1109/ICMLC.2018.8526955
Wen, S., Chen, J., Li, Z., Rad, A.B.: Fuzzy Q-learning obstacle avoidance algorithm of humanoid robot in unknown environment. In: 37th Chinese Control Conference (CCC), pp. 5186–5190. IEEE (2018)
Boyan, J., Littman, M.: Packet routing in dynamically changing networks: a reinforcement learning approach. In: Proceedings of 6th International Conference on Neural Information Processing Systems, Denver, Colorado, USA, pp. 671–678 (1993)
Kavalerov, A., Shilova, Y., Likhacheva, Y.: Adaptive Q-routing with random echo and route memory. In: 20th Conference of Open Innovations Association (FRUCT), pp. 138–145. IEEE (2018)
Kumar, S., Miikkulainen, R.: Dual reinforcement Q-routing: an on-line adaptive routing algorithm. In: Smart Engineering Systems: Neural Networks, Fuzzy Logic, Data Mining, and Evolutionary Programming, USA (1997). http://nn.cs.utexas.edu/?kumar:annie97
Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in UbuntuNet alliance. In: International Conference on Advances in Computing and Communication Engineering (ICACCE), pp. 349–355. IEEE, South Africa (2016)
Jalil, S., Rehmani, M., Chalup, H.: DQR: deep Q-routing in software defined networks. In: International Joint Conference on Neural Networks (IJCNN). IEEE (2020)
Kreutz, D., Ramos, F., Veríssimo, P., Rothenberg, C.: Software-defined networking: a comprehensive survey. Proc. IEEE 103, 14–76 (2015)
Du, J., Huang, X., Wu, F., Leng, S.: Reinforcement learning empowered QoS-aware adaptive Q-routing in ad-hoc networks. In: 2020 International Wireless Communications and Mobile Computing (IWCMC), pp. 551–556. IEEE (2020)
Harewood-Gill, D., Martin, T., Nejabati, R.: The performance of Q-learning within SDN controlled static and dynamic mesh networks. In: 2020 6th IEEE Conference on Network Softwarization (NetSoft), pp. 185–189 (2020)
Ryu: Component-Based Software Defined Networking Framework (2021). https://ryu-sdn.org/
Grotto Networking: SDN Fun (2019). https://www.grotto-networking.com/SDNfun.html
Hong, C., et al.: Achieving high utilization with software-driven WAN. In: ACM SIGCOMM Computer Communication Review, pp. 15–26. IEEE (2013)
Acknowledgements
We thank the Engineering and Physical Sciences Research Council Centre for Doctoral Training in Communications (EP/I028153/1 and EP/L016656/1) and Roke Manor for financial assistance and support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Harewood-Gill, D., Martin, T., Nejabati, R. (2022). Q-Routing Using Multiple QoS Metrics in SDN. In: Jansen, T., Jensen, R., Mac Parthaláin, N., Lin, CM. (eds) Advances in Computational Intelligence Systems. UKCI 2021. Advances in Intelligent Systems and Computing, vol 1409. Springer, Cham. https://doi.org/10.1007/978-3-030-87094-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-87094-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87093-5
Online ISBN: 978-3-030-87094-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)