Skip to main content
SpringerLink
Log in

Centrality Based Congestion Detection Using Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

The rising number of users and the demand for more diverse and specialized applications have led to a tremendous increase in network traffic. Managing diverse traffic demands from numerous applications is a challenging task for the existing traditional networking architecture. Hybrid software defined network is widely used to simplify operations providing flexible traffic management and automation. However, managing dynamic traffic demands and routing traffic flows is a challenging task. Therefore, in this paper, a centrality based Q-learning routing traffic engineering method for congestion detection and optimized traffic routing is proposed. The proposed method uses the reinforcement Q-learning algorithm to find an optimal path for routing the traffic. The centrality measures of the nodes are computed and ranked using the simple additive weighted method to detect the top k high-risk nodes. Simulations are carried out under different network scenarios for various traffic profiles. The results show that the proposed method outperformed the existing methods in terms of path length, delay, link utilization, throughput and computation time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ahvar, E., Ahvar, S., Raza, S.M., Vilchez, J.M.S., Lee, G.M.: Next generation of SDN in cloud-fog for 5G and beyond-enabled applications: opportunities and challenges. Network 1, 28–49 (2021). https://doi.org/10.3390/network1010004

    Article  Google Scholar 

  2. Singh, S., Jha, R.K.: A survey on software defined networking: architecture for next generation network. J. Netw. Syst. Manag. 25, 321–374 (2017). https://doi.org/10.1007/s10922-016-9393-9

    Article  Google Scholar 

  3. Vissicchio, S., Vanbever, L., Bonaventure, O.: Opportunities and research challenges of hybrid software defined networks. ACM SIGCOMM Comput. Commun. Rev. 44, 70–75 (2014). https://doi.org/10.1145/2602204.2602216

    Article  Google Scholar 

  4. Kreutz, D., Ramos, F.M., Veríssimo, P.E., Rothenberg, C.E., Azodolmolky, S., Uhlig, S.: Software-defined networking : a comprehensive survey. Proc. IEEE. 103, 14–76 (2015). https://doi.org/10.1109/JPROC.2014.2371999

    Article  Google Scholar 

  5. Ahmad, S., Mir, A.H.: Scalability, consistency, reliability and security in SDN controllers: a survey of diverse SDN controllers. J. Netw. Syst. Manag. (2021). https://doi.org/10.1007/s10922-020-09575-4

    Article  Google Scholar 

  6. Sanvito, D., Filippini, I., Capone, A., Paris, S., Leguay, J.: Clustered robust routing for traffic engineering in software-defined networks. Comput. Commun. 144, 175–187 (2019). https://doi.org/10.1016/j.comcom.2019.06.002

    Article  Google Scholar 

  7. Chen, F., Wu, C., Hong, X., Lu, Z., Wang, Z., Lin, C.: Engineering traffic uncertainty in the OpenFlow data plane. Proc. IEEE INFOCOM. (2016). https://doi.org/10.1109/INFOCOM.2016.752461

    Article  Google Scholar 

  8. Takahashi, Y., Ishibashi, K., Tsujino, M., Kamiyama, N., Shiomoto. K., Otoshi, T., Ohsita, Y., Murata, M.: Separating predictable and unpredictable flows via dynamic flow mining for effective traffic engineering. IEEE Int. Conf. Commun. (2016). https://doi.org/10.1109/ICC.2016.7511371

  9. Özçevik, M.E., Canberk, B., Duong, T.Q.: End to end delay modeling of heterogeneous traffic flows in software defined 5G networks. Ad Hoc Netw. 60, 26–39 (2017). https://doi.org/10.1016/j.adhoc.2017.02.006

    Article  Google Scholar 

  10. Kim, H., Feamster, N.: Improving network management with software defined networking. IEEE Commun. Mag. 51, 114–119 (2013). https://doi.org/10.1109/MCOM.2013.6461195

    Article  Google Scholar 

  11. Amin, R., Reisslein, M., Shah, N.: Hybrid SDN networks: a survey of existing approaches. IEEE Commun. Surv. Tutor. 20(4), 3259–3306 (2018). https://doi.org/10.1109/COMST.2018.2837161

    Article  Google Scholar 

  12. Vissicchio, S., Vanbever, L., Cittadini, L., Xie, G., Bonaventure, O.: Safe updates of hybrid SDN networks. IEEE/ACM Trans. Netw. 25(3), 1649–1662 (2017). https://doi.org/10.1109/TNET.2016.2642586

    Article  Google Scholar 

  13. Casas-Velasco, D.M., Rendon, O.M.C., da Fonseca, N.L.S.: Intelligent routing based on reinforcement learning for software-defined networking. IEEE Trans. Netw. Manag. 18(1), 870–881 (2021). https://doi.org/10.1109/TNSM.2020.3036911

    Article  Google Scholar 

  14. Wang, J., Yang, Y., Xiao, L., Nahrstedt, K.: Edge-based traffic engineering for OSPF networks. Comput. Networks. 48(4), 605–625 (2005). https://doi.org/10.1016/j.comnet.2004.11.008

    Article  MATH  Google Scholar 

  15. Chiesa, M., Kindler, G., Schapira, M.: Traffic engineering with equal-cost-multipath: an algorithmic perspective. IEEE/ACM Trans. Netw. 25(2), 779–792 (2017). https://doi.org/10.1109/TNET.2016.2614247

    Article  Google Scholar 

  16. Guo, Y., Wang, Z., Yin, X., Shi, X., Wu, J.: Traffic Engineering in SDN/OSPF Hybrid Network. In: Proc. - IEEE 22nd Int. Conf. on Netw. Protoc. (2014) https://doi.org/10.1109/ICNP.2014.90

  17. Buriol, L.S., Resende, M.G.C., Ribeiro, C.C., Thorup, M.: A hybrid genetic algorithm for the weight setting problem in OSPF/IS-IS routing. Netw. 46(1), 36–56 (2005). https://doi.org/10.1002/net.20070

    Article  MathSciNet  MATH  Google Scholar 

  18. Fortz, B., Thorup, M.: Internet traffic engineering by optimizing OSPF weights. Internet traffic engineering by optimizing OSPF weights. Proc. IEEE INFOCOM. 2, 519–528 (2000). https://doi.org/10.1109/INFCOM.2000.832225.

  19. Isravel, D.P., Silas, S., Rajsingh, E.B.: Preferential multi-attribute rank based forwarding node selection in software defined networks. In: Proc.- 3rd Int. Conf. Comput. Commun. Technol., pp. 73–79 (2019). https://doi.org/10.1109/ICCCT2.2019.8824798

  20. Mishra, A., Sahoo, A., Dalvi, B., Zhu, T.: WOSPF: a traffic engineering solution for OSPF networks. In: Proc.- IEEE Glob. Commun. Conf. GLOBECOM (2016). https://doi.org/10.1109/GLOCOM.2016.7842393

  21. Shirmarz, A., Ghaffari, A.: An adaptive greedy flow routing algorithm for performance improvement in software—defined network. Int. J. Numer. Model. (2019). https://doi.org/10.1002/jnm.2676

    Article  Google Scholar 

  22. Majidi, A., Jahanbakhsh, N., Gao, X., Zheng, J., Chen, G.: DC-ECN: a machine-learning based dynamic threshold control scheme for ECN marking in DCN. Comput. Commun. (2019). https://doi.org/10.1016/j.comcom.2019.10.028

    Article  Google Scholar 

  23. Wang, H., Xu, H., Huang, L., Wang, J., Yang, X.: Load-balancing routing in software defined networks with multiple controllers. Comput. Netw. 141, 82–91 (2018). https://doi.org/10.1016/j.comnet.2018.05.012

    Article  Google Scholar 

  24. Nasimi, M., Habibi, M.A., Han, B., Schotten, H.D.: Edge-assisted congestion control mechanism for 5G network using software-defined networking. In: Proc. -Int. Symp. Wirel. Commun. Syst. 1–5, (2018). https://doi.org/10.1109/ISWCS.2018.8491233

  25. Handigol, N., Seetharaman, S., McKeown, N., Johari, R.: Plug-n-Serve: load-balancing web traffic using OpenFlow. ACM Sigcomm Demo (2009).

  26. Chu, C.Y., Xi, K., Luo, M., Chao, H.J.: Congestion-aware single link failure recovery in hybrid SDN networks. Proc. IEEE INFOCOM. 26, 1086–1094 (2015). https://doi.org/10.1109/INFOCOM.2015.7218482

    Article  Google Scholar 

  27. Zakia, U., Yedder, H.B.: Dynamic load balancing in SDN-based data center networks. In: IEEE Annu. Inf. Techno., Electron. and Mob. Commun. Conf. (IEMCON), pp. 242–247 (2017). https://doi.org/10.1109/IEMCON.2017.8117206

  28. Tajiki, M.M., Akbari, B., Shojafar, M., Mokari, N.: Joint QoS and congestion control based on traffic prediction in SDN. Appl. Sci. 7(12), 1–15 (2017). https://doi.org/10.3390/app7121265

    Article  Google Scholar 

  29. Le, D.C., Zincir-Heywood, N.: A frontier: dependable, reliable and secure machine learning for network/system management. J. Netw. Syst. Manag. 28, 827–849 (2020). https://doi.org/10.1007/s10922-020-09512-5

    Article  Google Scholar 

  30. Xie, J., Yu, F.R., Huang, T., Xie, R., Liu, J., Wang, C., Liu, Y.: A survey of machine learning techniques applied to software defined networking (SDN): research issues and challenges. IEEE Commun. Surv. Tutorials. 21(1), 393–430 (2019). https://doi.org/10.1109/COMST.2018.2866942

    Article  Google Scholar 

  31. Dong, M., Li, Q., Zarchy, D., Godfrey, P.B., Schapira, M.: PCC: Re-architecting congestion control for consistent high performance. In: Proc. 12th USENIX Symp. Networked Syst. Des. Implement. NSDI '15, pp. 395–408 (2015).

  32. Dong, M., Tong, M., Zarchy, D., Arslan, E., Gilad, Y., Godfrey, B., Schapira, M.: PCC Vivace : online-learning congestion control. In: Proc. 15th USENIX Symp. Networked Syst. Des. Implement. (NSDI '18), pp. 343–356 (2018).

  33. Nguyen, T.T.T., Armitage, G.: A survey of techniques for internet traffic classification using machine learning. IEEE Commun. Surv. Tutorials. 10(4), 56–76 (2008). https://doi.org/10.1109/SURV.2008.080406

    Article  Google Scholar 

  34. Azzouni, A., Boutaba, R., Pujolle, G.: NeuRoute : Predictive dynamic routing for software-defined networks. In: Proc. - 13th Int. Conf. Netw. Serv. Manag. (CNSM), pp. 1–6 (2017). https://doi.org/10.23919/CNSM.2017.8256059

  35. Assefa, B.G., Ozkasap, O.: MER-SDN : machine learning framework for traffic aware energy efficient routing in SDN. In: IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress(DASC/PiCom/DataCom/CyberSciTech), pp. 74–980 (2018). https://doi.org/10.1109/DASC/PiCom/DataCom/CyberSciTec.2018.000-1

  36. Choi, S.P.M., Yeung, D.-Y.: Predictive Q-routing: a memory-based reinforcement learning approach to adaptive traffic control. Adv. Neural Inf. Process. Syst. 8, 945–951 (1996)

    Google Scholar 

  37. Chavula, J., Densmore, M., Suleman, H.: Using SDN and reinforcement learning for traffic engineering in ubuntunet alliance. In: Proc. Int. Conf. Adv. Comput. Commun. Eng. (ICACCE), pp. 349–355 (2016). https://doi.org/10.1109/ICACCE.2016.8073774

  38. Kong, Y., Zang, H., Ma, X.: Improving TCP congestion control with machine intelligence. In: Proc. Work. Network Meets AI and ML, pp. 60–66 (2018). https://doi.org/10.1145/3229543.3229550

  39. Lin, S., Akyildiz, I. F., Wang, P., Luo, M.: QoS-aware adaptive routing in multi-layer hierarchical software defined networks : a reinforcement learning approach. In: IEEE Int. Conf. Serv. Comput., pp. 25–33 (2016). https://doi.org/10.1109/SCC.2016.12

  40. Yu, C., Lan, J., Guo, Z., Hu, Y.: DROM: optimizing the routing in software-defined networks with deep reinforcement learning. IEEE Access 6, 64533–64539 (2018). https://doi.org/10.1109/ACCESS.2018.2877686

    Article  Google Scholar 

  41. Tessler, C., Shpigelman, Y., Dalal, G., Mandelbaum, A., Kazakov, D.H., Fuhrer, B., Chechik, G., Mannor, S.: Reinforcement learning for datacenter congestion control (2021). arXiv:2102.09337

  42. Bachl, M., Zseby, T., Fabini, J.: Rax: deep reinforcement learning for congestion control. IEEE Int. Conf. Commun. (2019). https://doi.org/10.1109/ICC.2019.8761187

    Article  Google Scholar 

  43. Lei, K., Liang, Y., Li, W.: Congestion control in SDN-based networks via multi-task deep reinforcement learning. IEEE Netw. 34, 28–34 (2020). https://doi.org/10.1109/MNET.011.1900408

    Article  Google Scholar 

  44. Zhang, J., Ye, M., Guo, Z., Yen, C.-Y., Choa, H.J.: CFR-RL: Traffic engineering with reinforcement learning in SDN. IEEE J. Sel. Areas Commun. 38(10), 2249–2259 (2020). https://doi.org/10.1109/JSAC.2020.3000371

    Article  Google Scholar 

  45. Agryzkov, T., Oliver, J.L., Tortosa, L., Vicent, J.: A new betweenness centrality measure based on an algorithm for ranking the nodes of a network. Appl. Math. Comput. 244, 467–478 (2014). https://doi.org/10.1016/j.amc.2014.07.026

    Article  MathSciNet  MATH  Google Scholar 

  46. Saxena, A., Gera, R., Iyengar, S.R.S.: A faster method to estimate closeness centrality ranking (2017). arXiv:1706.02083

  47. Liben-Nowell, D., Kleinberg, J.: The link-prediction problem for social networks. J. Am. Society Inf. Sci. Technol. 58(7), 1019–1031 (2007)

    Article  Google Scholar 

  48. Moradi, P., Shiri, M.E., Entezari, N.: Automatic skill acquisition in reinforcement learning agents using agents using connection bridge centrality. In: Kim, T., Vasilakos, T., Sakurai, K., Xiao, Y., Zhao, G., Ślęzak, D. (eds) Communication and Networking. FGCN 2010. Commun. Comput. Inf. Sci. 120, pp. 51–62. Springer, Berlin (2010). https://doi.org/10.1007/978-3-642-17604-3

  49. Zhang, J., Luo, Y.: Degree centralityIn, betweenness centrality , and closeness centrality in social network. In: Proc. - 2nd Int. Conf. Model. Simul. Appl. Math., vol. 132, pp. 300–303 (2017). https://doi.org/10.2991/msam-17.2017.68

  50. Okamoto, K., Chen, W., Li, X.Y.: Ranking of closeness centrality for large-scale social networks. In: Preparata, F.P., Wu, X., Yin, J. (eds) Frontiers in Algorithmics. FAW 2008. Lecture Notes in Comput. Sci. 5059, pp. 186–195. Springer, Berlin. https://doi.org/10.1007/978-3-540-69311-6_21

  51. Benzi, M., Klymko, C.: On the limiting behavior of parameter-dependent network centrality measures. SIAM J. Matrix Anal. Appl. 36(2), 686–706 (2015). https://doi.org/10.1137/130950550

    Article  MathSciNet  MATH  Google Scholar 

  52. Kaliszewski, I., Podkopaev, D.: Simple additive weighting—a metamodel for multiple criteria decision analysis methods. Expert Syst. Appl. 54, 155–161 (2016). https://doi.org/10.1016/j.eswa.2016.01.042

    Article  Google Scholar 

  53. Bhattacharyya, R., Bura, A., Rengarajan, D., Rumuly, M., Shakkottai, S., Kalathil, D., Mok, R.K.P., Dhamdhere, A.: QFlow : A reinforcement learning approach to high QoE video streaming over wireless networks. In: Proc. - 20th ACM Int. Symp. Mobile Ad Hoc Netw. Comput., pp. 251–260 (2019). https://doi.org/10.1145/3323679.3326523

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Karunya Institute of Technology and Sciences, Coimbatore, India

    Deva Priya Isravel, Salaja Silas & Elijah Blessing Rajsingh

Authors
  1. Deva Priya Isravel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Salaja Silas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Elijah Blessing Rajsingh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Deva Priya Isravel.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Isravel, D.P., Silas, S. & Rajsingh, E.B. Centrality Based Congestion Detection Using Reinforcement Learning Approach for Traffic Engineering in Hybrid SDN. J Netw Syst Manage 30, 2 (2022). https://doi.org/10.1007/s10922-021-09627-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-021-09627-3

Keywords

Search

Navigation