Skip to main content
SpringerLink
Log in

An Efficient and Decentralized Fuzzy Reinforcement Learning Bandwidth Controller for Multitenant Data Centers

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Cloud service providers rely on bandwidth overprovisioning to avoid Service Level Agreements’ violation (SLAs) when allocating tenants’ resources in multitenant cloud environments. Tenants’ network usage is usually dynamic, but the shared resources are often allocated statically and in batches, causing resource idleness. This paper envisions an opportunity for optimizing cloud service networks. As such, we propose an autonomous bandwidth allocation mechanism based on Fuzzy Reinforcement Learning (FRL) to reduce the idleness of cloud network resources. Our mechanism dynamically allocates resources, prioritizing tenants and allowing them to exceed the contracted bandwidth temporarily without violating the SLAs. We assess our mechanism by comparing FRL usage against pure Fuzzy Inference System (FIS) and pure Reinforcement Learning (RL). The evaluation scenario is an emulation in which tenants share resources from a cloud provider and generate traffic based on real HTTP traffic. The results show that our mechanism increases tenant’s cloud network utilization by 30% compared to FIS while maintaining the cloud traffic load within a healthy threshold and more stable than RL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Implementation available on https://github.com/scherly/FRL-MAS.

References

  1. Shen, H., Li, Z.: New bandwidth sharing and pricing policies to achieve a win-win situation for cloud provider and tenants. IEEE Trans. Parallel Distrib. Syst. 27(9), 2682–2697 (2015)

    Article  Google Scholar 

  2. Malbašić, T., Bojović, P.D., Bojović, Ž, Šuh, J., Vujošević, D.: Hybrid SDN networks: a multi-parameter server load balancing scheme. J. Netw. Syst. Manag. 30(2), 30 (2022)

    Article  Google Scholar 

  3. Son, J., Buyya, R.: Priority-aware VM allocation and network bandwidth provisioning in software-defined networking (SDN)-enabled clouds. IEEE Trans. Sustain. Comput. 4(1), 17–28 (2018)

    Article  Google Scholar 

  4. Nine, M.S.Z., Azad, M.A.K., Abdullah, S., Rahman, R.M.: Fuzzy logic based dynamic load balancing in virtualized data centers. In: 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–7. IEEE (2013)

  5. Wang, T., Ma, H., Zhou, Y., Zhang, R., Song, Z.: Fully accountable data sharing for pay-as-you-go cloud scenes. IEEE Trans. Depend. Secur. Comput. 18, 2005–2016 (2019)

    Article  Google Scholar 

  6. Guo, J., Song, Z., Cui, Y., Liu, Z., Ji, Y.: Energy-efficient resource allocation for multi-user mobile edge computing. In: GLOBECOM—2017 IEEE Global Communications Conference, Singapore, pp. 1–7 (2017)

  7. Dutreilh, X., Kirgizov, S., Melekhova, O., Malenfant, J., Rivierre, N., Truck, I.: Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In: ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, Venice, pp. 67–74 (2011)

  8. Barrett, E., Howley, E., Duggan, J.: Applying reinforcement learning towards automating resource allocation and application scalability in the cloud. Concurr. Comput.: Pract. Exp. 25(12), 1656–1674 (2013)

    Article  Google Scholar 

  9. Chen, Z., Hu, J., Min, G.: Learning-based resource allocation in cloud data center using advantage actor-critic. In: ICC—2019 IEEE International Conference on Communications, Singapore, pp. 1–6 (2019)

  10. Zhao, X., Wang, X., Ma, L., Zong, G.: Fuzzy approximation based asymptotic tracking control for a class of uncertain switched nonlinear systems. IEEE Trans. Fuzzy Syst. 28(4), 632–644 (2020)

    Article  Google Scholar 

  11. Zhang, X., Biagioni, D., Cai, M., Graf, P., Rahman, S.: An edge-cloud integrated solution for buildings demand response using reinforcement learning. IEEE Trans. Smart Grid 12, 420–431 (2020)

    Article  Google Scholar 

  12. Liu, C.H., Dai, Z., Zhao, Y., Crowcroft, J., Wu, D.O., Leung, K.: Distributed and energy-efficient mobile crowdsensing with charging stations by deep reinforcement learning. IEEE Trans. Mob. Comput. 20, 130–146 (2019)

    Article  Google Scholar 

  13. Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149. IEEE (2020)

  14. Chen, L., Li, B., Li, B.: Allocating bandwidth in datacenter networks: a survey. J. Comput. Sci. Technol. 29(5), 910–917 (2014)

    Article  Google Scholar 

  15. Mattos, D.M., Ferraz, L.H.G., Costa, L.H.M., Duarte, O.C.M.: Evaluating virtual router performance for a pluralist future Internet. In: Proceedings of the 3rd International Conference on Information and Communication Systems, Irbid, Jordan, pp. 1–7 (2012)

  16. Naseri, T.S., Gharehchopogh, F.S.: A feature selection based on the farmland fertility algorithm for improved intrusion detection systems. J. Netw. Syst. Manag. 30(3), 40 (2022)

    Article  Google Scholar 

  17. Ali-Eldin, A., Tordsson, J., Elmroth, E.: An adaptive hybrid elasticity controller for cloud infrastructures. In: Proceedings of Network Operations and Management Symposium (NOMS), pp. 204–212 (2012)

  18. Popa, L., Kumar, G., Chowdhury, M., Krishnamurthy, A., Ratnasamy, S., Stoica, I.: Faircloud: Sharing the network in cloud computing. In: Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, pp. 187–198 (2012)

  19. Lorido-Botran, T., Miguel-Alonso, J., Lozano, J.A.: A review of auto-scaling techniques for elastic applications in cloud environments. J. Grid Comput. 12(4), 559–592 (2014)

    Article  Google Scholar 

  20. Patikirikorala, T., Colman, A.: Feedback controllers in the cloud. In: Proceedings of Asia-Pacific Software Engineering Conference (APSEC). SN, pp. 1–6 (2010)

  21. Heinze, T., Pappalardo, V., Jerzak, Z., Fetzer, C.: Auto-scaling techniques for elastic data stream processing. In: Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems (DEBS), pp. 318–321. Association for Computing Machinery, New York, NY, USA (2014)

  22. Santos Filho, R.H., Ferreira, T.N., Mattos, D.M., Medeiros, D.S.: A lightweight reinforcement-learning-based mechanism for bandwidth provisioning on multitenant data center. In: 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 331–336. IEEE (2020)

  23. Glorennec, P.Y., Jouffe, L.: Fuzzy q-learning. In: Proceedings of 6th International Fuzzy Systems Conference, vol. 2, pp. 659–662. IEEE (1997)

  24. Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: A survey. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2042–2062 (2017)

    Article  MathSciNet  Google Scholar 

  25. Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proc. Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, pp. 2094–2100 (2016)

  26. Boutaba, R.E.A.: A comprehensive survey on machine learning for networking: evolution, applications and research opportunities. J. Internet Serv. Appl. 9(1), 16 (2018). https://doi.org/10.1186/s13174-018-0087-2

    Article  Google Scholar 

  27. Navin, N.K., Sharma, R.: A fuzzy reinforcement learning approach to thermal unit commitment problem. Neural Comput. Appl. 31(3), 737–750 (2019)

    Article  Google Scholar 

  28. Kofinas, P., Dounis, A., Vouros, G.: Fuzzy q-learning for multi-agent decentralized energy management in microgrids. Appl. Energy 219, 53–67 (2018)

    Article  Google Scholar 

  29. Jamshidi, P., Sharifloo, A., Pahl, C., Arabnejad, H., Metzger, A., Estrada, G.: Fuzzy self-learning controllers for elasticity management in dynamic cloud architectures. In: 2016 12th International ACM SIGSOFT Conference on Quality of Software Architectures (QoSA), pp. 70–79. IEEE (2016)

  30. Omidzade, F., Ghodousi, H., Shahverdi, K.: Comparing fuzzy SARSA learning and ant colony optimization algorithms in water delivery scheduling under water shortage conditions. J. Irrig. Drain. Eng. 146(9), 04020028 (2020)

    Article  Google Scholar 

  31. Wibowo, F.X.A., Gregory, M.A.: Updating guaranteed bandwidth in multi-domain software defined networks. In: 2017 27th International Telecommunication Networks and Applications Conference (ITNAC), pp. 1–6 (2017)

  32. Filho, R.H.S., Ferreira, T.N., Mattos, D.M.F., Medeiros, D.S.V.: A rapid fuzzy controller for decentralized bandwidth provisioning on a multitenant data center. In: Proceedings of Network of Future (NOF), pp. 1–8 (2020). To appear

  33. Dane, L., Gurkan, D.: Netforager: Geographically-distributed network performance monitoring of web applications. In: 2020 10th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0142–0149 (2020)

  34. Wette, P., Dräxler, M., Schwabe, A., Wallaschek, F., Zahraee, M.H., Karl, H.: Maxinet: Distributed emulation of software-defined networks. In: 2014 IFIP Networking Conference, pp. 1–9 (2014)

  35. Lantz, B., Heller, B., McKeown, N.: A network in a laptop: Rapid prototyping for software-defined networks. In: Proceedings of the 9th ACM SIGCOMM Workshop on Hot Topics in Networks. Hotnets-IX, pp. 19–1196, New York, NY, USA (2010)

  36. Turner, A.: Tcpreplay. http://tcpreplay. synfin. net/trac/ (2011)

  37. Networking, C.V.: Cisco global cloud index: Forecast and methodology, 2015-2020. white paper. Cisco Public, San Jose (2016)

  38. Wu, C., Yoshinaga, T., Chen, X., Zhang, L., Ji, Y.: Cluster-based content distribution integrating ITE and IEEE 802.11 p with fuzzy logic and Q-learning. IEEE Comput. Intell. Mag. 13(1), 41–50 (2018)

    Article  Google Scholar 

  39. Bai, W., Chen, L., Chen, K., Han, D., Tian, C., Wang, H.: Information-agnostic flow scheduling for commodity data centers. In: 12th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 15), pp. 455–468 (2015)

  40. Chen, L., Lingys, J., Chen, K., Liu, F.: Auto: Scaling deep reinforcement learning for datacenter-scale automatic traffic optimization. In: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication, pp. 191–205 (2018)

Download references

Acknowledgements

We thank CNPq, CAPES, FAPERJ, FAPESP (2018/23062-5), RNP, and City hall of Niterói/FEC/UFF (PDPA 2020) for partially funding this research.

Author information

Author notes

  1. Reiner H. Santos Filho, Tadeu N. Ferreira, Diogo M. F. Mattos and Dianne S. V. Medeiros contributed equally to this work.

Authors and Affiliations

  1. Department of Telecommunications Engineering, Universidade Federal Fluminense, Rua Passo da Pátria, 156, Niterói, Rio de Janeiro, 24210-240, Brazil

    Reiner H. Santos Filho, Tadeu N. Ferreira, Diogo M. F. Mattos & Dianne S. V. Medeiros

Authors
  1. Reiner H. Santos Filho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tadeu N. Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diogo M. F. Mattos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dianne S. V. Medeiros
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dianne S. V. Medeiros.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santos Filho, R.H., Ferreira, T.N., Mattos, D.M.F. et al. An Efficient and Decentralized Fuzzy Reinforcement Learning Bandwidth Controller for Multitenant Data Centers. J Netw Syst Manage 30, 53 (2022). https://doi.org/10.1007/s10922-022-09667-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-022-09667-3

Keywords

Search

Navigation