Skip to main content
Springer Nature Link
Log in

User association-based load balancing using reinforcement learning in 5G heterogeneous networks

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The deployment of heterogeneous networks (HetNets), which include macro and micro Base Stations (BSs), has emerged as an effective solution to address the challenges faced by 5G communication networks. HetNets leverage the strengths of various technologies to enhance coverage, capacity, quality of service (QoS), cost efficiency, and adaptability. Efficient methods for associating user equipment with BSs in HetNets are necessary to maximize overall network performance. This process, known as user association-based load balancing, is crucial due to the increasing number of devices requiring connectivity using limited available spectrum and BSs. This study considers a three-tier downlink HetNet consisting of macro-BSs, pico-BSs, and femto-BSs, operating under a log-distance channel model in both rural and urban areas, in accordance with 3GPP norms. The signal model for the proposed HetNet is devised to determine the signal-to-interference-plus-noise ratio (SINR) and transmission rates for each user association. Load balancing is then performed using the deep Q-learning approach, and the cumulative transmission rate achieved is calculated and tabulated. This approach is compared with the MaxSINR and Q-Learning algorithms. LB (load balancing) was performed for different network scenarios by varying the number of UEs and BSs and introducing mobility in the network. The corresponding results were analyzed and plotted. An extensive survey of the effect of user association LB on improving the cumulative transmission rate of HetNets was conducted successfully, and the variation in performance under different network scenarios was analyzed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

No datasets were generated or analyzed during the current study.

References

  1. Vannithamby R, Talwar S (2017) Towards 5G: applications, requirements and candidate technologies. Wiley, Hoboken

  2. Takaharu N (2011) LTE and LTE-advanced: radio technology aspects for mobile communications. In: 2011 XXXth URSI General Assembly and Scientific Symposium, pp 1–4. https://doi.org/10.1109/URSIGASS.2011.6050508

  3. Attarbashi ZS, Altaleb AE, Mohd Aman AH, Hassan Abdalla Hashim A, Eker S (2022) A review of 5G technology: architecture and challenges. In: 2022 IEEE 8th International Conference on Computing, Engineering and Design (ICCED), pp 1–4. https://doi.org/10.1109/ICCED56140.2022.10010607

  4. Adebusola JA, Ariyo AA, Elisha OA, Olubunmi AM, Julius OO (2020) An overview of 5G technology. In: 2020 International Conference in Mathematics, Computer Engineering and Computer Science (ICMCECS), pp 1–4. https://doi.org/10.1109/ICMCECS47690.2020.240853

  5. Qiuyan L, Zhigang S (2013) Design of picocells in heterogeneous networks. In: 2013 Fifth International Conference on Measuring Technology and Mechatronics Automation, pp 452–455. https://doi.org/10.1109/ICMTMA.2013.113

  6. Agarwal B, Togou MA, Marco M, Muntean G-M (2022) A comprehensive survey on radio resource management in 5G HETNETs: current solutions, future trends and open issues. IEEE Commun Surv Tutor 24(4):2495–2534. https://doi.org/10.1109/COMST.2022.3207967

    Article  Google Scholar 

  7. Song X, Wu L, Ren X, Gao J (2015) Load balancing algorithm based on neural network in heterogeneous wireless networks. In: Advances in Neural Networks—ISNN 2015: 12th International Symposium on Neural Networks, ISNN 2015, Jeju, South Korea, October 15–18, 2015, Proceedings, pp 463–472. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-319-25393-0_51

  8. Yajnanarayana V, Rydén H, Hévizi L (2020) 5G handover using reinforcement learning. In: 2020 IEEE 3rd 5G World Forum (5GWF), pp 349–354. https://doi.org/10.1109/5GWF49715.2020.9221072

  9. Buhurcu S, Çarkacıoğlu L (2024) Reinforcement learning based mobility load balancing in cellular networks: a two-layered approach. Signal Image Video Process, 1–9

  10. Omran A, Sboui L, Rong B, Rutagemwa H, Kadoch M (2019) Joint relay selection and load balancing using D2D communications for 5G hetnet mec. In: 2019 IEEE International Conference on Communications Workshops (ICC Workshops), pp 1–5. IEEE

  11. Gures E, Shayea I, Ergen M, El-Saleh AA (2022) Fuzzy logic-based load balancing algorithm in heterogeneous networks. In: 2022 Workshop on Microwave Theory and Techniques in Wireless Communications (MTTW), pp 210–215. https://doi.org/10.1109/MTTW56973.2022.9942604

  12. Attiah K, Banawan K, Gaber A, Elezabi A, Seddik K, Gadallah Y, Abdullah K (2020) Load balancing in cellular networks: a reinforcement learning approach. In: 2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC), pp 1–6. https://doi.org/10.1109/CCNC46108.2020.9045699

  13. Alizadeh A, Vu M (2020) Multi-armed bandit load balancing user association in 5G cellular hetnets. In: GLOBECOM 2020—2020 IEEE Global Communications Conference, pp 1–6. https://doi.org/10.1109/GLOBECOM42002.2020.9348164

  14. Xiong Z, Zhang Y, Niyato D, Deng R, Wang P, Wang L-C (2019) Deep reinforcement learning for mobile 5G and beyond: fundamentals, applications, and challenges. IEEE Veh Tech Mag 14(2):44–52. https://doi.org/10.1109/MVT.2019.2903655

    Article  Google Scholar 

  15. Vilà I, Pérez-Romero J, Sallent O, Umbert A (2021) A multi-agent reinforcement learning approach for capacity sharing in multi-tenant scenarios. IEEE Trans Veh Tech 70(9):9450–9465

    Article  Google Scholar 

  16. Khani M, Jamali S, Sohrabi MK (2024) Approximate Q-learning-based (AQL) network slicing in mobile edge-cloud for delay-sensitive services. J Superc 80(3):4226–4247

    Article  Google Scholar 

  17. Wu D, Kang J, Xu YT, Li H, Li J, Chen X, Rivkin D, Jenkin M, Lee T, Park I, Liu X, Dudek G (2021) Load balancing for communication networks via data-efficient deep reinforcement learning. In: 2021 IEEE Global Communications Conference (GLOBECOM), pp 01–07. https://doi.org/10.1109/GLOBECOM46510.2021.9685294

  18. Wu G, Zeng D (2022) Optimization of network resource management based on software-defined networking in the 5G environment. J Supercomput 78(15):16721–16744

    Article  Google Scholar 

  19. 3GPP: 3RD generation partnership project, "study on channel model for frequencies from 0.5 to 100 GHz (3gpp tr38.901 version 14.3.0 release 14)". Technical Report TR 38.901 V14.3.0 (2017), 3GPP (2017)

  20. Sun S, Rappaport TS, Rangan S, Thomas TA, Ghosh A, Kovacs IZ, Rodriguez I, Koymen O, Partyka A, Jarvelainen J (2016) Propagation path loss models for 5G urban micro-and macro-cellular scenarios. In: 2016 IEEE 83rd Vehicular Technology Conference (VTC Spring), pp 1–6. IEEE

  21. Jabbari B, Zhou Y, Hillier F (1998) Random walk modeling of mobility in wireless networks. In: VTC’98. 48th IEEE Vehicular Technology Conference. Pathway to Global Wireless Revolution (Cat. No. 98CH36151), vol 1, pp 639–643. IEEE

  22. Sutton RS, Barto AG (2018) Reinforcement learning, second edition: an introduction. MIT Press, Cambridge

  23. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

Download references

Acknowledgements

The authors wish to extend their sincere thanks to the Centre for Internet of Things (CIoT) at the Madras Institute of Technology Campus, Anna University, for providing the laboratory facilities essential for this research.

Author information

Authors and Affiliations

  1. Department of Electronics Engineering and Centre for Internet of Things, Madras Institute of Technology Campus, Anna University, Chennai, India

    Parameswaran Ramesh, P. T. V. Bhuvaneswari, V. S. Dhanushree, G. Gokul & S. Sahana

Authors
  1. Parameswaran Ramesh
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. P. T. V. Bhuvaneswari
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. V. S. Dhanushree
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. G. Gokul
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. S. Sahana
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors are equally contributed for this research.

Corresponding author

Correspondence to Parameswaran Ramesh.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramesh, P., Bhuvaneswari, P.T.V., Dhanushree, V.S. et al. User association-based load balancing using reinforcement learning in 5G heterogeneous networks. J Supercomput 81, 328 (2025). https://doi.org/10.1007/s11227-024-06788-1

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06788-1

Keywords