Skip to main content

Advertisement

Springer Nature Link
Log in

Adaptive UE Handover Management with MAR-Aided Multivariate DQN in Ultra-Dense Networks

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

Ultra-Dense Networks (UDNs) are a cornerstone of 5G, offering high-speed transmission and efficient resource management. However, managing frequent handovers in UDNs poses significant challenges, including increased handover failures and frequent triggering, which degrade user experience. This paper proposes an adaptive handover management approach using a multivariate Deep Q-Network (DQN) framework integrated with a Memory Anchor Repository (MAR) mechanism. The framework consists of three DQN models: \(\varvec{D}_\text {Dec}\) for handover decision-making, \(\varvec{D}_\text {TH}\) for adaptive adjustment of A2 and A4 thresholds, and \(\varvec{D}_\text {Tar}\) for target base station selection. These models leverage real-time features such as user location, movement direction, Signal-to-Interference-plus-Noise Ratio (SINR), and Reference Signal Received Power (RSRP). The MAR systematically stores and updates handover success rates at anchor points, enabling the system to learn from historical data and dynamically optimize handover decisions. Simulations conducted in a controlled UDN environment demonstrate that the proposed framework significantly reduces unnecessary handover attempts and failures. After 1250 training iterations, the overall handover failure rate decreases from 35% to 25%, with optimal performance observed using 25 anchor points. These results illustrate the framework’s potential to enhance UDN handover processes, improve overall Quality of Service (QoS), and elevate user experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Fig. 5
Algorithm 2
Fig. 6
Algorithm 3
Fig. 7
Fig. 8
Algorithm 4
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Data Availability

Datasets of the the simulation results are accessible through GitHub. URL: https://github.com/didgmd/Network-Optimization/tree/master/05_DQN_HO.

References

  1. Angjo, J., Shayea, I., Ergen, M., et al.: Handover management of drones in future mobile networks: 6g technologies. IEEE Access 9, 12803–12823 (2021). https://doi.org/10.1109/ACCESS.2021.3051097

    Article  MATH  Google Scholar 

  2. Zhao, D., Yan, Z., Wang, M., et al.: Is 5G handover secure and private? a survey. IEEE Internet Things J. 8(16), 12855–12879 (2021). https://doi.org/10.1109/JIOT.2021.3068463

    Article  MATH  Google Scholar 

  3. da Silva, Brilhante D., de Rezende, J.F., Marchetti, N.: Handover optimisation for high-capacity low-latency 5G NR mmWave communication. Ad Hoc Net. (2024)

  4. Bennaoui, A., Guezouri, M., Keche, M.: Improving VANET data dissemination efficiency with deep neural networks. J. Netw. Syst. Manag. 32(4), 81 (2024). https://doi.org/10.1007/s10922-024-09858-0

    Article  Google Scholar 

  5. Hu, H., Zhang, W., Xu, L., et al.: A mobility-aware service function chain migration strategy based on deep reinforcement learning. J. Netw. Syst. Manag. 31(1), 21 (2023). https://doi.org/10.1007/s10922-022-09713-0

    Article  MATH  Google Scholar 

  6. Hassen, H., Meherzi, S., Jemaa, Z.B.: Improved exploration strategy for Q-learning based multipath routing in SDN networks. J. Netw. Syst. Manag. 32(2), 25 (2024). https://doi.org/10.1007/s10922-024-09804-0

    Article  Google Scholar 

  7. Wang Y, Xiao Y, Song Y, et al (2023) Deep Reinforcement Learning Based Probabilistic Cognitive Routing: An Empirical Study with OMNeT++ and P4. In: 2023 19th International Conference on Network and Service Management (CNSM), IEEE, pp 1–7, https://doi.org/10.23919/CNSM59352.2023.10327868

  8. Mudvari, A., Tassiulas, L.: Joint SDN Synchronization and Controller Placement in Wireless Networks using Deep Reinforcement Learning. In: NOMS 2024-2024 IEEE Network Operations and Management Symposium, IEEE, pp. 1–9 (2024). https://doi.org/10.1109/NOMS59830.2024.10575746

  9. Okine, A.A., Adam, N., Naeem, F., et al.: Multi-agent deep reinforcement learning for packet routing in tactical mobile sensor networks. IEEE Trans. Netw. Serv. Manag. (2024). https://doi.org/10.1109/TNSM.2024.3352014

    Article  MATH  Google Scholar 

  10. Arakawa, K., Oki, E.: Availability-aware virtual network function placement based on multidimensional universal generating functions. Int. J. Netw. Manag. 34(2), e2252 (2024)

    Article  MATH  Google Scholar 

  11. Aboud, A., Touati, H., Hnich, B.: Markov Chain based Predictive Model for Efficient handover Management in Vehicle-to-Infrastructure Communications. In: 2021 International Wireless Communications and Mobile Computing (IWCMC), IEEE, pp. 1117–1122 (2021). https://doi.org/10.1109/IWCMC51323.2021.9498927

  12. Lee, C., Cho, H., Song, S., et al.: Prediction-based conditional handover for 5G mm-wave networks: a deep-learning approach. IEEE Veh. Technol. Mag. 15(1), 54–62 (2020). https://doi.org/10.1109/MVT.2019.2959065

    Article  MATH  Google Scholar 

  13. Masri, A., Veijalainen, T., Martikainen, H. et al.: Machine-Learning-Based Predictive Handover. In: 2021 IFIP/IEEE International Symposium on Integrated Network Management (IM), IEEE, pp. 648–652 (2021)

  14. Wang, H., Li, B.: Double-deep Q-learning-based handover management in mmWave heterogeneous networks with dual connectivity. Trans. Emerg. Telecommun. Technol. 35(1), e4907 (2024)

    Article  Google Scholar 

  15. Prado, A., Vijayaraghavan, H., Kellerer, W.: ECHO: Enhanced Conditional Handover boosted by Trajectory Prediction. In: 2021 IEEE Global Communications Conference (GLOBECOM), IEEE, pp. 01–06 (2021). https://doi.org/10.1109/GLOBECOM46510.2021.9685348

  16. Kwong, C.F., Shi, C., Liu, Q., Yang, S., Chieng, D., Kar, P.: Autonomous handover parameter optimisation for 5G cellular networks using deep deterministic policy gradient. Expert Syst. Appl. 246, 122871 (2024)

    Article  Google Scholar 

  17. Ohta, S., Nishio, T., Kudo, R., et al.: Point cloud-based proactive link quality prediction for millimeter-wave communications. IEEE Trans. Mach. Learn. Commun. Netw. (2023). https://doi.org/10.1109/TMLCN.2023.3319286

    Article  MATH  Google Scholar 

  18. Stanczak, J., Karabulut, U., Awada, A.: Conditional Handover in 5G - Principles, Future Use Cases and FR2 Performance. In: 2022 International Wireless Communications and Mobile Computing (IWCMC), IEEE, pp. 660–665, (2022). https://doi.org/10.1109/IWCMC55113.2022.9824571

  19. Iqbal, S.B., Awada, A., Karabulut, U. et al.: On the Modeling and Analysis of Fast Conditional Handover for 5G-Advanced. In: 2022 IEEE 33rd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), IEEE, pp. 595–601 (2022). https://doi.org/10.1109/PIMRC54779.2022.9977719

  20. Iqbal, S.B., Nadaf, S., Awada, A., et al.: On the analysis and optimization of fast conditional handover with hand blockage for mobility. IEEE Access 11, 30040–30056 (2023). https://doi.org/10.1109/ACCESS.2023.3260630

    Article  MATH  Google Scholar 

  21. Tayyab, M., Koudouridis, G.P., Gelabert, X., et al.: Uplink reference signals for power-efficient handover in cellular networks with mobile relays. IEEE Access 9, 24446–24461 (2021). https://doi.org/10.1109/ACCESS.2021.3056945

    Article  Google Scholar 

  22. Özkoç, M.F., Koutsaftis, A., Kumar, R., et al.: The impact of multi-connectivity and handover constraints on millimeter wave and terahertz cellular networks. IEEE J. Sel. Areas Commun. 39(6), 1833–1853 (2021). https://doi.org/10.1109/JSAC.2021.3071852

    Article  MATH  Google Scholar 

  23. Chiputa, M., Zhang, M., Ali, G.M.N., et al.: Enhancing handover for 5G mmWave mobile networks using jump markov linear system and deep reinforcement learning. Sensors 22(3), 746 (2022). https://doi.org/10.3390/s22030746

    Article  MATH  Google Scholar 

  24. Tong, H., Wang, T., Zhu, Y., et al.: Mobility-aware seamless handover With MPTCP in software-defined HetNets. IEEE Trans. Netw. Serv. Manag. 18(1), 498–510 (2021). https://doi.org/10.1109/TNSM.2021.3050627

    Article  Google Scholar 

  25. Rodoshi, R.T., Kim, T., Choi, W.: Fuzzy logic and accelerated reinforcement learning-based user association for dense C-RANs. IEEE Access 9, 117910–117924 (2021). https://doi.org/10.1109/ACCESS.2021.3107325

    Article  Google Scholar 

  26. Alablani, I.A., Arafah, M.A.: An adaptive cell selection scheme for 5G heterogeneous ultra-dense networks. IEEE Access 9, 64224–64240 (2021). https://doi.org/10.1109/ACCESS.2021.3075324

    Article  Google Scholar 

  27. Yan, X., Ma, M.: A lightweight and secure handover authentication scheme for 5G network using neighbour base stations. J. Netw. Comput. Appl. 193, 103204 (2021)

    Article  MATH  Google Scholar 

  28. Yan, X., Ma, M.: NSEHA: A Neighbor-based Secure and Efficient Handover Authentication Mechanism for 5G Networks. In: Proceedings of the 2021 9th International Conference on Communications and Broadband Networking, pp. 209–216 (2021). https://doi.org/10.1145/3456415.3456449

  29. Oulaaffart, M., Badonnel, R., Bianco, C.: An Automated SMT-based Security Framework for Supporting Migrations in Cloud Composite Services. In: NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium, IEEE, pp. 1–9 (2022). https://doi.org/10.1109/NOMS54207.2022.9789768

  30. Kwon, D., Son, S., Park, Y., et al.: Design of secure handover authentication scheme for urban air mobility environments. IEEE Access 10, 42529–42541 (2022). https://doi.org/10.1109/ACCESS.2022.3168843

    Article  Google Scholar 

  31. Haghrah, A., Abdollahi, M.P., Azarhava, H., et al.: (2023) A survey on the handover management in 5G-NR cellular networks: aspects, approaches and challenges. EURASIP J. Wirel. Commun. Netw. 1, 52 (2023). https://doi.org/10.1186/s13638-023-02261-4

    Article  Google Scholar 

  32. Mollel, M.S., Abubakar, A.I., Ozturk, M., et al.: A survey of machine learning applications to handover management in 5G and beyond. IEEE Access 9, 45770–45802 (2021). https://doi.org/10.1109/ACCESS.2021.3067503

    Article  Google Scholar 

  33. Tanveer, J., Haider, A., Ali, R., et al.: An overview of reinforcement learning algorithms for handover management in 5G ultra-dense small cell networks. Appl. Sci. 12(1), 426 (2022). https://doi.org/10.3390/app12010426

    Article  MATH  Google Scholar 

  34. Liu, Q., Kwong, C.F., Wei, S., et al.: Intelligent handover triggering mechanism in 5G ultra-dense networks via clustering-based reinforcement learning. Mobile Netw. Appl. 26, 27–39 (2021). https://doi.org/10.1007/s11036-020-01718-w

    Article  Google Scholar 

  35. Koda, Y., Yamamoto, K., Nishio, T. et al.: Reinforcement learning based predictive handover for pedestrian-aware mmWave networks. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), IEEE, pp. 692–697 (2018) https://doi.org/10.1109/INFCOMW.2018.8406993

  36. Santana, J.P.M., Abrão, T.: Power-Profile in Q-Learning NOMA random access protocols for throughput maximization. J. Netw. Syst. Manag. 32(3), 48 (2024). https://doi.org/10.1007/s10922-024-09823-x

    Article  Google Scholar 

  37. Palas, M.R., Islam, M.R., Roy, P., et al.: Multi-criteria handover mobility management in 5G cellular network. Comput. Commun. 174, 81–91 (2021). https://doi.org/10.1016/j.comcom.2021.04.020

    Article  MATH  Google Scholar 

  38. Alizadeh, A., Lim, B., Vu, M.: Multi-Agent Q-Learning for real-time load balancing user association and handover in mobile networks. IEEE Trans. Wirel. Commun. (2024). https://doi.org/10.1109/TWC.2024.3357702

    Article  Google Scholar 

  39. Rhee, I., Shin, M., Hong, S., et al.: On the Levy-Walk nature of human mobility. IEEE/ACM Trans. Netw. 19(3), 630–643 (2011). https://doi.org/10.1109/TNET.2011.2120618

    Article  MATH  Google Scholar 

  40. Mumtaz, T., Muhammad, S., Aslam, M.I., et al.: Dual Connectivity-Based Mobility Management and Data Split Mechanism in 4G/5G Cellular Networks. IEEE Access 8, 86495–86509 (2020). https://doi.org/10.1109/ACCESS.2020.2992805

    Article  MATH  Google Scholar 

  41. Islam, N., Kandeepan, S., Chavez, K.G., et al.: A MDP-based Energy Efficient and Delay Aware Handover Algorithm. In: 2019 13th International Conference on Signal Processing and Communication Systems (ICSPCS), IEEE, pp. 1–5 (2019). https://doi.org/10.1109/ICSPCS47537.2019.9008697

  42. Campbell, J.S., Givigi, S.N., Schwartz, H.M.: Multiple-model Q-learning for stochastic reinforcement delays. In: 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC), IEEE, pp. 1611–1617 (2014). https://doi.org/10.1109/SMC.2014.6974146

  43. Wu, M., Huang, W., Sun, K., et al.: A DQN-Based Handover Management for SDN-Enabled Ultra-Dense Networks. In: 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), IEEE, pp. 1–6 (2020). https://doi.org/10.1109/VTC2020-Fall49728.2020.9348779

  44. Wei, Y., Lung, C.H., Ajila, S., et al.: Deep Q-Networks Assisted Pre-connect Handover Management for 5G Networks. In: 2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring), IEEE, pp. 1–6 (2023). https://doi.org/10.1109/VTC2023-Spring57618.2023.10199527

Download references

Acknowledgements

This work was supported by the Scientific Research Fund of Liaoning Provincial Education Department (Grant No. 202414226-1).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shenyang University of Technology, No.111, Shenliao West Road, Economic & Technological Development Zone, Shenyang, 110870, Liaoning, China

    Weiran Wang, Heng Yang & Xue Liu

  2. School of Mechanical Engineering, Shenyang University of Technology, No.111, Shenliao West Road, Economic & Technological Development Zone, Shenyang, 110870, Liaoning, China

    Shanshan Li

  3. School of Software, Shenyang University of Technology, No.111, Shenliao West Road, Economic & Technological Development Zone, Shenyang, 110870, Liaoning, China

    Zhaojun Wan

Authors
  1. Weiran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shanshan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xue Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhaojun Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Heng Yang or Shanshan Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Yang, H., Li, S. et al. Adaptive UE Handover Management with MAR-Aided Multivariate DQN in Ultra-Dense Networks. J Netw Syst Manage 33, 17 (2025). https://doi.org/10.1007/s10922-024-09895-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-024-09895-9

Keywords

Search

Navigation