Skip to main content
SpringerLink
Log in

Deep reinforcement learning–based multi–channel spectrum sharing technology for next generation multi–operator cellular networks

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

The mobile network operators (MNOs) need to efficiently utilize spectrum resources to meet increasing user demands for massive and ubiquitous connectivity. The licensed spectrum resources are scarce, costly and difficult to acquire. Consequently, the available bandwidth becomes a challenge. In this paper, a deep reinforcement learning (DRL)–based method has been utilized to share the spectrum in a multi–channels multi–operator environment. To intelligently and dynamically assign suitable channels, the proposed DRL model implemented at each MNO takes the load on the gNodeBs (gNBs), such as, the number of packets in the gNB queue and resource requirements of user equipments, such as, achievable data rate of users, into account to estimate the suitable channel selections. The scheduler then utilizes this channel information for efficient channel allocations. The performance of the proposed DRL–based spectrum sharing scheme has been compared with the conventional scheduling–based spectrum allocation scheme using extensive simulations. Results indicate that the dynamicity in network environment and traffic demands can be reasonably handled by the proposed DRL–based multi–channel spectrum sharing scheme, since it adapts feasibly to the varying number of channels, number of UEs, and network traffic conditions, compared to those of the conventional scheme. Furthermore, the proposed scheme shows superior performance gains in terms of throughput, resource utilization, delay, transmission time, and packet drop rates, compared to those of the conventional scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

No data availability

Notes

  1. In general, DQN models are designed to select the highest Q–value from all output nodes that maximizes the reward [25]

  2. It is noted that changing the number of UEs at the input layer or changing the number of channels at the output layer requires re–training the network

References

  1. Kuhlins, C., Rathonyi, B., Zaidi, A., & Hogan, M. (2016). Cellular networks for massive IoT: Enabling low power wide area applications ericsson. Sweden, White Paper: Stockholm.

    Google Scholar 

  2. David, K., & Berndt, H. (2018). 6G vision and requirements: Is there any need for beyond 5G? IEEE Vehicular Technology Magazine, 13(3), 72–80.

    Article  Google Scholar 

  3. Telecommincation Management; Network Sharing; Concepts and Requirements, Jun. 2016.

  4. Minimum requirements related to technical performance for IMT-2020 radio interface(s), 2017.

  5. Shirvanimoghaddam, M., Dohler, M., & Johnson, S. J. (2017). Massive non-orthogonal multiple access for cellular IoT: Potentials and limitations. IEEE Communications Magazine, 55(9), 55–61.

    Article  Google Scholar 

  6. Zhou, H., Cheng, N., Yu, Q., Shen, X. S., Shan, D., & Bai, F. (2016). Toward multi-radio vehicular data piping for dynamic DSRC/TVWS spectrum sharing. IEEE Journal on Selected Areas in Communications, 34(10), 2575–2588.

    Article  Google Scholar 

  7. Semasinghe, P., Maghsudi, S., & Hossain, E. (2017). Game theoretic mechanisms for resource management in massive wireless IoT systems. IEEE Communications Magazine, 55(2), 121–127.

    Article  Google Scholar 

  8. Cheng, P., Chen, Z., Ding, M., Li, Y., Vucetic, B., & Niyato, D. (2020). Spectrum intelligent radio: Technology, development, and future trends. IEEE Communications Magazine, 58(1), 12–18.

    Article  Google Scholar 

  9. Liu, M., Song, T., Hu, J., Yang, J., & Gui, G. (2018). Deep learning-inspired message passing algorithm for efficient resource allocation in cognitive radio networks. IEEE Transactions on Vehicular Technology, 68(1), 641–653.

    Article  Google Scholar 

  10. Zhu, P., Li, J., Wang, D., & You, X. (2020). Machine-learning-based opportunistic spectrum access in cognitive radio networks. IEEE Wireless Communication, 27(1), 38–44.

    Article  Google Scholar 

  11. Cui, Q., Ni, W., Li, S., Zhao, B., Liu, R. P., & Zhang, P. (2020). Learning-assisted clustered access of 5G/B5G networks to unlicensed spectrum. IEEE Wireless Communications, 27(1), 31–37.

    Article  Google Scholar 

  12. Xu, T., Zhou, T., Tian, J., Sang, J., & Hu, H. (2020). Intelligent spectrum sensing: When reinforcement learning meets automatic repeat sensing in 5G communications. IEEE Wireless Communications, 27(1), 46–53.

    Article  Google Scholar 

  13. Ruan, L., & Wong, E. (2018) Machine intelligence in allocating bandwidth to achieve low–latency performance, in: Proceedings of IEEE international conference on optical network design and modeling (ONDM).

  14. Feriani, A., & Hossain, E. (2021). Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial. IEEE Communications Surveys & Tutorials, 23(2), 1226–1252.

    Article  Google Scholar 

  15. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning Nature, 7553:436–444.

  16. Zhu, J., Song, Y., Jiang, D., & Song, H. (2018). A new deep-Q-learning-based transmission scheduling mechanism for the cognitive internet of things. IEEE Internet of Things Journal, 5(4), 2375–2385.

    Article  Google Scholar 

  17. Li, Z., Guo, C., & Xuan, Y. (2019). A multi–agent deep reinforcement learning based spectrum allocation framework for D2D communications. Version 3, Dec. Available: arXiv:org/abs/1904.06615.

  18. Song, H., Liu, L., Ashdown, J., & Yi, Y. (2021). A deep reinforcement learning framework for spectrum management in dynamic spectrum access. IEEE Internet Things Journal, 8(14), 11208–11218.

    Article  Google Scholar 

  19. Liu, M., Song, T., Hu, J., Yang, J., & Gui, G. (2018). Deep learning-inspired message passing algorithm for efficient resource allocation in cognitive radio networks. IEEE Transactions on Vehicular Technology, 68(1), 641–653.

    Article  Google Scholar 

  20. Lee, W., Kim, M., & Cho, D. (2019). Deep cooperative sensing: Cooperative spectrum sensing based on convolutional neural networks. IEEE Transactions on Vehicular Technology, 68(3), 3005–3009.

    Article  Google Scholar 

  21. Hossain, M. S., & Muhammad, G. (2020). A deep-tree-model-based radio resource distribution for 5G networks. IEEE Wireless Communications, 27(1), 62–67.

    Article  Google Scholar 

  22. Wang, J. B., Wang, J., Wu, Y., Wang, J. Y., Zhu, H., Lin, M., & Wang, J. (2018). A machine learning framework for resource allocation assisted by cloud computing. IEEE Network, 32(2), 144–151.

    Article  Google Scholar 

  23. Testolin, A., Zanforlin, M., Grazia, M. D. F. D., Munaretto, D., Zanella, A., Zorzi, M., & Zorzi, M. (2014) . A machine learning approach to QoE–based video admission control and resource allocation in wireless systems, in: Proceedings of 13th Annual Mediterranean Ad Hoc Networking Workshop (MED–HOC–NET).

  24. 3rd generation partnership project; technical specification group radio access network; evolved universal terrestrial radio access (E–UTRA) and evolved universal terrestrial radio access network (E–UTRAN); Overall description; Stage 2 (Release 17), 3GPP TS 36.300 V17.1.0 (2022-06), ver. 17.0.1.

  25. Riedmiller, M. (2005) . Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method, in: Proceedings of European Conference on Machine Learning, Springer, Berlin & Heidelberg.

  26. Silva, V., Abrão, T., & Jeszensky, P. J. (2004) . Statistically correct simulation models for the generation of multiple uncorrelated rayleigh fading waveforms, in: Proceedings of IEEE international symposium on spread spectrum techniques and applications–Programme and book of abstracts (IEEE Cat. No. 04TH8738), pp. 472–476. IEEE

  27. ETSI, T. LTE: Evolved universal terrestrial radio access (E–UTRA), Physical Layer Procedures–corresponding to 3GPP TS36 213. 3GPP TS, 136 (213), version 10.

Download references

Acknowledgements

This work was supported in part by Samsung Research in Samsung Electronics.

Author information

Authors and Affiliations

  1. Department of Electrical and Computer Engineering, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon, Gyeonggi-do, 16419, Republic of Korea

    Minsu Shin, Tahira Mahboob, Danish Mehmood Mughal & Min Young Chung

Authors
  1. Minsu Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tahira Mahboob
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Danish Mehmood Mughal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Young Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Young Chung.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shin, M., Mahboob, T., Mughal, D.M. et al. Deep reinforcement learning–based multi–channel spectrum sharing technology for next generation multi–operator cellular networks. Wireless Netw 29, 809–820 (2023). https://doi.org/10.1007/s11276-022-03179-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-022-03179-3

Keywords

Search

Navigation