Abstract
Scheduling plays a pivotal role in multi-user wireless communications, since the quality of service of various users largely depends upon the allocated radio resources. In this paper, we propose a novel scheduling algorithm with contiguous frequency-domain resource allocation (FDRA) based on deep reinforcement learning (DRL) that jointly selects users and allocates resource blocks (RBs). The scheduling problem is modeled as a Markov decision process, and a DRL agent determines which user and how many consecutive RBs for that user should be scheduled at each RB allocation step. The state space, action space, and reward function are delicately designed to train the DRL network. More specifically, the original quasi-continuous action space, which is inherent to contiguous FDRA, is refined into a finite and discrete action space to obtain a trade-off between the inference latency and system performance. Simulation results show that the proposed DRL-based scheduling algorithm outperforms other representative baseline schemes while having lower online computational complexity.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
3GPP TR 38.824, V16.0.0. Study on physical layer enhancements for NR ultra-reliable and low latency case, March 2019
3GPP TR 38.901, V16.1.0. Study on channel model for frequencies from 0.5 to 100 GHz, December 2019
3GPP TS 38.211, V16.3.0. NR; Physical channels and modulation, September 2020
3GPP TS 38.213, V16.3.0. NR; Physical layer procedures for control, September 2020
3GPP TS 38.214, V16.3.0. NR; Physical layer procedures for data, September 2020
3GPP TS 38.331, V16.1.0. NR; Radio resource control protocol specification, July 2020
He, X., Wang, K., Huang, H., Miyazaki, T., Wang, Y., Guo, S.: Green resource allocation based on deep reinforcement learning in content-centric IoT. IEEE Trans. Emerg. Top. Comput. 8(3), 781–796 (2020)
Li, X., Alkhateeb, A.: Deep learning for direct hybrid precoding in millimeter wave massive MIMO systems. In: 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pp. 800–805 (2019)
Li, X., Alkhateeb, A., Tepedelenlioğlu, C.: Generative adversarial estimation of channel covariance in vehicular millimeter wave systems. In: 2018 52nd Asilomar Conference on Signals, Systems, and Computers, pp. 1572–1576 (2018)
Luong, N.C., et al.: Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21(4), 3133–3174 (2019)
Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (2014)
Sun, S., Moon, S.: Practical scheduling algorithms with contiguous resource allocation for next-generation wireless systems. IEEE Wirel. Commun. Lett. 10(4), 725–729 (2021)
Sun, S., Moon, S., Fwu, J.: Practical link adaptation algorithm with power density offsets for 5G uplink channels. IEEE Wirel. Commun. Lett. 9(6), 851–855 (2020)
Tse, D.: Forward link multiuser diversity through proportional fair scheduling. Presentation at Bell Labs, August 1999
Tsiropoulou, E.E., Kapoukakis, A., Papavassiliou, S.: Uplink resource allocation in SC-FDMA wireless networks: a survey and taxonomy. Comput. Netw. 96, 1–28 (2016)
Wong, I.C., Oteri, O.F., McCoy, J.W.: Resource allocation in multi data stream communication link. U.S. Patent 7 911 934, March 2011
Xu, Z., Wang, Y., Tang, J., Wang, J., Gursoy, M.C.: A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs. In: 2017 IEEE International Conference on Communications (ICC), pp. 1–6 (2017)
Yan, H., Ashikhmin, A., Yang, H.: Optimally supporting IoT with cell-free massive MIMO. In: 2020 IEEE Global Communications Conference (GLOBECOM), pp. 1–6 (2020)
Yan, H., Ashikhmin, A., Yang, H.: A scalable and energy efficient IoT system supported by cell-free massive MIMO. IEEE Internet Things J. (2021)
Yan, H., Lu, I.T.: BS-UE association and power allocation in heterogeneous massive MIMO systems. IEEE Access 8, 184045–184060 (2020)
Ye, H., Li, G.Y., Juang, B.F.: Deep reinforcement learning based resource allocation for V2V communications. IEEE Trans. Veh. Technol. 68(4), 3163–3173 (2019)
Zhang, C., Patras, P., Haddadi, H.: Deep learning in mobile and wireless networking: a survey. IEEE Commun. Surv. Tutor. 21(3), 2224–2287 (2019)
Zhao, N., Liang, Y., Niyato, D., Pei, Y., Wu, M., Jiang, Y.: Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks. IEEE Trans. Wirel. Commun. 18(11), 5141–5152 (2019)
Acknowledgment
The authors would like to thank the Next Generation and Standards Group (NGS) in Intel Corporation for their great support of this work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, S., Li, X. (2021). Deep-Reinforcement-Learning-Based Scheduling with Contiguous Resource Allocation for Next-Generation Wireless Systems. In: Arai, K. (eds) Intelligent Computing. Lecture Notes in Networks and Systems, vol 284. Springer, Cham. https://doi.org/10.1007/978-3-030-80126-7_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-80126-7_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80125-0
Online ISBN: 978-3-030-80126-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)