Jithin Jagannath
ABSTRACT
Dynamic resource allocation plays a critical role in the next generation of intelligent wireless communication systems. Machine learning has been leveraged as a powerful tool to make strides in this domain. In most cases, the progress has been limited to simulations due to the challenging nature of hardware deployment of these solutions. In this paper, for the first time, we design and deploy deep reinforcement learning (DRL)-based power control agents on the GPU embedded software defined radios (SDRs). To this end, we propose an end-to-end framework (MR-iNet Gym) where the simulation suite and the embedded SDR development work cohesively to overcome real-world implementation hurdles. To prove feasibility, we consider the problem of distributed power control for code-division multiple access (DS-CDMA)-based LPI/D transceivers. We first build a DS-CDMA ns3 module that interacts with the OpenAI Gym environment. Next, we train the power control DRL agents in this ns3-gym simulation environment in a scenario that replicates our hardware testbed. Next, for edge (embedded on-device) deployment, the trained models are optimized for real-time operation without loss of performance. Hardware-based evaluation verifies the efficiency of DRL agents over traditional distributed constrained power control (DCPC) algorithm. More significantly, as the primary goal, this is the first work that has established the feasibility of deploying DRL to provide optimized distributed resource allocation for next-generation of GPU-embedded radios.
- Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arxiv: 1606.01540 [cs.LG]Google Scholar
- Tamer ElBatt and Anthony Ephremides. 2004. Joint scheduling and power control for wireless ad hoc networks. IEEE Transactions on Wireless communications , Vol. 3, 1 (2004), 74--85.Google ScholarDigital Library
- Piotr Gawłowicz and Anatolij Zubow. 2019. ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research. In ACM Intl. Conf. on Modeling, Analysis and Sim. of Wireless and Mobile Systems (Miami Beach, USA).Google ScholarDigital Library
- Sudheer A Grandhi, Jens Zander, and Roy Yates. 1994. Constrained power control. Wireless Personal Communications , Vol. 1, 4 (1994), 257--270.Google ScholarCross Ref
- Ade Hermawan, Rizki Ginanjar, Dong-Seong Kim, and Jae-Min Lee. 2020. CNN-Based Automatic Modulation Classification for Beyond 5G Communications. IEEE Communications Letters , Vol. 24, 5 (2020), 1038--1041.Google ScholarCross Ref
- Md Habibul Islam, Ying-Chang Liang, and Anh Tuan Hoang. 2007. Distributed power and admission control for cognitive radio networks using antenna arrays. In 2007 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks. IEEE, 250--253.Google ScholarDigital Library
- Anu Jagannath, Jithin Jagannath, and Tommaso Melodia. 2021. Redefining Wireless Communication for 6G: Signal Processing Meets Deep Learning with Deep Unfolding . IEEE Transaction on Artificial Intelligence (2021).Google Scholar
- Jithin Jagannath, Nicholas Polosky, Dan Connor, Lakshmi Theagarajan, Brendan Sheaffer, Svetlana Foulke, and Pramod Varshney. 2018. Artificial Neural Network based Automatic Modulation Classifier for Software Defined Radios. In Proc. of IEEE Intl, Conf. on Communications (ICC). Kansas City, USA.Google Scholar
- Jithin Jagannath, Nicholas Polosky, Anu Jagannath, Francesco Restuccia, and Tommaso Melodia. 2019. Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey. Ad Hoc Networks (Elsevier) (2019).Google Scholar
- Jithin Jagannath, Nicholas Polosky, Anu Jagannath, Francesco Restuccia, and Tommaso Melodia. 2020. Neural Networks for Signal Intelligence: Theory and Practice, . In Machine Learning for Future Wireless Communications, F.L. Luo (Ed.). John Wiley & Sons, Limited. 2019029933Google Scholar
- Xingjian Li, Jun Fang, Wen Cheng, Huiping Duan, Zhi Chen, and Hongbin Li. 2018. Intelligent power control for spectrum sharing in cognitive radios: A deep reinforcement learning approach. IEEE access , Vol. 6 (2018).Google Scholar
- Nguyen Cong Luong, Dinh Thai Hoang, Shimin Gong, Dusit Niyato, Ping Wang, Ying-Chang Liang, and Dong In Kim. 2019. Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. IEEE Communications Surveys Tutorials , Vol. 21, 4 (2019), 3133--3174.Google ScholarDigital Library
- Yasar Sinan Nasir and Dongning Guo. 2020. Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks. In 2020 54th Asilomar Conference on Signals, Systems, and Computers. IEEE, 398--402.Google Scholar
- Keyvan Ramezanpour and Jithin Jagannath. 2022. Intelligent Zero Trust Architecture for 5G/6G Networks: Principles, Challenges, and the Role of Machine Learning in the context of O-RAN . arXiv preprint arXiv:2105.01478 (2022).Google Scholar
- Kaiming Shen and Wei Yu. 2018. Fractional programming for communication systems-Part I: Power control and beamforming. IEEE Transactions on Signal Processing , Vol. 66, 10 (2018), 2616--2630.Google ScholarCross Ref
- Haoran Sun, Xiangyi Chen, Qingjiang Shi, Mingyi Hong, Xiao Fu, and Nicholas D Sidiropoulos. 2018. Learning to optimize: Training deep neural networks for interference management. IEEE Transactions on Signal Processing (2018).Google Scholar
- John Tadrous, Ahmed Sultan, Mohammed Nafie, and Amr El-Keyi. 2010. Power control for constrained throughput maximization in spectrum shared networks. In IEEE Global Communications Conference (GLOBECOM) .Google ScholarCross Ref
- Chen Tessler, Nadav Merlis, and Shie Mannor. 2019. Stabilizing off-policy reinforcement learning with conservative policy gradients. (2019).Google Scholar
- Mingbo Xiao, Ness B Shroff, and Edwin KP Chong. 2003. A utility-based power-control scheme in wireless cellular systems. IEEE/ACM Transactions On Networking , Vol. 11, 2 (2003), 210--221.Google ScholarDigital Library
Index Terms
- MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio
Recommendations
Marconi-Rosenblatt Framework for Intelligent Networks (MR-iNet Gym): For Rapid Design and Implementation of Distributed Multi-agent Reinforcement Learning Solutions for Wireless Networks
AbstractWe present the Marconi-Rosenblatt Framework for Intelligent Networks (MR-iNet Gym) an open-source architecture designed for accelerating research and development of novel reinforcement learning applied to distributed wireless networks. To ensure ...
IoT Sensor Gym: Training Autonomous IoT Devices with Deep Reinforcement Learning
IoT '19: Proceedings of the 9th International Conference on the Internet of ThingsWe describe IoT Sensor Gym, a framework to train the behavior of constrained IoT devices using deep reinforcement learning. We focus on the main architectural choices to align problems from the IoT domain with cutting-edge reinforcement learning ...
Deep Reinforcement Learning based reliable spectrum sensing under SSDF attacks in Cognitive Radio networks
AbstractIn Spectrum Sensing Data Falsification (SSDF) attacks, malicious Secondary Users (SUs) actively send erroneous local sensing results to the Fusion Centre (FC) that influence the actual outcomes of Cooperative Spectrum Sensing (CSS). ...
Comments