Multi-agent Deep Reinforcement Learning-Based UAV-Enable NOMA Communication Networks Optimization

Abstract

In natural disasters or emergency situations, communication base stations may suffer damage or become unavailable. In such scenarios, Unmanned Aerial Vehicles (UAVs) can serve as communication relay devices, assisting in the restoration of disrupted communication services. In this paper, we investigate the optimization of multi-UAV-enable communication networks with Non-Orthogonal Multiple Access (NOMA). Our objective is to maximize the system throughput while ensuring fairness by optimizing decisions such as user-UAV association, subchannel number, subchannel allocation, transmission power allocation, and UAV trajectory. Due to the complexity of the problem, we decompose the original problem into two subproblems: 1) the user-UAV association problem and 2) the communication resource allocation and UAV trajectory optimization problem. To address the first subproblem, we propose a User-UAV Association (U2A) algorithm based on K-means. However, due to the tight coupling between subchannel number and subchannel allocation decisions, solving the second subproblem directly is challenging. Therefore, we propose a novel Two Time-scale communication Resource Allocation and Trajectory Optimization (TTRATO) algorithm based on Multi-Agent Deep Reinforcement Learning (MADRL). Simulation experiments demonstrate the effectiveness of our proposed algorithms.

This work was supported in part by the National Natural Science Foundation of China (Nos. 62202059, 61872044)