Abstract
Unmanned aerial vehicle (UAV)-mounted base station is an emerging technology to provide controlled wireless communication for ground users. The high mobility of the UAV adds additional degrees of freedom for optimal deployment, which can significantly improve the the achievable rate. Therefore, the optimal deployment of multiple-UAV is one of the critical challenges in UAV-assisted communication systems. We formulate the UAVs deployment by considering a realistic UAV-ground channel model to maximize the sum rate. The problem formulated is a non-convex and NP-hard problem. To alleviate this problem, a multi-agent deep deterministic policy gradient (DDPG) approach is proposed to obtain optimal UAVs deployment. Each UAV is modeled as an independent agent, learns the mapping of the observed state to the position deployment based on centralized training and distributed execution architecture. Indeed, we developed an adaptive UAVs deployment scheme that uses the distributed DDPG algorithm. Finally, simulation results reveal how the suggested approach for UAV-assisted communication systems outperforms others, by which the average sum rate can be increased by \(17\%\) to \(86\%\).
Similar content being viewed by others
Data Availability
Data sharing is not applicable to this article as no new data were created or analyzed in this study.
References
Chettri, L., & Bera, R. (2019). A comprehensive survey on internet of things (iot) toward 5g wireless systems. IEEE Internet of Things Journal, 7(1), 16–32.
Liu, X., Ding, H., & Hu, S. (2020). Uplink resource allocation for noma-based hybrid spectrum access in 6g-enabled cognitive internet of things. IEEE Internet of Things Journal, 8(20), 15049–15058.
Hua, M., Wang, Y., Li, C., Huang, Y., & Yang, L. (2019). Energy-efficient optimization for uav-aided cellular offloading. IEEE Wireless Communications Letters, 8(3), 769–772.
Liu, X., Sun, Q., Lu, W., Wu, C., & Ding, H. (2020). Big-data-based intelligent spectrum sensing for heterogeneous spectrum communications in 5g. IEEE Wireless Communications, 27(5), 67–73.
Li, B., Fei, Z., & Zhang, Y. (2018). Uav communications for 5g and beyond: Recent advances and future trends. IEEE Internet of Things Journal, 6(2), 2241–2263.
Ullah, Z., Al-Turjman, F., & Mostarda, L. (2020). Cognition in uav-aided 5g and beyond communications: A survey. IEEE Transactions on Cognitive Communications and Networking, 6(3), 872–891.
Gupta, N., Agarwal, S., & Mishra, D. (2021). UAV deployment for throughput maximization in a UAV-Assisted cellular communications. Paper presented at the 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 1055–1060, September 2021
Zhang, C., Zhang, L., Zhu, L., Zhang, T., Xiao, Z., & Xia, X.-G. (2021). 3d deployment of multiple uav-mounted base stations for uav communications. IEEE Transactions on Communications, 69(4), 2473–2488.
Fadlullah, Z. M., Takaishi, D., Nishiyama, H., Kato, N., & Miura, R. (2016). A dynamic trajectory control algorithm for improving the communication throughput and delay in uav-aided networks. IEEE Network, 30(1), 100–105.
Li, H., & Zhao, X. (2020). Throughput maximization with energy harvesting in uav-assisted cognitive mobile relay networks. IEEE Transactions on Cognitive Communications and Networking, 7(1), 197–209.
Van der Bergh, B., Chiumento, A., & Pollin, S. (2016). Lte in the sky: Trading off propagation benefits with interference costs for aerial nodes. IEEE Communications Magazine, 54(5), 44–50.
Al-Hourani, A., Kandeepan, S., & Jamalipour, A. (2014). Modeling air-to-ground path loss for low altitude platforms in urban environments. Paper presented at the IEEE global communications conference, 2898–2904, December
Zhang, S., Zhang, H., Di, B., & Song, L. (2019). Cellular uav-to-x communications: Design and optimization for multi-uav networks. IEEE Transactions on Wireless Communications, 18(2), 1346–1359.
Xiao, Z., Dong, H., Bai, L., Wu, D. O., & Xia, X.-G. (2019). Unmanned aerial vehicle base station (uav-bs) deployment with millimeter-wave beamforming. IEEE Internet of Things Journal, 7(2), 1336–1349.
Zhang, J., Wu, Y., Min, G., Hao, F., & Cui, L. (2020). Balancing energy consumption and reputation gain of uav scheduling in edge computing. IEEE Transactions on Cognitive Communications and Networking, 6(4), 1204–1217.
Al-Hourani, A., Kandeepan, S., & Lardner, S. (2014). Optimal lap altitude for maximum coverage. IEEE Wireless Communications Letters, 3(6), 569–572.
Alzenad, M., El-Keyi, A., & Yanikomeroglu, H. (2017). 3-d placement of an unmanned aerial vehicle base station for maximum coverage of users with different qos requirements. IEEE Wireless Communications Letters, 7(1), 38–41.
Alzenad, M., El-Keyi, A., Lagum, F., & Yanikomeroglu, H. (2017). 3-d placement of an unmanned aerial vehicle base station (uav-bs) for energy-efficient maximal coverage. IEEE Wireless Communications Letters, 6(4), 434–437.
Košmerl, J., & Vilhar, A. (2014). Base stations placement optimization in wireless networks for emergency communications. Paper presented at IEEE international conference on communications workshops (ICC), 200–205 , Jun
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016). Efficient deployment of multiple unmanned aerial vehicles for optimal wireless coverage. IEEE Communications Letters, 20(8), 1647–1650.
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2017). Wireless communication using unmanned aerial vehicles (uavs): Optimal transport theory for hover time optimization. IEEE Transactions on Wireless Communications, 16(12), 8052–8066.
Lyu, J., Zeng, Y., Zhang, R., & Lim, T. J. (2016). Placement optimization of uav-mounted mobile base stations. IEEE Communications Letters, 21(3), 604–607.
Zhao, H., Wang, H., Wu, W., & Wei, J. (2018). Deployment algorithms for uav airborne networks toward on-demand coverage. IEEE Journal on Selected Areas in Communications, 36(9), 2015–2031.
Rahmani, M., Dehghani, M. J., Xiao, P., Bashar, M., & Debbah, M. (2022). Multi-agent reinforcement learning-based pilot assignment for cell-free massive mimo systems. IEEE Access, 10, 120492–120502.
Liu, X., Sun, C., Zhou, M., Wu, C., Peng, B., & Li, P. (2020). Reinforcement learning-based multislot double-threshold spectrum sensing with bayesian fusion for industrial big spectrum data. IEEE Transactions on Industrial Informatics, 17(5), 3391–3400.
Liu, C. H., Chen, Z., Tang, J., Xu, J., & Piao, C. (2018). Energy-efficient uav control for effective and fair communication coverage: A deep reinforcement learning approach. IEEE Journal on Selected Areas in Communications, 36(9), 2059–2070.
Challita, U., Saad, W., & Bettstetter, C. (2018). Cellular-connected UAVs over 5G: Deep reinforcement learning for interference management. Preprint at https://arxiv.org/pdf/1801.05500.pdf
Liu, X., Liu, Y., & Chen, Y. (2019). Reinforcement learning in multiple-uav networks: Deployment and movement design. IEEE Transactions on Vehicular Technology, 68(8), 8036–8049.
Zhou, Y., Ma, X., Hu, S., Zhou, D., Cheng, N., & Lu, N. (2021). Qoe-driven adaptive deployment strategy of multi-uav networks based on hybrid deep reinforcement learning. IEEE Internet of Things Journal
Dai, Z., Zhang, Y., Zhang, W., Luo, X., & He, Z. (2022). A multi-agent collaborative environment learning method for uav deployment and resource allocation. IEEE Transactions on Signal and Information Processing over Networks, 8, 120–130.
Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016). Unmanned aerial vehicle with underlaid device-to-device communications: Performance and tradeoffs. IEEE Transactions on Wireless Communications, 15(6), 3949–3963.
Zeng, Y., Wu, Q., & Zhang, R. (2019). Accessing from the sky: A tutorial on uav communications for 5g and beyond. Proceedings of the IEEE, 107(12), 2327–2375.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. Cambridge, MA, USA, Boston: MIT press.
Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. [online]. Available: https://arxiv.org/pdf/1509.02971.pdf
Rahmani, M., Bashar, M., Dehghani, M.J., Xiao, P., Tafazolli, R., & Debbahz, M. Deep Reinforcement Learning-based Power Allocation in Uplink Cell-Free Massive MIMO. Paper presented at the IEEE Wireless Communications and Networking Conference (IEEE WCNC 2022)-Boosting Verticals into Wireless Orbit
Rahmani, M., Bashar, M., Dehghani, M.J., Akbari, A., Xiao, P., Tafazolli, R., & Debbah, M.: (2022). Deep reinforcement learning-based sum rate fairness trade-off for cell-free mmimo. IEEE Transactions on Vehicular Technology
Liu, Y., Liu, K., Han, J., Zhu, L., Xiao, Z., & Xia, X.-G. (2020). Resource allocation and 3-d placement for uav-enabled energy-efficient iot communications. IEEE Internet of Things Journal, 8(3), 1322–1333.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Baghnoi, F.M., Jamali, J., Taghizadeh, M. et al. Multi-agent based optimal UAV deployment for throughput maximization in 5 G communications. Wireless Netw 30, 2285–2296 (2024). https://doi.org/10.1007/s11276-023-03641-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-023-03641-w