Skip to main content
Log in

Multi-agent based optimal UAV deployment for throughput maximization in 5 G communications

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Unmanned aerial vehicle (UAV)-mounted base station is an emerging technology to provide controlled wireless communication for ground users. The high mobility of the UAV adds additional degrees of freedom for optimal deployment, which can significantly improve the the achievable rate. Therefore, the optimal deployment of multiple-UAV is one of the critical challenges in UAV-assisted communication systems. We formulate the UAVs deployment by considering a realistic UAV-ground channel model to maximize the sum rate. The problem formulated is a non-convex and NP-hard problem. To alleviate this problem, a multi-agent deep deterministic policy gradient (DDPG) approach is proposed to obtain optimal UAVs deployment. Each UAV is modeled as an independent agent, learns the mapping of the observed state to the position deployment based on centralized training and distributed execution architecture. Indeed, we developed an adaptive UAVs deployment scheme that uses the distributed DDPG algorithm. Finally, simulation results reveal how the suggested approach for UAV-assisted communication systems outperforms others, by which the average sum rate can be increased by \(17\%\) to \(86\%\).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

References

  1. Chettri, L., & Bera, R. (2019). A comprehensive survey on internet of things (iot) toward 5g wireless systems. IEEE Internet of Things Journal, 7(1), 16–32.

    Article  Google Scholar 

  2. Liu, X., Ding, H., & Hu, S. (2020). Uplink resource allocation for noma-based hybrid spectrum access in 6g-enabled cognitive internet of things. IEEE Internet of Things Journal, 8(20), 15049–15058.

    Article  Google Scholar 

  3. Hua, M., Wang, Y., Li, C., Huang, Y., & Yang, L. (2019). Energy-efficient optimization for uav-aided cellular offloading. IEEE Wireless Communications Letters, 8(3), 769–772.

    Article  Google Scholar 

  4. Liu, X., Sun, Q., Lu, W., Wu, C., & Ding, H. (2020). Big-data-based intelligent spectrum sensing for heterogeneous spectrum communications in 5g. IEEE Wireless Communications, 27(5), 67–73.

    Article  Google Scholar 

  5. Li, B., Fei, Z., & Zhang, Y. (2018). Uav communications for 5g and beyond: Recent advances and future trends. IEEE Internet of Things Journal, 6(2), 2241–2263.

    Article  Google Scholar 

  6. Ullah, Z., Al-Turjman, F., & Mostarda, L. (2020). Cognition in uav-aided 5g and beyond communications: A survey. IEEE Transactions on Cognitive Communications and Networking, 6(3), 872–891.

    Article  Google Scholar 

  7. Gupta, N., Agarwal, S., & Mishra, D. (2021). UAV deployment for throughput maximization in a UAV-Assisted cellular communications. Paper presented at the 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 1055–1060, September 2021

  8. Zhang, C., Zhang, L., Zhu, L., Zhang, T., Xiao, Z., & Xia, X.-G. (2021). 3d deployment of multiple uav-mounted base stations for uav communications. IEEE Transactions on Communications, 69(4), 2473–2488.

    Article  Google Scholar 

  9. Fadlullah, Z. M., Takaishi, D., Nishiyama, H., Kato, N., & Miura, R. (2016). A dynamic trajectory control algorithm for improving the communication throughput and delay in uav-aided networks. IEEE Network, 30(1), 100–105.

    Article  Google Scholar 

  10. Li, H., & Zhao, X. (2020). Throughput maximization with energy harvesting in uav-assisted cognitive mobile relay networks. IEEE Transactions on Cognitive Communications and Networking, 7(1), 197–209.

    Article  Google Scholar 

  11. Van der Bergh, B., Chiumento, A., & Pollin, S. (2016). Lte in the sky: Trading off propagation benefits with interference costs for aerial nodes. IEEE Communications Magazine, 54(5), 44–50.

    Article  Google Scholar 

  12. Al-Hourani, A., Kandeepan, S., & Jamalipour, A. (2014). Modeling air-to-ground path loss for low altitude platforms in urban environments. Paper presented at the IEEE global communications conference, 2898–2904, December

  13. Zhang, S., Zhang, H., Di, B., & Song, L. (2019). Cellular uav-to-x communications: Design and optimization for multi-uav networks. IEEE Transactions on Wireless Communications, 18(2), 1346–1359.

    Article  Google Scholar 

  14. Xiao, Z., Dong, H., Bai, L., Wu, D. O., & Xia, X.-G. (2019). Unmanned aerial vehicle base station (uav-bs) deployment with millimeter-wave beamforming. IEEE Internet of Things Journal, 7(2), 1336–1349.

    Article  Google Scholar 

  15. Zhang, J., Wu, Y., Min, G., Hao, F., & Cui, L. (2020). Balancing energy consumption and reputation gain of uav scheduling in edge computing. IEEE Transactions on Cognitive Communications and Networking, 6(4), 1204–1217.

    Article  Google Scholar 

  16. Al-Hourani, A., Kandeepan, S., & Lardner, S. (2014). Optimal lap altitude for maximum coverage. IEEE Wireless Communications Letters, 3(6), 569–572.

    Article  Google Scholar 

  17. Alzenad, M., El-Keyi, A., & Yanikomeroglu, H. (2017). 3-d placement of an unmanned aerial vehicle base station for maximum coverage of users with different qos requirements. IEEE Wireless Communications Letters, 7(1), 38–41.

    Article  Google Scholar 

  18. Alzenad, M., El-Keyi, A., Lagum, F., & Yanikomeroglu, H. (2017). 3-d placement of an unmanned aerial vehicle base station (uav-bs) for energy-efficient maximal coverage. IEEE Wireless Communications Letters, 6(4), 434–437.

    Article  Google Scholar 

  19. Košmerl, J., & Vilhar, A. (2014). Base stations placement optimization in wireless networks for emergency communications. Paper presented at IEEE international conference on communications workshops (ICC), 200–205 , Jun

  20. Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016). Efficient deployment of multiple unmanned aerial vehicles for optimal wireless coverage. IEEE Communications Letters, 20(8), 1647–1650.

    Article  Google Scholar 

  21. Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2017). Wireless communication using unmanned aerial vehicles (uavs): Optimal transport theory for hover time optimization. IEEE Transactions on Wireless Communications, 16(12), 8052–8066.

    Article  Google Scholar 

  22. Lyu, J., Zeng, Y., Zhang, R., & Lim, T. J. (2016). Placement optimization of uav-mounted mobile base stations. IEEE Communications Letters, 21(3), 604–607.

    Article  Google Scholar 

  23. Zhao, H., Wang, H., Wu, W., & Wei, J. (2018). Deployment algorithms for uav airborne networks toward on-demand coverage. IEEE Journal on Selected Areas in Communications, 36(9), 2015–2031.

    Article  Google Scholar 

  24. Rahmani, M., Dehghani, M. J., Xiao, P., Bashar, M., & Debbah, M. (2022). Multi-agent reinforcement learning-based pilot assignment for cell-free massive mimo systems. IEEE Access, 10, 120492–120502.

    Article  Google Scholar 

  25. Liu, X., Sun, C., Zhou, M., Wu, C., Peng, B., & Li, P. (2020). Reinforcement learning-based multislot double-threshold spectrum sensing with bayesian fusion for industrial big spectrum data. IEEE Transactions on Industrial Informatics, 17(5), 3391–3400.

    Article  Google Scholar 

  26. Liu, C. H., Chen, Z., Tang, J., Xu, J., & Piao, C. (2018). Energy-efficient uav control for effective and fair communication coverage: A deep reinforcement learning approach. IEEE Journal on Selected Areas in Communications, 36(9), 2059–2070.

    Article  Google Scholar 

  27. Challita, U., Saad, W., & Bettstetter, C. (2018). Cellular-connected UAVs over 5G: Deep reinforcement learning for interference management. Preprint at https://arxiv.org/pdf/1801.05500.pdf

  28. Liu, X., Liu, Y., & Chen, Y. (2019). Reinforcement learning in multiple-uav networks: Deployment and movement design. IEEE Transactions on Vehicular Technology, 68(8), 8036–8049.

    Article  Google Scholar 

  29. Zhou, Y., Ma, X., Hu, S., Zhou, D., Cheng, N., & Lu, N. (2021). Qoe-driven adaptive deployment strategy of multi-uav networks based on hybrid deep reinforcement learning. IEEE Internet of Things Journal

  30. Dai, Z., Zhang, Y., Zhang, W., Luo, X., & He, Z. (2022). A multi-agent collaborative environment learning method for uav deployment and resource allocation. IEEE Transactions on Signal and Information Processing over Networks, 8, 120–130.

    Article  Google Scholar 

  31. Mozaffari, M., Saad, W., Bennis, M., & Debbah, M. (2016). Unmanned aerial vehicle with underlaid device-to-device communications: Performance and tradeoffs. IEEE Transactions on Wireless Communications, 15(6), 3949–3963.

    Article  Google Scholar 

  32. Zeng, Y., Wu, Q., & Zhang, R. (2019). Accessing from the sky: A tutorial on uav communications for 5g and beyond. Proceedings of the IEEE, 107(12), 2327–2375.

    Article  Google Scholar 

  33. Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. Cambridge, MA, USA, Boston: MIT press.

    Google Scholar 

  34. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015). Continuous control with deep reinforcement learning. [online]. Available: https://arxiv.org/pdf/1509.02971.pdf

  35. Rahmani, M., Bashar, M., Dehghani, M.J., Xiao, P., Tafazolli, R., & Debbahz, M. Deep Reinforcement Learning-based Power Allocation in Uplink Cell-Free Massive MIMO. Paper presented at the IEEE Wireless Communications and Networking Conference (IEEE WCNC 2022)-Boosting Verticals into Wireless Orbit

  36. Rahmani, M., Bashar, M., Dehghani, M.J., Akbari, A., Xiao, P., Tafazolli, R., & Debbah, M.: (2022). Deep reinforcement learning-based sum rate fairness trade-off for cell-free mmimo. IEEE Transactions on Vehicular Technology

  37. Liu, Y., Liu, K., Han, J., Zhu, L., Xiao, Z., & Xia, X.-G. (2020). Resource allocation and 3-d placement for uav-enabled energy-efficient iot communications. IEEE Internet of Things Journal, 8(3), 1322–1333.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jasem Jamali.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baghnoi, F.M., Jamali, J., Taghizadeh, M. et al. Multi-agent based optimal UAV deployment for throughput maximization in 5 G communications. Wireless Netw 30, 2285–2296 (2024). https://doi.org/10.1007/s11276-023-03641-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03641-w

Keywords

Navigation