Skip to main content
SpringerLink
Log in

Novel Reinforcement Learning Guided Enhanced Variable Weight Grey Wolf Optimization (RLV-GWO) Algorithm for Multi-UAV Path Planning

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

The multi agent path planning strategy for unmanned aerial vehicles (UAVs) might play a crucial role in seeking the most feasible path in 3D environment owning to power limitations and other environmental factors. UAVs path planning is a high precision task that is needed for wide range of commercial, military, and rescue operations. However, the path planning for multi-UAVs is a tedious task due to needing an optimal path between source and destination points. There are numerous algorithms available in literature for providing solutions to path planning problems; however, they do not provide an efficient solution, especially in the face of three-dimensional (3-D) aviation workspace. Hence, in this paper, novel reinforcement learning based enhanced variable weight grey wolf optimization algorithm named RLV-GWO is proposed to address this issue. In the proposed algorithm, two prominent variants of GWO named modified grey wolf optimizer (MGWO) and variable weight grey wolf optimizer (VM-GWO) are integrated to advance the performance of baseline model GWO through mitigating the convergence rate and escalating the algorithm speed by assigning variable weights to participating candidates during exploration and hunting process. Furthermore, reinforcement learning is introduced to control the switching operations of individual candidates adaptively based on their accumulated performance. The simulation results demonstrate that the proposed RLV-GWO algorithm can acquire feasible and effective path planning solution for UAVs in three dimensional environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Code Availability

The codes shall be available on reasonable request to the readers.

References

  1. Qu, C., Gai, W., Zhang, J., & Zhong, M. (2020). A novel hybrid grey wolf optimizer algorithm for unmanned aerial vehicle (UAV) path planning. Knowledge-Based systems, 194, 105530. https://doi.org/10.1016/j.knosys.2020.105530

    Article  Google Scholar 

  2. Ming, G., Hong, T., & Peng, Z. (2021). The research status of robot cluster path planning technology. Journal of National University of Defense technology, 43(01), 127–138.

    Google Scholar 

  3. Liu, J., Wei, X., & Huang, H. (2021). An improved grey wolf optimization algorithm and its application in path planning. IEEE Access, 9, 121944–121956. https://doi.org/10.1109/ACCESS.2021.3108973

    Article  Google Scholar 

  4. Dewangan, R. K., & Saxena, P. (2022). Three dimensional route planning for multiple unmanned aerial vehicles using Salp Swarm Algorithm. Journal of Experimental and Theoretical Artificial Intelligence, 49, 1–20. https://doi.org/10.1007/s10489-018-1384-y

    Article  Google Scholar 

  5. Singh, L., & Jaffrey, Z. A. (2018). Computerized detection of breast cancer in digital mammograms. International Journal of Computers and Applications, 40(2), 98–109. https://doi.org/10.1080/1206212X.2017.1395131

    Article  Google Scholar 

  6. Singh, L., Alam, A., & Kumar, K. V. (2021). Design of thermal imaging-based health condition monitoring and early fault detection technique for porcelain insulators using machine learning. Environmental Technology & Innovation, 24, 1–14. https://doi.org/10.1016/j.eti.2021.102000

    Article  Google Scholar 

  7. Kala, R., Shukla, A., & Tiwari, R. (2010). Fusion of probabilistic A* algorithm and fuzzy inference system for robotic path planning. Artificial Intelligence Review, 33, 307–327. https://doi.org/10.1007/s10462-010-9157-y

    Article  Google Scholar 

  8. Valavanis, K., & Vachtsevanos, G. J. (2015). Handbook of unmanned aerial vehicles. Springer.

    Book  MATH  Google Scholar 

  9. Radmanesh, M., Kumar, M., & Sarim, M. (2018). Grey wolf optimization-based sense and avoid algorithm in a Bayesian framework for multiple UAV path planning in an uncertain environment. Aerospace science & technology, 77, 168–179.

    Article  Google Scholar 

  10. Orozco-Rosas, U., Montiel, O., & Sepulveda, R. (2019). Mobile robot path planning using membrane evolutionary artificial potential field. Applied soft computing, 77, 236–251.

    Article  Google Scholar 

  11. Morales, N., Toledo, J., & Acosta, L. (2016). Path planning using a multiclass support vector machine. Applied Soft Computing, 43, 498–509. https://doi.org/10.1016/j.asoc.2016.02.037

    Article  Google Scholar 

  12. Wang, W., Deng, H., & Wu, X. (2018). Path planning of loaded pin-jointed bar mechanisms using Rapidly-exploring random tree method. Computers & Structures, 209, 65–73.

    Article  Google Scholar 

  13. Kobilarov, M. (2012). Cross-entropy randomized motion planning. The International Journal of Robotics Research, 31(7), 153–160.

    Article  Google Scholar 

  14. Das, P. K., Behera, H. S., & Panigrahi, B. K. (2016). A hybridization of an improved particle swarm optimization and gravitational search algorithm for multi-robot path planning. Swarm Evolutionary Computation, 28, 14–28. https://doi.org/10.1016/j.swevo.2015.10.011

    Article  Google Scholar 

  15. Bansal, J. C. (2019). Particle swarm optimization, evolutionary and swarm intelligence algorithms. Studies in Computational Intelligence, 8, 11–23. https://doi.org/10.1007/978-3-319-91341-4_2

    Article  Google Scholar 

  16. Deng, W., Xu, J., & Zhao, H. (2019). An improved ant colony optimization algorithm based on hybrid strategies for scheduling problem. IEEE Access, 7, 20281–20292. https://doi.org/10.1109/ACCESS.2019.2897580

    Article  Google Scholar 

  17. Yu, X. (2018). A self adaptive artificial bee colony algorithm based on global best for global optimization soft computing–A fusion of foundations. Methodologies and Applications, 22(9), 2935–2952. https://doi.org/10.1007/s00500-017-2547-1

    Article  MathSciNet  Google Scholar 

  18. Majumder, A., Majumder, A., & Bhaumik, R. (2021). Teaching–learning-based optimization algorithm for path planning and task allocation in multi-robot plant inspection system. Arabian Journal for Science and Engineering, 46, 8999–9021. https://doi.org/10.1007/s13369-021-05710-8

    Article  Google Scholar 

  19. Heidari, A. A., Mirjalili, S., Faris, H., Aljarah, I., Mafarja, M., & Chen, H. (2019). Harris hawks optimization: Algorithm and applications. Future Generation computer systems, 97, 849–872. https://doi.org/10.1016/j.future.2019.02.028

    Article  Google Scholar 

  20. Mafarja, M., & Mirjalili, S. (2017). Hybrid whale optimization algorithm with simulated annealing for feature selection. Neurocomputing, 260, 302–312. https://doi.org/10.1016/j.neucom.2017.04.053

    Article  Google Scholar 

  21. Dewangan, R. K., Shukla, A., & Godfrey, W. (2019). Three dimensional path planning using grey wolf optimizer for UAVs. Applied Intelligence, 49(7), 2201–2217. https://doi.org/10.1007/s10489-018-1384-y

    Article  Google Scholar 

  22. Yan, Y., Liang, Y., Zhang, H., Zhang, W., Feng, H., Wang, B., & Liao, Q. (2019). A two stage optimization method for unmanned aerial vehicle inspection of an oil and gas pipeline network. Petroleum Science, 16, 458–468. https://doi.org/10.1007/s12182-019-0305-y

    Article  Google Scholar 

  23. Qie, H., Shi, D., Shen, T., Xu, X., Li, Y., & Wang, L. (2019). Joint optimization of Multi-UAV Target Assignment and path planning based on Multi-Agent Reinforcement Learning. IEEE Access, 7, 146264–146272. https://doi.org/10.1109/ACCESS.2019.2943253

    Article  Google Scholar 

  24. Sun, S., Song, B., Wang, P., Dong, H., & Chen, X. (2019). Real time mission motion planner for multi-UAVs Cooperative Work using tri-level programming. IEEE transactions on Intelligent Transportation systems, 20, 1303–1315.

    Google Scholar 

  25. Phung, M. D., Quach, C. H., Dinh, T. H., & Ha, Q. (2017). Enhanced discrete particle swarm optimization path planning for UAV vision-based surface inspection. Automation in Construction, 81, 25–33. https://doi.org/10.1016/j.autcon.2017.04.013

    Article  Google Scholar 

  26. Shima, T., Rasmussen, S. J., Sparks, A. G., & Passino, K. M. (2006). Multiple task assignments for cooperating uninhabited aerial vehicles using genetic algorithms. Computers & Operations Research, 33(11), 3252–3269. https://doi.org/10.1016/j.cor.2005.02.039

    Article  MATH  Google Scholar 

  27. Shorakaei, H., Vahdani, M., Imani, B., & Gholami, A. (2016). Optimal cooperative path planning of unmanned aerial vehicles by a parallel genetic algorithm. Robotica, 34(4), 823–836.

    Article  Google Scholar 

  28. Ge, F., Li, K., Han, Y., Xu, W., & Wang, Y. (2020). Path planning of UAV for oilfield inspections in a three-dimensional dynamic environment with moving obstacles based on an improved pigeon-inspired optimization algorithm. Applied Intelligence, 50, 2800–2817. https://doi.org/10.1007/s10489-020-01650-2

    Article  Google Scholar 

  29. Khandelwal, A., Bhargava, A., Sharma, A., & Sharma, H. (2018). Modified Grey wolf optimization algorithm for transmission network expansion planning problem. Arabian Journal for Science and Engineering, 43, 2899–2908. https://doi.org/10.1007/s13369-017-2967-3

    Article  MATH  Google Scholar 

  30. Kumar, R., Singh, L., & Tiwari, R. (2021). Path planning for the autonomous robots using modified grey wolf optimization approach. Journal of Int. & Fuzzy systems, 40, 9453–9470. https://doi.org/10.3233/JIFS-201926

    Article  Google Scholar 

  31. Gao, Z. M., & Zhao, J. (2019). An improved grey wolf optimization algorithm with variable weights. Computational Intelligence and Neuroscience, 298, 1282. https://doi.org/10.1155/2019/2981282

    Article  Google Scholar 

  32. Gupta, S., & Deep, K. (2019). An efficient grey wolf optimizer with opposition based learning and chaotic local search for integer and mixed integer optimization problems. Arabian Journal for Science and Engineering, 44, 7277–7296. https://doi.org/10.1007/s13369-019-03806-w

    Article  Google Scholar 

  33. Kumar, R., Singh, L., & Tiwari, R. (2020). Comparison of two metaheuristic algorithms for Path Planning in Robotics. IEEE conference. https://doi.org/10.1109/IC3A48958.2020.233289

    Article  Google Scholar 

  34. Pradhan, M., Roy, P. K., & Pal, T. (2016). Grey wolf optimization applied to economic load dispatch problems. International Journal Electron Power Energy System., 83, 325–334. https://doi.org/10.1016/j.ijepes.2016.04.034

    Article  Google Scholar 

  35. Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Grey wolf optimizer. Advances in Engineering Software, 69(3), 46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007

    Article  Google Scholar 

  36. Zhang, X., & Duan, H. (2015). An improved constrained differential evolution algorithm for unmanned aerial vehicle Global route planning. Application of soft computing, 26, 270–284.

    Article  Google Scholar 

  37. Zhang, D. F., & Duan, H. B. (2018). Social class pigeon-inspired optimization and timestamp segmentation for multi-UAV operative path planning. Neurocomputing, 313, 2029–2046. https://doi.org/10.1016/j.neucom.2018.06.032

    Article  Google Scholar 

  38. Wang, Y. B., Bai, P., & Liang, H. L. (2019). Reconnaissance mission conducted by UAV swarm based on distributed PSO path planning algorithms. IEEE Access, 7, 105086–105099. https://doi.org/10.1109/ACCESS.2019.2932008

    Article  Google Scholar 

  39. Wolpert, D., & Macready, W. (1997). No free lunch theorems for optimization. IEEE Transactions Evolutionary Computing, 1(1), 67–82. https://doi.org/10.1109/4235.585893

    Article  Google Scholar 

  40. Qu, C., Gai, W., Zhong, M., & Zhang, J. (2020). A novel reinforcement learning based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning. Applied soft computing, 89, 10699. https://doi.org/10.1016/j.asoc.2020.106099

    Article  Google Scholar 

  41. Loquercio, A., Kaufmann, E., Ranftl, R., Dosovitskiy, A., Koltun, V., & Scaramuzza, D. (2019). Deep drone racing: From simulation to reality with domain randomization. IEEE Transactions on Robotics, 36, 1552–3098. https://doi.org/10.1109/TRO.2019.2942989

    Article  Google Scholar 

Download references

Funding

This research was not supported by any funding agency.

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Dr. A.P.J Abdul Kalam Technical University, Lucknow, UP, 226031, India

    Rajeev Kumar

  2. Department Electronics and Communication Engineering, Noida Institute of Engineering and Technology, Greater Noida, UP, 201306, India

    Laxman Singh

  3. Deputy General Manager (DGM), CEEKH Edunix Pvt Ltd, Noida, UP, India

    Rajdev Tiwari

Authors
  1. Rajeev Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Laxman Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajdev Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laxman Singh.

Ethics declarations

Conflicts of interest

Authors have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, R., Singh, L. & Tiwari, R. Novel Reinforcement Learning Guided Enhanced Variable Weight Grey Wolf Optimization (RLV-GWO) Algorithm for Multi-UAV Path Planning. Wireless Pers Commun 131, 2093–2123 (2023). https://doi.org/10.1007/s11277-023-10534-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-023-10534-w

Keywords

Search

Navigation