Abstract
When the multi-UAVs cooperatively attack multi-tasks, the dynamic changes of environments can lead to a failure of the tasks. So a novel path re-planning algorithm of multiple Q-learning based on cooperative fuzzy C means clustering is proposed. Our approach first reflects the dynamic changes of re-planning space by updating the fuzzy cooperative matrix. Then, the key way-points on the current global paths are used as the initial clustering centers for the cooperative fuzzy C means clustering, which generates the classifications of space points for multi-tasks. Furthermore, we use the classifications as the state space of each task and the fuzzy cooperative matrix as the reward function of the Q-learning. So a multi Q-learning algorithm is presented to synchronously re-plan the paths for multi-UAVs at every step. The simulation results show that the method subtracts the re-planning space of the tasks and improves the search efficiency of the learning algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Pierre, D.M., Zakaria, N., Pal, A.J.: Self-Organizing Map approach to determining compromised solutions for multi-objective UAV path planning. In: 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV), pp. 995–1000. IEEE (2012)
Brezak, M., Petrovic, I.: Real-time approximation of clothoids with bounded error for path planning applications. IEEE Trans. Robot. 30(2), 507–515 (2014)
Berger, J., Boukhtouta, A., Benmoussa, A.: A new mixed-integer linear programming model for rescue path planning in uncertain adversarial environment. Comput. Oper. Res. 39(12), 3420–3430 (2012)
Zucker, M., Kuffner, J., Branicky, M.: Multipartite RRTs for rapid replanning in dynamic environments. In: 2007 IEEE International Conference on Robotics and Automation, pp. 1603–1609. IEEE (2007)
Wu, J., Zhang, D.-h.: Path planning for dyanmic target based on kalman filtering algorithm, D* algorithm. Electron. Opt. Control 21(8), 50–53 (2014)
Wagner, G., Choset, H.: M*: A complete multirobot path planning algorithm with performance bounds. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3260–3267. IEEE (2011)
Kok, J., Gonzalez, L.F., Kelson, N.: FPGA implementation of an evolutionary algorithm for autonomous unmanned aerial vehicle on-board path planning. IEEE Trans. Evol. Comput. 17(2), 272–281 (2013)
Lu, L., Gong, D.: Robot path planning in unknown environments using particle swarm optimization. In: Fourth International Conference on Natural Computation, ICNC 2008, vol. 4, pp. 422–426. IEEE (2008)
Ragi, S., Chong, E.K.P.: UAV path planning in a dynamic environment via partially observable Markov decision process. IEEE Trans. Aerosp. Electron. Syst. 49(4), 2397–2412 (2013)
Feyzabadi, S., Carpin, S.: Risk-aware path planning using hirerachical constrained Markov decision processes. In: 2014 IEEE International Conference on Automation Science and Engineering (CASE), pp. 297–303. IEEE (2014)
Zhang, B., Mao, Z., Liu, W., et al.: Geometric reinforcement learning for path planning of UAVs. J. Intell. Robot. Syst. 77(2), 391–409 (2015)
Wu, Y.-r., Wu, Y.-l., Ding, W., et al.: Dual-Aircraft cooperative path planning based on Q learning. Electron. Opt. Control 21(8), 15–19 (2014)
Zhao, M., Zhao, L., Su, X., Ma, P., Zhang, Y.: A cultural algorithm with spatial fuzzy set to solve multi-UAVs cooperative path planning in a three dimensional environment. J. Harbin Inst. Technol. 47(10), 29–34 (2015)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Xu, X.: Reinforcement Learning, Approximate Dynamic Programming. Science Press, Beijing (2010)
Yin, X.: The Study of Multi-Agent Cooperative Reinforcement Learning Methods. The National University of Defense Technology, Changsha (2003)
Tan, X.: Fuzzy Clustering Algorithm Based on Collaborative Research. Changsha University of Science & Technology, Changsha (2013)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Su, Xh., Zhao, M., Zhao, Ll., Zhang, Yh. (2016). A Novel Multi Stage Cooperative Path Re-planning Method for Multi UAV. In: Booth, R., Zhang, ML. (eds) PRICAI 2016: Trends in Artificial Intelligence. PRICAI 2016. Lecture Notes in Computer Science(), vol 9810. Springer, Cham. https://doi.org/10.1007/978-3-319-42911-3_40
Download citation
DOI: https://doi.org/10.1007/978-3-319-42911-3_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42910-6
Online ISBN: 978-3-319-42911-3
eBook Packages: Computer ScienceComputer Science (R0)