Abstract
Navigating an unknown indoor environment like a building is quite challenging as Global Positioning System-based solutions are hard or near impossible. However, it is a crucial problem as it has many applications, such as search and rescue. The rescue operations should start at the earliest to avoid casualties. Hence, the navigation process should be completed quickly. The navigation process can be done and accelerated with multiple agents/robots. The Q-learning-based approach can help navigation, but it is suitable only if state and action spaces are low. This paper suggests a distributed multi-agent deep Q-learning algorithm for indoor navigation in a complex environment where the number of states is large.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Comerio, M.C.: Disaster Hits Home: New Policy for Urban Housing Recovery. University of California Press, Berkeley (1998)
Van der Hoek, W., Wooldridge, M.: Multi-agent systems. Found. Artif. Intell. 3, 887–928 (2008)
Dorri, A., Kanhere, S.S., Jurdak, R.: Multi-agent systems: a survey. IEEE Access 6, 28573–28593 (2018)
Nath, A., Niyogi, R., Singh, T., Kumar, V.: Multi-agent Q-learning based navigation in an unknown environment. In: Barolli, L., Hussain, F., Enokido, T. (eds.) AINA 2022. LNNS, vol. 449, pp. 330–340. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99584-3_29
Ruan, X., Ren, D., Zhu, X., Huang, J.: Mobile robot navigation based on deep reinforcement learning. In: 2019 Chinese Control and Decision Conference (CCDC), pp. 6174-6178. IEEE (2019)
Nath, A., Arun, A.R., Niyogi, R.: A distributed approach for road clearance with multi-robot in urban search and rescue environment. Int. J. Intell. Rob. Appl. 3(4), 392–406 (2019). https://doi.org/10.1007/s41315-019-00111-5
Meyer, J.A., Filliat, D.: Map-based navigation in mobile robots:: Ii. a review of map-learning and path-planning strategies. Cogn. Syst. Res. 4(4), 283-317 (2003)
Bhalla, S., Ganapathi Subramanian, S., Crowley, M.: Deep multi-agent reinforcement learning for autonomous driving. In: 33\(^{rd}\) Canadian Conference on Artificial Intelligence (CCAI-2020), pp. 67–78. Ontario, Ottawa (2020)
Liu, I.J., Jain, U., Yeh, R.A., Schwing, A.: Cooperative exploration for multi-agent deep reinforcement learning. In: 38\(^{th}\) International Conference on Machine Learning (ICML-2021). Virtual mode, pp. 6826–6836 (2021)
Quan, H., Li, Y., Zhang, Y.: A novel mobile robot navigation method based on deep reinforcement learning. Int. J. Adv. Rob. Syst. 17(3), 1-11. SAGE (2020)
Zhou, Z., Zhu, P., Zeng, Z., Xiao, J., Lu, H., Zhou, Z.: Robot navigation in a crowd by integrating deep reinforcement learning and online planning. Appl. Intell. 52, 1–17 (2022)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)
Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, pp. 2094–2100 (2016)
Fan, J., Wang, Z., Xie, Y., Yang, Z.: A theoretical analysis of deep Q-learning. In: Learning for Dynamics and Control, pp. 486-489. PMLR (2020 )
Achiam, J., Knight, E., Abbeel, P.: Towards characterizing divergence in deep q-learning. arXiv preprint arXiv:1903.08894 (2019)
Jin, X., Xu, C., Feng, J., Wei, Y., Xiong, J., Yan, S.: Deep learning with s-shaped rectified linear activation units. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1 (2016)
Neves, M., Vieira, M., Neto, P.: A study on a Q-learning algorithm application to a manufacturing assembly problem. J. Manuf. Syst. 59, 426–440 (2021)
Acknowledgment
The authors are grateful to the anonymous referees for their insightful criticism, which helped to make the paper better. The second author was in part supported by a research grant from Google.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nath, A., Niyogi, R., Singh, T., Kumar, V. (2023). Multi-agent Deep Q-Learning Based Navigation. In: Barolli, L. (eds) Advanced Information Networking and Applications. AINA 2023. Lecture Notes in Networks and Systems, vol 654. Springer, Cham. https://doi.org/10.1007/978-3-031-28451-9_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-28451-9_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28450-2
Online ISBN: 978-3-031-28451-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)