Abstract
Collaborative task execution in an unknown and dynamic environment is an important and challenging research area in autonomous robotic systems. It is essential to start the task execution in situations like search and rescue at the earliest. However, the time duration between team announcement and the arrival of team members at the location of a task delays the execution of the task. The distributed approaches for task execution assume that the path is known. However, in an environment, say, a building, the position of the doors may not be known, and some of the doors may get closed during task execution. Hence, an agent should first learn the map of the environment. The learning of the map of an unknown environment can be accelerated with multiple agents. This paper proposes a distributed multi-agent Q-learning-based approach for navigation in an unknown environment. The proposed approach is implemented using ARGoS, a realistic multi-robot simulator.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Nath, A., Arun, A.R, Niyogi, R.: An approach for task execution in dynamic multi-robot environment. In: 31\(^{st}\) Australasian Joint Conference on Artificial Intelligence (AI-2018). Wellington, New Zealand, pp. 71–76, (2018)
Nath, A., AR, A., Niyogi, R.: A distributed approach for autonomous cooperative transportation in a dynamic multi-robot environment. In: 35\(^{th}\) Annual ACM Symposium on Applied Computing (SAC-2020). Brno, Czech Republic, pp. 792-799 (2020)
Nath, A., Arun, A.R., Niyogi, R.: DMTF: a distributed algorithm for multi-team formation. In: 12\(^{th}\) International Conference on Agents and Artificial Intelligence (ICAART-2020). Valletta, Malta, pp. 152-160 (2020)
Nath, A., Niyogi, R.: Formal modeling, verification, and analysis of a distributed task execution algorithm. In: Barolli, L., Woungang, I., Enokido, T. (eds.) AINA 2021. LNNS, vol. 225, pp. 370–382. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-75100-5_32
Abdallah, S., Lesser, V.: Organization-based cooperative coalition formation. In: ACM International Conference on Intelligent Agent Technology (IAT-2004). Beijing, China, pp. 162-168 (2004)
Tošić, P.T., Agha, G.A.: Maximal clique based distributed coalition formation for task allocation in large-scale multi-agent systems. In: Ishida, T., Gasser, L., Nakashima, H. (eds.) MMAS 2004. LNCS (LNAI), vol. 3446, pp. 104–120. Springer, Heidelberg (2005). https://doi.org/10.1007/11512073_8
Meyer, J.A., Filliat, D.: Map-based navigation in mobile robots: a review of map-learning and path-planning strategies. Cogn. Syst. Res. 4(4), 283–317 (2003)
Bhalla, S., Ganapathi Subramanian, S., Crowley, M.: Deep multi agent reinforcement learning for autonomous driving. In: 33\(^{rd}\) Canadian Conference on Artificial Intelligence (CCAI-2020). Ottawa, Ontario, pp. 67-78 (2020)
Liu, I.J., Jain, U., Yeh, R.A., Schwing, A.: Cooperative exploration for multi-agent deep reinforcement learning. In: 38\(^{th}\) International Conference on Machine Learning (ICML-2021). Virtual mode, pp. 6826-6836 (2021)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Neves, M., Vieira, M., Neto, P.: A study on a Q-Learning algorithm application to a manufacturing assembly problem. J. Manuf. Syst. 59, 426–440 (2021)
ARGoS simulator www.argos-sim.info/
Pinciroli, C., et al.: ARGoS: a modular, parallel, multi-engine simulator for multi-robot systems. Swarm Intell. 6(4), 271-295 (2012)
Gazebo //gazebosim.org/
Acknowledgements
The second author was in part supported by a research grant from Google.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nath, A., Niyogi, R., Singh, T., Kumar, V. (2022). Multi-agent Q-learning Based Navigation in an Unknown Environment. In: Barolli, L., Hussain, F., Enokido, T. (eds) Advanced Information Networking and Applications. AINA 2022. Lecture Notes in Networks and Systems, vol 449. Springer, Cham. https://doi.org/10.1007/978-3-030-99584-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-030-99584-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-99583-6
Online ISBN: 978-3-030-99584-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)