Abstract
This paper studies the framework of a multi-agent system for the surveillance of zones. In such systems, patrolling agents are deployed to observe mobile targets. The observation problem consists in maximizing the number of viewed mobile targets by at least one agent of the mas. However, formal methods relying on potential field to solve this problem, such as the A-CMOMMT, that implement observation strategies, do not adapt them to the target’s behavior. In this article, we propose a trained method using a reinforcement learning (RL) approach to cope with naive and evasive targets in order to improve the observation of mobile targets, while protecting the patrolling agents from collisions. This paper compares our force field reinforcement learning (FFRL) method with some significant formal observation methods on various scenarios. It shows that FFRL has a better target’s observation than the studied methods for both naive and evasive targets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., Zaremba, W.: Openai gym (2016)
Canese, L., Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Giardino, D., Re, M., Spanò, S.: Multi-agent reinforcement learning: a review of challenges and applications. Appl. Sci. 11(11) (2021). https://doi.org/10.3390/app11114948, https://www.mdpi.com/2076-3417/11/11/4948
Chahal, J., Belbachir, A., Seghrouchni, A.E.F.: I-CMOMMT: a multiagent approach for patrolling and observation of mobile targets with a continuous environment representation. In: Proceedings of the International Conference on Software Engineering and Knowledge Engineering, SEKE (2021). https://doi.org/10.18293/SEKE2021-135
Fernández, F., Borrajo, D., Parker, L.: A reinforcement learning algorithm in cooperative multi-robot domains. J. Intell. Rob. Syst. 43, 161–174 (2005)
Kolling, A., Carpin, S.: Cooperative observation of multiple moving targets: an algorithm and its formalization. Int. J. Robot. Res. 26(9), 935–953 (2007). https://doi.org/10.1177/0278364907080424
Li, X., Chen, J., Deng, F., Li, H.: Profit-driven adaptive moving targets search with uav swarms. Sensors 19(7) (2019). https://doi.org/10.3390/s19071545, https://www.mdpi.com/1424-8220/19/7/1545
Liang, E., Liaw, R., Moritz, P., Nishihara, R., Fox, R., Goldberg, K., Gonzalez, J.E., Jordan, M.I., Stoica, I.: Rllib: abstractions for distributed reinforcement learning (2018)
Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. Neural Inf. Process. Syst. (NIPS) (2017)
Machado, A., Alessandro, A.: Multi-agent movement coordination in patrolling. In: First Workshop on Agents in Computer Games, at The 3rd International Conference on Computers and Games (CG’02) (2002)
Parker, L.E.: Distributed algorithms for multi-robot observation of multiple moving targets. Auton. Robot. 12, 231–255 (2002). https://doi.org/10.1023/A:1015256330750
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms (2017)
Terry, J.K., Black, B., Grammel, N., Jayakumar, M., Hari, A., Sulivan, R., Santos, L., Perez, R., Horsch, C., Dieffendahl, C., Williams, N.L., Lokesh, Y., Sullivan, R., Ravi, P.: Pettingzoo: gym for multi-agent reinforcement learning (2020). arXiv:2009.14471
Terry, J.K., Grammel, N., Hari, A., Santos, L., Black, B.: Revisiting parameter sharing in multi-agent deep reinforcement learning (2021)
Touzet, C.F.: Distributed lazy q-learning for cooperative mobile robots. International Int. J. Adv. Robot. Syst. 1(1), 1 (2004). https://doi.org/10.5772/5614
Yan, P., Jia, T., Bai, C.: Searching and tracking an unknown number of targets: a learning-based method enhanced with maps merging. Sensors 21(4) (2021). https://doi.org/10.3390/s21041076, https://www.mdpi.com/1424-8220/21/4/1076
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chahal, J., Seghrouchni, A.E.F., Belbachir, A. (2022). A Force Field Reinforcement Learning Approach for the Observation Problem. In: Camacho, D., Rosaci, D., Sarné, G.M.L., Versaci, M. (eds) Intelligent Distributed Computing XIV. IDC 2021. Studies in Computational Intelligence, vol 1026. Springer, Cham. https://doi.org/10.1007/978-3-030-96627-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-96627-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-96626-3
Online ISBN: 978-3-030-96627-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)