Skip to main content
SpringerLink
Log in

Cooperative multi-robot patrol with Bayesian learning

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Patrolling indoor infrastructures with a team of cooperative mobile robots is a challenging task, which requires effective multi-agent coordination. Deterministic patrol circuits for multiple mobile robots have become popular due to their exceeding performance. However their predefined nature does not allow the system to react to changes in the system’s conditions or adapt to unexpected situations such as robot failures, thus requiring recovery behaviors in such cases. In this article, a probabilistic multi-robot patrolling strategy is proposed. A team of concurrent learning agents adapt their moves to the state of the system at the time, using Bayesian decision rules and distributed intelligence. When patrolling a given site, each agent evaluates the context and adopts a reward-based learning technique that influences future moves. Extensive results obtained in simulation and real world experiments in a large indoor environment show the potential of the approach, presenting superior results to several state of the art strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://www.cybercars.org.

  2. http://www.ydreamsrobotics.com/projects.

  3. The degree (or valency) of a vertex of a graph \(deg(v_i)\), is the number of edges incident to the vertex.

  4. Entropy is a general measure for the uncertainty of a belief. When applied to a discrete random variable, it evaluates to its shortest description, being as high as the variable’s uncertainty (Rocha et al. 2005).

  5. The source code of the patrolling approaches tested are available at: https://github.com/davidbsp/patrolling_sim.

  6. Available at http://wiki.ros.org/wifi_comm.

  7. A video of an experiment with 6 robots running CBLS is available at: https://sites.google.com/site/davidbsp2014/videos/auro.

References

  • Agmon, N., Fok, C. L., Emaliach, Y., Stone, P., Julien, C., & Vishwanath, S. (2012). On coordination in practical multi-robot patrol. In Proceedings of the IEEE international conference on robotics and automation (ICRA 2012) (pp. 650–656), May 14–18. Saint Paul, MN.

  • Agmon, N., Urieli, D., & Stone, P. (2011). Multiagent patrol generalized to complex environmental conditions. In Proceedings of the 25th conference on artificial intelligence (AAAI 2011), San Francisco, CA, August 7–11.

  • Agmon, N., Kaminka, G. A., & Kraus, S. (2011). Multi-robot adversarial patrolling: Facing a full-knowledge opponent. Journal of Artificial Intelligence Research, 42, 887–916.

    MathSciNet  MATH  Google Scholar 

  • Aguirre, O., & Taboada, H. (2012). An evolutionary game theory approach for intelligent patrolling. In Procedia computer science (Vol. 12, Part II, pp. 140–145). Amsterdam: Elsevier.

  • Ahmadi, M., & Stone, P. (2006). A multi-robot system for continuous area sweeping tasks. In Proceedings of the international conference on robotics and automation (ICRA 2006). Orlando, FL, May 15–19.

  • Almeida, A., Ramalho, G., Santana, H., Tedesco, P., Menezes, T., Corruble, V., & Chaveleyre, Y. (2004). Recent advances on multi-agent patrolling. In Advances in artificial intelligence—SBIA 2004. Lecture Notes in Computer Science (Vol. 3171, pp 474-483). Berlin: Springer.

  • Applegate, D., Cook, W., & Rohe, A. (2003). Chained Lin-Kernighan for large traveling salesman problems. INFORMS Journal on Computing, 15, 82–92.

  • Baglietto, M., Cannata, G., Capezio, F., & Sgorbissa, A. (2009). Multi-robot uniform frequency coverage of significant locations in the environment. In Distributed autonomous robotic systems (Vol. 8, pp. 3–14). Berlin: Springer.

  • Basilico, N., Gatti, N., Rossi, T., Ceppi, S., & Amigoni, F. (2009). Extending algorithms for mobile robot patrolling in the presence of adversaries to more realistic settings. In Proceedings of the 2009 IEEE/WIC/ACM international conference on intelligent agent technology (IAT’09) (pp. 557–564). Milan.

  • Cannata, G., & Sgorbissa, A. (2011). A minimalist algorithm for multirobot continuous coverage. IEEE Transactions on Robotics, 27(2), 297–312.

    Article  Google Scholar 

  • Chevaleyre, Y. (2004). Theoretical analysis of the multi-agent patrolling problem. In Proceedings of the 2004 international conference on agent intelligent technologies (IAT’04), (pp. 30–308). Beijing, September 20–24.

  • Elmaliach, Y., Shiloni, A., & Kaminka, G. A. (2008). A realistic model of frequency-based multi-robot polyline patrolling. In Proceedings of the 7th international joint conference on autonomous agents and multiagent systems (AAMAS 2008), (Vol. 1, pp. 63–70).

  • Fabrizi, E., & Saffiotti, A. (2000). Extracting topology-based maps from gridmaps. In Proceedings of the 2000 IEEE international conference on robotics and automation (ICRA’2000) (pp. 2972–2978). San Francisco, CA, April 2000.

  • Fazli, P., Davoodi, A., & Mackworth, A. K. (2013). Multi-robot repeated area coverage. Autonomous Robots, 34(4), 251–276. Springer Science.

    Article  Google Scholar 

  • Fiedler, M. (1973). Algebraic connectivity of graphs. Czechoslovak Mathematical Journal, 23(2), 298–305.

    MathSciNet  MATH  Google Scholar 

  • Gabriely, Y., & Rimon, E. (2001). Spanning-tree based coverage of continuous areas by a mobile robot. Annals of mathematics and artificial intelligence (Vol. 31, pp. 77–98). Hingham, MA: Kluwer Academic Publishers.

  • Iocchi, L., Marchetti, L., & Nardi, D. (2011). Multi-robot patrolling with coordinated behaviours in realistic environments. In Proceedings of the international conference on intelligent robots and systems (IROS’2011) (pp. 2796–2801).

  • Ishiwaka, Y., Sato, T., & Kakazu, Y. (2003). An approach to the pursuit problem on a heterogeneous multiagent system using reinforcement learning. Robotics and Autonomous Systems (RAS), 43(4), 245–256. Elsevier.

    Article  Google Scholar 

  • Jansen, M., & Sturtevant, N. (2008). Direction maps for cooperative pathfinding. In Proceedings of the 4th artificial intelligence and interactive digital entertainment conference (AAAIDE’08). Stanford, CA, October 22–24.

  • Jansen, F., & Nielsen, T. (2007). Bayesian networks and decision graphs (2nd ed.). Berlin: Springer.

    Book  Google Scholar 

  • Keskin, B. B., Li, S., Steil, D., & Spiller, S. (2012). Analysis of an integrated maximum covering and patrol routing problem. Transportation Research Part E, 48, 215–232. Elsevier.

    Article  Google Scholar 

  • Lauri, F., & Koukam, A. (2014). Robustness analysis of multi-agent patrolling strategies using reinforcement learning. In Proceedings of the international conference on swarm intelligence based optimization (ICSIBO 2014), Mulhouse, May 13–14.

  • Marier, J., Besse, C., & Chaib-draa, B. (2010). Solving the continuous time multiagent patrol problem. In Proceedings of the 2010 IEEE international conference on robotics and automation (ICRA 2010), Anchorage, AK.

  • Marino, A., Antonelli, G., Aguiar, A. P., & Pascoal, A. (2012). A new approach to multi-robot harbour patrolling: Theory and experiments. In Proceedings of the 2012 IEEE/RSJ international conference on intelligent robots and systems (IROS 2012). Vilamoura, Portugal, October 7–12.

  • Marino, A., Parker, L. E., Antonelli, G., & Caccavale, F. (2013). A decentralized architecture for multi-robot systems based on the null-space-behavioral control with application to multi-robot border patrolling. Journal of Intelligent and Robotic Systems, 71, 423–444.

    Article  Google Scholar 

  • Murphy, R. (2004). Human–robot interaction in rescue robotics. IEEE Transactions on Systems, Man and Cybernetics Part C, 34(2), 138–153.

    Article  Google Scholar 

  • Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Journal of Autonomous Agents and Multi-Agent Systems, 11(3), 387–434.

    Article  Google Scholar 

  • Parker, L. E. (2008). Distributed intelligence: Overview of the field and its application in multi-robot systems. Journal of Physical Agents, 2(2), 5–14. Special issue on Multi-robot systems.

    Google Scholar 

  • Pasqualetti, F., Franchi, A., & Bullo, F. (2012). On cooperative patrolling: Optimal trajectories, complexity analysis, and approximation algorithms. IEEE Transactions on Robotics, 28(3), 592–606.

    Article  Google Scholar 

  • Pippin, C., Christensen, H., & Weiss, L. (2013). Performance based task assignment in multi-robot patrolling. In Proceedings of the 2013 ACM symposium on applied computing (SAC ’13) (pp. 70–76). Coimbra, March 18–22.

  • Portugal, D., & Rocha, R. P. (2013). Retrieving topological information for mobile robots provided with grid maps. In Agents and artificial intelligence. Communications in Computer and Information Science (CCIS) series, (Vol. 358, pp. 204-217). Berlin: Springer.

  • Portugal, D., Couceiro, M. S., & Rocha, R. P. (2013). Applying bayesian learning to multi-robot patrol. In Proceedings of the 2013 international symposium on safety, security and rescue robotics (SSRR 2013), Linköping, Oct 21–26.

  • Portugal, D., Couceiro, M., & Rocha, R. P. (2013). Concurrent Bayesian learners for multi-robot patrolling missions. In Proceedings of the 2013 IEEE international conference on robotics and automation (ICRA 2013), Workshop on Towards Fully Decentralized Multi-Robot Systems: Hardware, Software and Integration, Karlsruhe, May 6–10.

  • Portugal, D., Pippin, C., Rocha, R. P., & Christensen, H. (2014). Finding optimal routes for multi-robot patrolling in generic graphs. In Proceedings of the 2014 IEEE/RSJ international conference on intelligent robots and systems (IROS 2014). Chicago, Sep. 14–18.

  • Portugal, D., & Rocha, R. P. (2013). Multi-robot patrolling algorithms: examining performance and scalability. Advanced Robotics Journal, 27(5), 325–336.

    Article  Google Scholar 

  • Portugal, D., & Rocha, R. P. (2013). Distributed multi-robot patrol: A scalable and fault-tolerant framework. Robotics and Autonomous Systems (RAS), 61(12), 1572–1587.

    Article  Google Scholar 

  • Poulet, C., Corruble, V., & Seghrouchini, A. (2012). Working as a team: Using social criteria in the timed patrolling problem. In Proceedings of the international conference on tools with artificial intelligence (ICTAI’2012). Athens, GR, Nov. 7–9.

  • Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., & Ng, A. (2009). ROS: an open-source robot operating system. In Proceedings of the IEEE international conference on robotics and automation (ICRA’2009), Workshop On Open Source Software, Kobe, May 12–17.

  • Robotics, Activ Media. (2006). Pioneer 3 operations manual, version 3. : Mobile Robots Inc.

  • Rocha, R., Dias, J., & Carvalho, A. (2005). Cooperative multi-robot systems: A study of vision-based 3-D mapping using information theory. Robotics and Autonomous Systems (RAS), 53(3–4), 282–311. Elsevier.

    Article  Google Scholar 

  • Ruan, S., Meirina, C., Yu, F., Pattipati, K. R., & Popp, R. L. (2005). Patrolling in a stochastic environment. In Proceedings of the 10th international command and control research and technology symposium, McLean, VA, June 13–16.

  • Sak, T., Wainer, J., & Goldenstein, S. (2008). Probabilistic Multiagent Patrolling. In Advances in Artificial Intelligence, SBIA 2008. Lecture Notes in Computer Science, (Vol. 5249, pp. 124–133). Berlin: Springer.

  • Sampaio, P., Ramalho, G., & Tedesco, P. (2010). The gravitational strategy for the timed patrolling. In Proceedings of the IEEE international conference on tools with artificial intelligence (ICTAI’10) (pp. 113-120). Arras, France, Oct. 27–29.

  • Santana, H., Ramalho, G., Corruble, V., & Ratitch, B. (2004). Multi-agent patrolling with reinforcement learning. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (Vol. 3, pp. 1122–1129), New York, NY.

  • Sempé, F., & Drogoul, A. (2003). Adaptive patrol for a group of robots. In Proceedings of the international conference on intelligent robots and systems (IROS 2003). Las Vegas.

  • Smith, S., & Rus, D. (December 2010). Multi-robot monitoring in dynamic environments with guaranteed currency of observations. In Proceedings of the IEEE conference on decision and control (pp. 514–521). Atlanta, GA.

  • Stone, P., & Veloso, M. (2000). Multiagent systems: A survey from a machine learning perspective. Autonomous Robots, 8(3), 345–383.

    Article  Google Scholar 

  • Stranders, R., de Cote, E. M., Rogers, A., & Jennings, N. R. (2012). Near-optimal continuous patrolling with teams of mobile information gathering agents. In Artificial intelligence. Amsterdam: Elsevier.

  • Vaughan, R. (2008). Massively multi-robot simulation in stage. Journal of Swarm Intelligence, 2(2–4), 189–208.

    Article  Google Scholar 

  • Vocabulary.com Online Dictionary, September 2015. Available at: http://www.vocabulary.com/dictionary/patrol.

  • Yanovski, V., Wagner, I. A., & Bruckstein, A. M. (2003). A distributed ant algorithm for efficiently patrolling a network. Algorithmica, 37, 165–186.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported by a PhD scholarship (SFRH/BD/64426/2009), the CHOPIN research project (PTDC/EEA-CRO/119000/2010) and by the ISR—Institute of Systems and Robotics (project PEst-C/EEI/UI0048/2011), all of them funded by the Portuguese science agency “Fundação para a Ciência e a Tecnologia” (FCT). The authors gratefully acknowledge Prof. Hélder Araújo (ISR) for conceding three robot platforms used in the experiments; Marios Belk (University of Cyprus) for his help in the statistical analysis of the results; Luís Santos, Micael S. Couceiro and Gonçalo Cabrita (ISR) for their contribution and feedback; João M. Santos, Gonçalo Augusto, João Martins, Nuno L. Ferreira, José S. Pereira, André Araújo and João B. Campos for their assistance during the experiments with real robots.

Author information

Authors and Affiliations

  1. Institute of Systems and Robotics, University of Coimbra, 3030-290, Coimbra, Portugal

    David Portugal & Rui P. Rocha

Authors
  1. David Portugal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui P. Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Portugal.

Appendix: Extended details of the simulations with CBLS with different \(\omega \)

Appendix: Extended details of the simulations with CBLS with different \(\omega \)

See Tables 13, 14, 15, 16, 17, 18, 19, 20, 21.

Table 13 Experiments using CBLS in Environment A with \(\omega =1.0\) (Extended version of Table 1)
Full size table
Table 14 Experiments using CBLS in Environment A with \(\omega ={3}/{4}\) (Extended version of Table 1)
Full size table
Table 15 Experiments using CBLS in Environment A with \(\omega ={2}/{3}\) (Extended version of Table 1)
Full size table
Table 16 Experiments using CBLS in Environment B with \(\omega =1.0\) (Extended version of Table 2)
Full size table
Table 17 Experiments using CBLS in Environment B with \(\omega ={3}/{4}\) (Extended version of Table 2)
Full size table
Table 18 Experiments using CBLS in Environment B with \(\omega ={2}/{3}\) (Extended version of Table 2)
Full size table
Table 19 Experiments using CBLS in Environment C with \(\omega =1.0\) (Extended version of Table 3)
Full size table
Table 20 Experiments using CBLS in Environment C with \(\omega ={3}/{4}\) (Extended version of Table 3)
Full size table
Table 21 Experiments using CBLS in Environment C with \(\omega ={2}/{3}\) (Extended version of Table 3)
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Portugal, D., Rocha, R.P. Cooperative multi-robot patrol with Bayesian learning. Auton Robot 40, 929–953 (2016). https://doi.org/10.1007/s10514-015-9503-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-015-9503-7

Keywords

Search

Navigation