Skip to main content
SpringerLink
Log in

Automatic Extraction of Understandable Controllers from Video Observations of Swarm Behaviors

  • Conference paper
  • First Online:
Swarm Intelligence (ANTS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13491))

Included in the following conference series:

Abstract

Swarm behavior emerges from the local interaction of agents and their environment often encoded as simple rules. Extracting the rules by watching a video of the overall swarm behavior could help us study and control swarm behavior in nature, or artificial swarms that have been designed by external actors. It could also serve as a new source of inspiration for swarm robotics. Yet extracting such rules is challenging as there is often no visible link between the emergent properties of the swarm and their local interactions. To this end, we develop a method to automatically extract understandable swarm controllers from video demonstrations. The method uses evolutionary algorithms driven by a fitness function that compares eight high-level swarm metrics. The method is able to extract many controllers (behavior trees) in a simple collective movement task. We then provide a qualitative analysis of behaviors that resulted in different trees, but similar behaviors. This provides the first steps toward automatic extraction of swarm controllers based on observations.

Z. S. Abdallah and S. Hauert—Both authors have contributed equally to the work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amornbunchornvej, C., Berger-Wolf, T.: Framework for inferring following strategies from time series of movement data. ACM Trans. Knowl. Discov. Data 14(3), 35:1–35:22 (2020)

    Google Scholar 

  2. Bonnet, F., et al.: Robots mediating interactions between animals for interspecies collective behaviors. Sci. Robot. 4(28), eaau7897 (2019)

    Google Scholar 

  3. Brambilla, M., Ferrante, E., Birattari, M., Dorigo, M.: Swarm robotics: a review from the swarm engineering perspective. Swarm Intell. 7(1), 1–41 (2013)

    Article  Google Scholar 

  4. Carrillo-Zapata, D., et al.: Mutual shaping in swarm robotics: user studies in fire and rescue, storage organization, and bridge inspection. Front. Robot. AI 7, 53 (2020)

    Article  Google Scholar 

  5. Chatty, A., Gaussier, P., Kallel, I., Laroque, P., Alimi, A.M.: Learning by imitation for the improvement of the individual and the social behaviors of self-organized autonomous agents. In: Tan, Y., Shi, Y., Mo, H. (eds.) ICSI 2013. LNCS, vol. 7929, pp. 44–52. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38715-9_6

    Chapter  Google Scholar 

  6. Chung, N., Miasojedow, B., Michał, S., Gambin1, A.: Jaccard/Tanimoto similarity test and estimation methods for biological presence-absence data. BMC Bioinform. 20 (2019)

    Google Scholar 

  7. Erbas, M.D., Bull, L., Winfield, A.F.T.: On the evolution of behaviors through embodied imitation. Artif. Life 21(2), 141–165 (2015)

    Article  Google Scholar 

  8. Eriksson, A., Nilsson Jacobi, M., Nyström, J., Tunstrøm, K.: Determining interaction rules in animal swarms. Behav. Ecol. 21(5), 1106–1111 (2010)

    Article  Google Scholar 

  9. Ferguson, E.A., Matthiopoulos, J., Insall, R.H., Husmeier, D.: Inference of the drivers of collective movement in two cell types: dictyostelium and melanoma. J. R. Soc. Interface 13(123), 20160695 (2016)

    Google Scholar 

  10. Francesca, G., Birattari, M.: Automatic design of robot swarms: achievements and challenges. Front. Robot. AI 3, 29 (2016)

    Article  Google Scholar 

  11. Harriott, C., Seiffert, A., Hayes, S., Adams, J.: Biologically-inspired human-swarm interaction metrics. Proc. Hum. Factors Ergon. Soc. Ann. Meeting 58, 1471–1475 (2014)

    Article  Google Scholar 

  12. Herbert-Read, J.E., Perna, A., Mann, R.P., Schaerf, T.M., Sumpter, D.J.T., Ward, A.J.W.: Inferring the rules of interaction of shoaling fish. Proc. Natl. Acad. Sci. 108(46), 18726–18731 (2011)

    Article  Google Scholar 

  13. Hogg, E., Hauert, S., Harvey, D., Richards, A.: Evolving behaviour trees for supervisory control of robot swarms. Artif. Life Robot. 25(4), 569–577 (2020). https://doi.org/10.1007/s10015-020-00650-2

    Article  Google Scholar 

  14. Hu, T.K., Gama, F., Chen, T., Wang, Z., Ribeiro, A., Sadler, B.: VGAI: end-to-end learning of vision-based decentralized controllers for robot swarms, pp. 4900–4904 (2021)

    Google Scholar 

  15. Jones, S., Studley, M., Hauert, S., Winfield, A.: Evolving behaviour trees for swarm robotics. In: Groß, R., et al. (eds.) Distributed Autonomous Robotic Systems. SPAR, vol. 6, pp. 487–501. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73008-0_34

    Chapter  Google Scholar 

  16. Li, J., Tan, Y.: A two-stage imitation learning framework for the multi-target search problem in swarm robotics. Neurocomputing 334, 249–264 (2019)

    Article  Google Scholar 

  17. Li, W., Gauci, M., Groß, R.: Turing learning: a metric-free approach to inferring behavior and its application to swarms. Swarm Intell. 10(3), 211–243 (2016). https://doi.org/10.1007/s11721-016-0126-1

    Article  Google Scholar 

  18. Mann, R.P.: Bayesian inference for identifying interaction rules in moving animal groups. PLoS ONE 6(8), e22827 (2011)

    Google Scholar 

  19. Manning, M.D., Harriott, C.E., Hayes, S.T., Adams, J.A., Seiffert, A.E.: Heuristic evaluation of swarm metrics’ effectiveness. In: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended Abstracts, p. 17–18 (2015)

    Google Scholar 

  20. Maxeiner, H.: Imitation learning of fish and swarm behavior with Recurrent Neural Networks. Master’s thesis, Dahlem Center for Machine Learning and Robotics (2019)

    Google Scholar 

  21. Nedjah, N., Junior, L.S.: Review of methodologies and tasks in swarm robotics towards standardization. Swarm Evol. Comput. 50, 100565 (2019)

    Google Scholar 

  22. Peyer, K.E., Zhang, L., Nelson, B.J.: Bio-inspired magnetic swimming microrobots for biomedical applications (2012)

    Google Scholar 

  23. Prorok, A., Blumenkamp, J., Li, Q., Kortvelesy, R., Liu, Z., Stump, E.: The holy grail of multi-robot planning: learning to generate online-scalable solutions from offline-optimal experts. arXiv abs/2107.12254 (2021)

    Google Scholar 

  24. Ruangdech, S., Hauert, S., Homer, M.: Inferring swarm models using a single monitoring robot. In: Artificial Life Conference Proceedings, no. 31, pp. 278–279 (2019)

    Google Scholar 

  25. Şahin, E.: Swarm robotics: from sources of inspiration to domains of application. In: Şahin, E., Spears, W.M. (eds.) SR 2004. LNCS, vol. 3342, pp. 10–20. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-30552-1_2

    Chapter  Google Scholar 

  26. Schaerf, T.M., Herbert-Read, J.E., Ward, A.J.W.: A statistical method for identifying different rules of interaction between individuals in moving animal groups. J. R. Soc. Interface 18(176), rsif.2020.0925, 20200925 (2021)

    Google Scholar 

  27. Schilling, F., Lecoeur, J., Schiano, F., Floreano, D.: Learning vision-based flight in drone swarms by imitation. IEEE Robot. Autom. Lett. 4(4), 4523–4530 (2019)

    Article  Google Scholar 

  28. Yu, X., Wu, W., Feng, P., Tian, Y.: Swarm inverse reinforcement learning for biological systems. In: 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 274–279 (2021)

    Google Scholar 

  29. Zhou, S., Phielipp, M.J., Sefair, J.A., Walker, S.I., Amor, H.B.: Clone swarms: learning to predict and control multi-robot systems by imitation. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4092–4099 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bristol Robotics Laboratory, University of Bristol, Bristol, UK

    Khulud Alharthi & Sabine Hauert

  2. Department of Engineering Mathematics, University of Bristol, Bristol, UK

    Zahraa S. Abdallah & Sabine Hauert

  3. Department of Computer Science, College of Computers and Information Technology, Taif University, Taif, Saudi Arabia

    Khulud Alharthi

Authors
  1. Khulud Alharthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zahraa S. Abdallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sabine Hauert
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Khulud Alharthi .

Editor information

Editors and Affiliations

  1. Université Libre de Bruxelles, Brussels, Belgium

    Marco Dorigo

  2. University of Lübeck, Lübeck, Germany

    Heiko Hamann

  3. University of Manchester, Manchester, UK

    Manuel López-Ibáñez

  4. University of Málaga, Málaga, Spain

    José García-Nieto

  5. Stellenbosch University, Stellenbosch, South Africa

    Andries Engelbrecht

  6. Worcester Polytechnic Institute, Worcester, MA, USA

    Carlo Pinciroli

  7. Université Libre de Bruxelles, Brussels, Belgium

    Volker Strobel

  8. Université Libre de Bruxelles, Brussels, Belgium

    Christian Camacho-Villalón

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Alharthi, K., Abdallah, Z.S., Hauert, S. (2022). Automatic Extraction of Understandable Controllers from Video Observations of Swarm Behaviors. In: Dorigo, M., et al. Swarm Intelligence. ANTS 2022. Lecture Notes in Computer Science, vol 13491. Springer, Cham. https://doi.org/10.1007/978-3-031-20176-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20176-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20175-2

  • Online ISBN: 978-3-031-20176-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation