Skip to main content
Springer Nature Link
Log in

A machine education approach to swarm decision-making in best-of-n problems

  • Published:
Swarm Intelligence Aims and scope Submit manuscript

Abstract

In swarm decision making, hand-crafting agents’ rules that use local information to achieve desirable swarm-level behaviours is a non-trivial design problem. Instead of relying entirely on swarm experts for designing these local rules, machine learning (ML) algorithms can be utilised for learning some of the local rules by mapping an agent’s perception to an appropriate action. To facilitate this process, we propose the use of Machine Education (ME) as a systematic approach for designing a curriculum for teaching the agents the required skills to autonomously select appropriate behaviours. We study the use of ME in the context of decision-making in best-of-n problems. The proposed approach draws on swarm robotics expertise for identifying agents’ capabilities and limitations, the skills required for generating the desirable behaviours, and the corresponding performance measures. In addition, ME utilises ML expertise for the selection and development of the ML algorithms suitable for each skill. The results of the experimental evaluations demonstrate the superior efficacy of the ME-based approach over the state-of-the-art approaches with respect to speed and accuracy. In addition, our approach shows considerable robustness to changes in swarm size and to changes in sensing and communication noise. Our findings promote the use of ME for teaching swarm members the required skills for achieving complex swarm tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Abbass, H. A. (2015). Big-data-to-decisions red teaming systems. Computational red teaming (pp. 105–158). Cham: Springer.

    Google Scholar 

  • Abbass, H. A., Elsawah, S., Petraki, E., & Hunjet, R. (2019). Machine education: Designing semantically ordered and ontologically guided modular neural networks. In Symposium Series on Computational Intelligence, pages 948–955. IEEE. https://doi.org/10.1109/SSCI44817.2019.9003083.

  • Andreas, J., Klein, D., & Levine, S. (2017). Modular multitask reinforcement learning with policy sketches. In International Conference on Machine Learning, pages 166–175. Proceedings of Machine Learning Research. https://doi.org/10.5555/3305381.3305399.

  • Bartashevich, P., & Mostaghim, S. (2019a). Benchmarking collective perception: New task difficulty metrics for collective decision-making. In P. Moura Oliveira, P. Novais, & L. P. Reis (Eds.), Progress in artificial intelligence (pp. 699–711). Cham: Springer.

    Chapter  Google Scholar 

  • Bartashevich, P., & Mostaghim, S. (2019b). Ising model as a switch voting mechanism in collective perception. In P. Moura Oliveira, P. Novais, & L. P. Reis (Eds.), Progress in artificial intelligence (pp. 617–629). Cham: Springer.

    Chapter  Google Scholar 

  • Barto, A. G. & Dietterich, T. G. (2004). Reinforcement learning and its relationship to supervised learning. In Handbook of Learning and Approximate Dynamic Programming, pages 45–63. Wiley-IEEE Press. https://doi.org/10.1109/9780470544785.ch2.

  • Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. In International Conference on Machine Learning, pages 41–48. Association for Computing Machinery. https://doi.org/10.1145/1553374.1553380.

  • Bounceur, A., Bezoui, M., Noreen, U., Euler, R., Lalem, F., Hammoudeh, M., & Jabbar, S. (2017). LOGO: A new distributed leader election algorithm in wsns with low energy consumption. International Conference on Future Internet Technologies and Trends (pp. 1–16). Springer. https://doi.org/10.1007/978-3-319-73712-6_1

  • Buşoniu, L., Babuška, R., & De Schutter, B. (2010). Multi-agent reinforcement learning: An overview. In D. Srinivasan & L. Jain (Eds.), Innovations in multi-agent systems and applications-1 (pp. 183–221). Berlin, Heidelberg: Springer.

    Chapter  Google Scholar 

  • Clayton, N. R. & Abbass, H. (2019). Machine teaching in hierarchical genetic reinforcement learning: Curriculum design of reward functions for swarm shepherding. In Congress on Evolutionary Computation, pages 1259–1266. IEEE. https://doi.org/10.1109/CEC.2019.8790157.

  • Couture-Beil, A., Vaughan, R. T., & Mori, G. (2010). Selecting and commanding individual robots in a multi-robot system. In Canadian Conference on Computer and Robot Vision, pages 159–166. IEEE. https://doi.org/10.1109/CRV.2010.28.

  • Dick, W., Carey, L., & Carey, J. O. (2009). The systematic design of instruction. Pearson.

  • Ebert, J. T., Gauci, M., Mallmann-Trenn, F., & Nagpal, R. (2020). Bayes bots: collective bayesian decision-making in decentralized robot swarms. In International Conference on Robotics and Automation, pages 7186–7192. IEEE. https://doi.org/10.1109/ICRA40945.2020.9196584.

  • Ebert, J. T., Gauci, M., & Nagpal, R. (2018). Multi-feature collective decision making in robot swarms. In Proceedings of the International Conference on Autonomous Agents and MultiAgent Systems, pages 1711–1719. International Foundation for Autonomous Agents and Multiagent Systems. https://doi.org/10.5555/3237383.3237953.

  • Gee, A. & Abbass, H. (2019). Transparent machine education of neural networks for swarm shepherding using curriculum design. In International Joint Conference on Neural Networks, pages 1–8. IEEE. https://doi.org/10.1109/IJCNN.2019.8852209.

  • Giusti, A., Nagi, J., Gambardella, L. M., & Di Caro, G. A. (2012). Distributed consensus for interaction between humans and mobile robot swarms. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, pages 1503–1504. International Foundation for Autonomous Agents and Multiagent Systems. https://doi.org/10.5555/2343896.2344082.

  • Graves, K. (2008). The language curriculum: A social contextual perspective. In Language Teaching, volume 41, pages 147–181. Cambridge University Press. https://doi.org/10.1017/S0261444807004867.

  • Hamann, H. (2018). Opinion dynamics with mobile agents: Contrarian effects by spatial correlations. Frontiers in Robotics and AI. https://doi.org/10.3389/frobt.2018.00063.

    Article  Google Scholar 

  • Hussein, A., & Abbass, H. A. (2021). Stable belief estimation in shepherd-assisted swarm collective decision making. Shepherding UxVs for human-swarm teaming: An artificial intelligence approach to unmanned X Vehicles (pp. 165–185). New York: Springer.

    Chapter  Google Scholar 

  • Hussein, A., Elsawah, S., & Abbass, H. (2020). Swarm collective wisdom: A fuzzy-based consensus approach for evaluating agents confidence in global states. In International Conference on Fuzzy Systems. IEEE. https://doi.org/10.1109/FUZZ48607.2020.9177680.

  • Khan, F., Mutlu, B., & Zhu, J. (2011). How do humans teach: On curriculum learning and teaching dimension. In International Conference on Neural Information Processing Systems, 24, 1449–1457.

  • Leu, G., Lakshika, E., Tang, J., Merrick, K., & Barlow, M. (2017). Machine education-the way forward for achieving trust-enabled machine agents. In NIPS’17 Workshop: Teaching Machines, Robots, and Humans.

  • Meyer, K. A. (2014). Student Engagement Online: What Works and Why: ASHE Higher Education Report, Volume 40, Number 6. Wiley.

  • Peng, B., MacGlashan, J., Loftin, R., Littman, M. L., Roberts, D. L., & Taylor, M. E. (2018). Curriculum design for machine learners in sequential decision tasks. In IEEE Transactions on Emerging Topics in Computational Intelligence, 2, 268–277.

    Google Scholar 

  • Prasetyo, J., De Masi, G., Ranjan, P., & Ferrante, E. (2018). The best-of-n problem with dynamic site qualities: Achieving adaptability with stubborn individuals. In International Conference on Swarm Intelligence, pages 239–251. Springer. 10.1007/978-3-030-00533-7\_19.

  • Richards, J. C. (2013). Curriculum approaches in language teaching: Forward, central, and backward design. In RELC booktitle, volume 44, pages 5–33. SAGE Publications Sage UK: London, England. https://doi.org/10.1177/0033688212473293.

  • Richards, J. C. (2017). Curriculum Development in Language Teaching. Cambridge Professional Learning: Cambridge University Press.

  • Rummery, G. A., & Niranjan, M. (1994). On-line Q-learning using connectionist systems (Vol. 37). UK: University of Cambridge.

    Google Scholar 

  • Şahin, E. (2004). Swarm robotics: From sources of inspiration to domains of application. In International Workshop on Swarm Robotics, pages 10–20. Springer. https://doi.org/10.1007/978-3-540-30552-1_2.

  • Scheidler, A., Brutschy, A., Ferrante, E., & Dorigo, M. (2015). The \(k\)-unanimity rule for self-organized decision-making in swarms of robots. In IEEE Transactions on Cybernetics, 46, 1175–1188. https://doi.org/10.1109/TCYB.2015.2429118.

    Article  Google Scholar 

  • Strobel, V., Castelló Ferrer, E., & Dorigo, M. (2018). Managing byzantine robots via blockchain technology in a swarm robotics collective decision making scenario. In Proceedings of the International Conference on Autonomous Agents and MultiAgent Systems, 541–549.

  • Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. Cambridge: MIT press.

    MATH  Google Scholar 

  • Szepesvári, C. (2010). Algorithms for reinforcement learning. In Synthesis Lectures on Artificial Intelligence and Machine Learning, volume 4, pages 1–103. Morgan & Claypool Publishers. https://doi.org/10.2200/S00268ED1V01Y201005AIM009.

  • Tang, J., Leu, G., & Abbass, H. A. (2018). Networking the boids is more robust against adversarial learning. In IEEE Transactions on Network Science and Engineering, 5, 141–155. https://doi.org/10.1109/TNSE.2017.2745108

    Article  MathSciNet  Google Scholar 

  • Tarapore, D., Christensen, A. L., & Timmis, J. (2017). Generic, scalable and decentralized fault detection for robot swarms. PLOS ONE. https://doi.org/10.1371/booktitle.pone.0182058.

    Article  Google Scholar 

  • Valentini, G., Brambilla, D., Hamann, H., & Dorigo, M. (2016). Collective perception of environmental features in a robot swarm. In International Conference on Swarm Intelligence, pages 65–76. Springer. https://doi.org/10.1007/978-3-319-44427-7_6.

  • Valentini, G., Ferrante, E., & Dorigo, M. (2017). The best-of-n problem in robot swarms: Formalization, state of the art, and novel perspectives. Frontiers in Robotics and AI. https://doi.org/10.3389/frobt.2017.00009.

    Article  Google Scholar 

  • Valentini, G., Hamann, H., & Dorigo, M. (2015). Efficient decision-making in a self-organizing robot swarm: On the speed versus accuracy trade-off. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, 1305–1314.

  • Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine learning, 8, 279–292. https://doi.org/10.1007/BF00992698.

    Article  MATH  Google Scholar 

  • Zhu, X. (2015). Machine teaching: An inverse problem to machine learning and an approach toward optimal education. In Proceedings of the AAAI Conference on Artificial Intelligence, page 4083–4087. AAAI Press.

  • Zhu, X., Singla, A., Zilles, S., & Rafferty, A. N. (2018). An overview of machine teaching. In arXiv preprintarXiv:1801.05927.

Download references

Author information

Authors and Affiliations

  1. School of Engineering and IT, UNSW-Canberra, University of New South Wales, Canberra, Australia

    Aya Hussein, Sondoss Elsawah & Hussein A. Abbass

  2. Faculty of Education, University of Canberra, Canberra, Australia

    Eleni Petraki

Authors
  1. Aya Hussein
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Sondoss Elsawah
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Eleni Petraki
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Hussein A. Abbass
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Aya Hussein.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was funded by the Australian Research Council Discovery Grant number DP200101211.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 10871 KB)

Supplementary material 2 (pdf 350 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hussein, A., Elsawah, S., Petraki, E. et al. A machine education approach to swarm decision-making in best-of-n problems. Swarm Intell 16, 59–90 (2022). https://doi.org/10.1007/s11721-021-00206-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11721-021-00206-5

Keywords