skip to main content
10.1145/3638529.3653992acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article
Open access

Emergent Behavior in Evolutionary Swarms for Machine Olfaction

Published: 14 July 2024 Publication History

Abstract

Navigation via olfaction (scent) is one of the most primitive forms of exploration used by organisms. Machine olfaction is a growing field within sensing systems and AI and many of its use cases are motivated by swarm intelligence. With this work, we are specifically interested in demonstrating the collaborative ability that evolutionary optimization can enable in swarm navigation via machine olfaction. We designate each particle of the swarm as a reinforcement learning (RL) agent and show how agent rewards can be directly correlated to maximize the swarm's reward signal. In doing so, we show how different behaviors emerge within swarms depending on which RL algorithms are used. We are motivated by the application of machine olfaction and evaluate multiple swarm permutations against a suite of scent navigation tasks to demonstrate preferences exhibited by the swarm. Our results indicate that swarms can be designed to achieve desired behaviors as a function of the algorithm each agent demonstrates. This paper contributes to the field of cooperative co-evolutionary algorithms by proposing a method by which evolutionary techniques can significantly improve how swarms of simple agents collaborate to solve complex tasks faster than a single large agent can under identical conditions.

Supplemental Material

PDF File
Supplementary Material

References

[1]
Pieter Abbeel and Andrew Y. Ng. 2004. Apprenticeship Learning via Inverse Reinforcement Learning. In Proceedings of the Twenty-First International Conference on Machine Learning (Banff, Alberta, Canada) (ICML '04). Association for Computing Machinery, New York, NY, USA, 1.
[2]
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. CoRR abs/1606.01540 (2016). arXiv:1606.01540 http://arxiv.org/abs/1606.01540
[3]
John Crimaldi, Hong Lei, Andreas Schaefer, Michael Schmuker, Brian H. Smith, Aaron C. True, Justus V. Verhagen, and Jonathan D. Victor. 2022. Active sensing in a dynamic olfactory world. Journal of Computational Neuroscience 50, 1 (01 Feb 2022), 1--6.
[4]
Marco Dorigo and Mauro Birattari. 2010. Ant Colony Optimization. Springer US, Boston, MA, 36--39.
[5]
Andries P. Engelbrecht. 2007. Computational Intelligence: An Introduction (2nd ed.). Wiley Publishing.
[6]
Jonas Eschmann, Dario Albani, and Giuseppe Loianno. 2023. Learning to Fly in Seconds. arXiv:2311.13081 [cs.RO]
[7]
Nathan Fortier, John W. Sheppard, and Karthik Ganesan Pillai. 2012. DOSI: Training artificial neural networks using overlapping swarm intelligence with local credit assignment. In The 6th International Conference on Soft Computing and Intelligent Systems, and The 13th International Symposium on Advanced Intelligence Systems. 1420--1425.
[8]
Kordel K. France and John W. Sheppard. 2023. Factored Particle Swarm Optimization for Policy Co-training in Reinforcement Learning. In Proceedings of the Genetic and Evolutionary Computation Conference (Lisbon, Portugal) (GECCO '23). Association for Computing Machinery, New York, NY, USA, 30--38.
[9]
Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. 2020. D4RL: Datasets for Deep Data-Driven Reinforcement Learning. arXiv:2004.07219 [cs.LG]
[10]
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, Gabriel Dulac-Arnold, John Agapiou, Joel Z. Leibo, and Audrunas Gruslys. 2018. Deep Q-Learning from Demonstrations. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence (New Orleans, Louisiana, USA) (AAAI'18/IAAI'18/EAAI'18). AAAI Press.
[11]
Maximilian Hüttenrauch, Adrian Šošić, and Gerhard Neumann. 2019. Deep reinforcement learning for swarm systems. J. Mach. Learn. Res. 20, 1 (jan 2019), 1966--1996.
[12]
J. Kennedy and R. Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95 - International Conference on Neural Networks, Vol. 4. 1942--1948 vol.4.
[13]
Theta Diagnostics LLC. 2024. Alchemy Explainable Machine Learning Framework. https://thetadx.ai/alchemy
[14]
Vidya Muthukumar. 2020. Learning from an unknown environment. Ph. D. Dissertation. EECS Department, University of California, Berkeley.
[15]
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. arXiv:2203.02155 [cs.CL]
[16]
Adam P. Piotrowski, Jaroslaw J. Napiorkowski, and Agnieszka E. Piotrowska. 2020. Population size in Particle Swarm Optimization. Swarm and Evolutionary Computation 58 (2020), 100718.
[17]
Edoardo M. Ponti, Alessandro Sordoni, Yoshua Bengio, and Siva Reddy. 2022. Combining Modular Skills in Multitask Learning. arXiv:2202.13914 [cs.LG]
[18]
Gautam Reddy, Boris I. Shraiman, and Massimo Vergassola. 2022. Sector search strategies for odor trail tracking. Proceedings of the National Academy of Sciences 119, 1 (2022), e2107431118. arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.2107431118
[19]
John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 1889--1897. https://proceedings.mlr.press/v37/schulman15.html
[20]
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. arXiv:1707.06347 [cs.LG]
[21]
Satpreet H. Singh, Floris van Breugel, Rajesh P. N. Rao, and Bingni W. Brunton. 2023. Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nature Machine Intelligence 5, 1 (01 Jan 2023), 58--70.
[22]
Shane Strasser and John W. Sheppard. 2017. Convergence of Factored Evolutionary Algorithms. In Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms (Copenhagen, Denmark) (FOGA '17). Association for Computing Machinery, New York, NY, USA, 81--94.
[23]
Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction (second ed.). The MIT Press.
[24]
Christopher Watkins. 1989. Learning from Delayed Rewards. Ph. D. Dissertation. Department of Computer Science, Kings College, Cambridge University.
[25]
Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games. arXiv:2103.01955 [cs.LG]

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GECCO '24: Proceedings of the Genetic and Evolutionary Computation Conference
July 2024
1657 pages
ISBN:9798400704949
DOI:10.1145/3638529
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 July 2024

Check for updates

Author Tags

  1. evolutionary robotics
  2. reinforcement learning
  3. machine olfaction

Qualifiers

  • Research-article

Conference

GECCO '24
Sponsor:
GECCO '24: Genetic and Evolutionary Computation Conference
July 14 - 18, 2024
VIC, Melbourne, Australia

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 182
    Total Downloads
  • Downloads (Last 12 months)182
  • Downloads (Last 6 weeks)37
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media