GymSlug: Deep Reinforcement Learning Toward Bio-inspired Control Based on Aplysia californica Feeding

Abstract

Developing robots with animal-like flexibility, adaptability, and robustness remains challenging. However, the neuromuscular system of animals can provide bioinspiration for robotic controller design. In this work, we have developed a bio-inspired simulation environment, GymSlug, for reinforcement learning of motor control sequences based on our prior models of feeding behavior in the marine mollusk Aplysia californica. Using a range of model-free deep reinforcement learning algorithms, we train agents capable of producing motor neural control sequences, muscle activities, and feeding apparatus behavior that are qualitatively similar to behaviors observed in the animal during swallowing of unbreakable seaweed. The robustness of the trained agent is demonstrated by its ability to adapt to a previously unseen environment with breakable seaweed of varying strength. In addition, the environment can be easily reconfigured to train agents for additional tasks, including effective egestion of inedible objects. Our extensible simulation environment provides a platform for developing novel controllers to test biological hypotheses, learn control policies for neurorobotic models, and develop new approaches for soft robotic grasping control inspired by Aplysia.

This work was supported by NSF DBI 2015317 as part of the NSF/CIHR/DFG/FRQ/UKRI-MRC Next Generation Networks for Neuroscience Program.