Abstract
In recent years, soft robots with “softness” have been attracting much attention. Since soft robots have “softness”, they are expected to be able to perform delicate tasks that only humans can do. On the other hand, it is challenging to control. Therefore, in this research, we focused on reservoir computing with a biologically inspired learning algorithm. Reward-modulated Hebbian learning, one of the reservoir computing frameworks, is based on Hebbian learning rules and rewards and allows us to train the network without explicit teacher signals. The rewards are provided depending on the predicted and actual state of the environment influenced by the exploratory noise. We demonstrate that our model successfully controls the robot arm so that the tip position of the arm draws a given target trajectory.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Sheoherd, R.F., et al.: Multigait soft robot. PNAS of USA 108(51), 20400–20403 (2011)
Kim, S., Laschi, C., Trimmer, B.: Soft robotics: a bioinspired evolution in robotics. Trends Biotechnol. 31, 287–294 (2013)
Trivedi, D., Rahn, C.D., Kier, W.M., Walker, I.D.: Soft robotics: biological inspiration, state of the art, and future research. Appl. Bionics Biomech. 5(3), 99–117 (2008)
Li, T., et al.: From the octopus to soft robots control: an octopus inspired behavior control architecture for soft robots. Vie et Milieu, 61, 211–217 (2012)
Nie, X., Song, B., Ge, Y., Chen, W.W., Weerasooriya, T.: Dynamic tensile of soft materials. Exp. Mach. 49(4), 451–458 (2009)
Jaeger, H.: Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the “echo state networks” approach. GMD report, German Nation. Res. Center Inf. Technol. 159, 1–46 (2002)
Hoezer, G.M., Legenstein, R., Maass, W.: Emergence of complex computational structures from chaotic neural networks through reward-modulated hebbian learning. Cerebr. Cortex 24(3), 677–690 (2012)
Izawa, J., Kondo, T., Ito, K.: Biological robot arm motion through reinforcement learning. Proceed. 2002 IEEE Int. Conf. Robot. Autom., 4, 3398–3403 (2002)
Kambara, H., Kim, K., Shin, D., Sato, M., Koike, Y.: Learning and generation of goal-directed arm reaching from scratch. Neural Netw. 22(4), 348–361 (2009)
Todorov, E., Erez, T., Tassa, Y.: MuJoCo: a physics engine for model-based control. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5026–5033 (2012)
Katori, Y.: Network model for dynamics of perception with reservoir computing and predictive coding. In: Delgado-García, J.M., Pan, X., Sánchez-Campusano, R., Wang, R. (eds.) Advances in Cognitive Neurodynamics (VI). ACN, pp. 89–95. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-8854-4_11
Sussillo, D., Abbott, L.F.: Generating coherent patterns of activity from chaotic neural networks. Neuron, 63(4), 554–557 (2009)
Katayama, M., Kawato, M.: Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models. Biol. Cybern. 69, 353–362 (1993)
Acknowledgements
This paper is based on results obtained from a project, JPNP16007, commissioned by the New Energy and Industrial Technology Development Organization (NEDO) and is supported by JSPS KAKENHI Grant Number 21H05163, 20H04258, 20H00596, and JST CREST(JPMJCR18K2).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Minato, K., Katori, Y. (2021). Robot Arm Control Using Reward-Modulated Hebbian Learning. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Communications in Computer and Information Science, vol 1517. Springer, Cham. https://doi.org/10.1007/978-3-030-92310-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-92310-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92309-9
Online ISBN: 978-3-030-92310-5
eBook Packages: Computer ScienceComputer Science (R0)