Abstract
Artificial muscles are recently developed actuators extremely promising for compliant robotic systems. Their accurate closed-loop control is challenging due to their highly nonlinear behavior.
In this work, we model an artificial muscle pair adopting a non-pulley configuration mimicking more realistically the behavior of biological muscles. Inspired by how the brain regulates dopamine-based learning from interaction with the environment, it is possible to design efficient reinforcement learning control algorithms. Therefore, we propose a reinforcement learning-based controller bioinspired by the parallels between the behavior of temporal difference errors and the activity of dopaminergic neurons. Simulated experiments conducted in a virtual scenario show that the control action can accurately tackle the nonlinear control problem.
The proposed solution could be extended to the dynamic control of more realistic and complex anthropomorphic limb systems due to its inherent adaptability and control effectiveness regardless of the complexity of the environment.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Tondu, B.: Modelling of the McKibben artificial muscle: a review. J. Intell. Mater. Syst. Struct. 23 225–253 (2012). https://doi.org/10.1177/1045389X11435435
Caldwell, D., Medrano-Cerda, G., Goodwin, M.: Control of pneumatic muscle actuators. Control Syst. 15, 40–48 (1995). https://doi.org/10.1109/37.341863
Chou, C., Hannaford, B.: Measurement and modeling of McKibben pneumatic artificial muscles. Robot. Autom. IEEE Trans. 12, 90–102 (1996). https://doi.org/10.1109/70.481753
Garbuliński, J., Balasankula, S., Wereley, N.: Characterization and analysis of extensile fluidic artificial muscles. Actuators 10, 26 (2021). https://doi.org/10.3390/act10020026
Schulte, H.: The characteristics of the McKibben artificial muscle (1961)
Obiajulu, S., Roche, E., Pigula, F. Walsh, C.: Soft pneumatic artificial muscles with low threshold pressures for a cardiac compression device. In: Proceedings Of The ASME Design Engineering Technical Conference, vol. 6 (2013). https://doi.org/10.1115/DETC2013-13004
Tondu, B.: Single linear integral action control for closed-loop positioning of a biomimetic actuator with artificial muscles. In: 2015 European Control Conference (ECC) (2015). https://doi.org/10.1109/ECC.2015.7331088
Centurelli, A., Rizzo, A., Tolu, S., Falotico, E.: Open-loop model-free dynamic control of a soft manipulator for tracking tasks. In: 2021 20th International Conference On Advanced Robotics (ICAR), pp. 128–133 (2021). https://doi.org/10.1109/ECC.2015.7331088
Braikia, K., Chettouh, M., Tondu, B., Acco, P., Hamerlain, M.: Improved control strategy of 2-sliding controls applied to a flexible robot arm. Adv. Robot. 25, 1515–1538 (2011). https://doi.org/10.1163/016918611X579510
Chiang, C., Chen, Y.: Neural network fuzzy sliding mode control of pneumatic muscle actuators. Eng. Appl. Artif. Intell. 65, 68–86 (2017). https://doi.org/10.1016/j.engappai.2017.06.021
Tian, S., Ding, G., Yan, D., Lin, L., Shi, M.: Nonlinear controlling of artificial muscle system with neural networks. In: 2004 IEEE International Conference On Robotics and Biomimetics, pp. 56–59 (2004). https://doi.org/10.1109/ROBIO.2004.1521751
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. (A Bradford Book, 2018)
Chadderdon, G.L., Neymotin, S.A., Kerr, C.C., Lytton, W.W.: Reinforcement learning of targeted movement in a spiking neuronal model of motor cortex. PLoS ONE 7(10), e47251 (2012). https://doi.org/10.1371/journal.pone.0047251
Centurelli, A., Arleo, L., Rizzo, A., Tolu, S., Laschi, C., Falotico, E.: Closed-loop dynamic control of a soft manipulator using deep reinforcement learning. IEEE Robot. Autom. Lett. 7, 4741–4748 (2022). https://doi.org/10.1109/LRA.2022.3146903
Schultz, W., Dayan, P., Montague, P.: A neural substrate of prediction and reward. Science. 275, 1593–1599 (1997). https://doi.org/10.1126/science.275.5306.1593
Hannaford, B., Winters, J., Chou, C., Marbot, P.: The anthroform biorobotic arm: a system for the study of spinal circuits. Ann. Biomed. Eng. 23, 399–408 (1995). https://doi.org/10.1007/BF02584440
Matsutani, Y., Tahara, K., Kino, H., Ochi, H.: Complementary compound set-point control by combining muscular internal force feedforward control and sensory feedback control including a time delay. Adv. Robot. 32, 411–425 (2018). https://doi.org/10.1080/01691864.2018.1453375
Hogan, N.: Adaptive control of mechanical impedance by coactivation of antagonist muscles. IEEE Trans. Autom. Control 29, 681–690 (1984). https://doi.org/10.1109/TAC.1984.1103644
Romero, F., Alonso, F.: A comparison among different hill-type contraction dynamics formulations for muscle force estimation. Mech. Sci. 7, 19–29 (2016). https://doi.org/10.5194/ms-7-19-2016
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Foggetti, M., Tolu, S. (2023). Bioinspired Reinforcement Learning Control for a Biomimetic Artificial Muscle Pair. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2023. Lecture Notes in Computer Science, vol 14134. Springer, Cham. https://doi.org/10.1007/978-3-031-43085-5_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-43085-5_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43084-8
Online ISBN: 978-3-031-43085-5
eBook Packages: Computer ScienceComputer Science (R0)