Skip to main content

Advertisement

Springer Nature Link
Log in

A Neural Architecture for Performing Actual and Mentally Simulated Movements During Self-Intended and Observed Bimanual Arm Reaching Movements

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Dexterous reaching, pointing, and grasping play a critical role in human interactions with tools and the environment, and it also allows individuals to interact with one another effectively in social settings. Developing robotic systems with mental simulation and imitation learning abilities for such tasks seems a promising way to enhance robot performance as well as to enable interactions with humans in a social context. In spite of important advances in artificial intelligence and smart robotics, current robotic systems lack the flexibility and adaptability that humans so naturally exhibit. Here we present and study a neural architecture that captures some critical visuo-spatial transformations that are required for the cognitive processes of mental simulation and imitation. The results show that our neural model can perform accurate, flexible and robust 3D unimanual and bimanual actual/imagined reaching movements while avoiding extreme joint positions and generating kinematics similar to those observed with humans. In addition, using visuo-spatial transformations, the neural model was able to observe/imitate bimanual arm reaching movements independently of the viewpoint, distance and anthropometry between the demonstrator and imitator. Our model is a first step towards developing a more advanced neurally-inspired hierarchical architecture that integrates mental simulation and sensorimotor processing as it learns to imitate dexterous bimanual arm movements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Shadmehr R (2005) The computational neurobiology of reaching and pointing: a foundation for motor learning. MIT, Cambridge

    Google Scholar 

  2. Kamper DG, Cruz EG, Siegel MP (2003) Stereotypical fingertip trajectories during grasp. J Neurophysiol 90(6):3702–3710. doi:10.1152/jn.00546.2003

    Article  Google Scholar 

  3. Billard AG, Calinon S, Guenter F (2006) Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robot Auton Syst 54(5):370–384. doi:10.1016/j.robot.2006.01.007

    Article  Google Scholar 

  4. Cuijpers RH, van Schie HT, Koppen M, Erlhagen W, Bekkering H (2006) Goals and means in action observation: a computational approach. Neural Netw 19(3):311–322. doi:10.1016/j.neunet.2006.02.004

    Article  MATH  Google Scholar 

  5. Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Siciliano B, Khatib O (eds) Springer handbook of robotics. Springer, Berlin, pp 1371–1394. Retrieved from http://www.springerlink.com/index/10.1007/978-3-540-30301-5_60

  6. Nicolescu M, Mataric M (2009) Task learning through imitation and human–robot interaction. In: Dautenhahn K, Nehaniv C (eds) Models and mechanisms of imitation and social learning in robots, humans and animals. Cambridge University Press, Cambridge, pp 407–424

  7. Argall BD, Chernova S, Veloso M, Browning B (2009) A survey of robot learning from demonstration. Robot Auton Syst 57(5):469–483. doi:10.1016/j.robot.2008.10.024

    Article  Google Scholar 

  8. Umar Suleman M, Awais MM (2011) Learning from demonstration in robots: experimental comparison of neural architectures. Robot Comput Integr Manuf 27(4):794–801. doi:10.1016/j.rcim.2010.10.010

    Article  Google Scholar 

  9. Tani J (2007) On the interactions between top-down anticipation and bottom-up regression. Front Neurorobotics 1. doi:10.3389/neuro.12.002.2007

  10. Mohan V, Morasso P, Metta G, Kasderidis S (2011) Actions and Imagined Actions in Cognitive Robots. In: Cutsuridis V, Hussain A, Taylor JG (eds) Perception–action cycle. Springer, New York, pp 539–572. Retrieved from http://www.springerlink.com/index/10.1007/978-1-4419-1452-1_17

  11. Toussaint M (2006) A sensorimotor map: modulating lateral interactions for anticipation and planning. Neural Comput 18(5):1132–1155. doi:10.1162/089976606776240995

    Article  MATH  Google Scholar 

  12. Shanahan M (2006) A cognitive architecture that combines internal simulation with a global workspace. Conscious Cogn 15(2):433–449. doi:10.1016/j.concog.2005.11.005

    Article  Google Scholar 

  13. Morasso P (1981) Spatial control of arm movements. Exp Brain Res 42(2). doi:10.1007/BF00236911

  14. Abend W, Bizzi E, Morasso P (1982) Human arm trajectory formation. Brain 105(Pt 2):331–348

    Article  Google Scholar 

  15. Flash T, Hogan N (1985) The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci 5(7):1688–1703

    Google Scholar 

  16. Nakano E, Imamizu H, Osu R, Uno Y, Gomi H, Yoshioka T, Kawato M (1999) Quantitative examinations of internal representations for arm trajectory planning: minimum commanded torque change model. J Neurophysiol 81(5):2140–2155

    Google Scholar 

  17. Gordon J, Ghilardi MF, Ghez C (1994) Accuracy of planar reaching movements. I. Independence of direction and extent variability. Exp Brain Res 99(1):97–111

    Article  Google Scholar 

  18. Gentili R, Han CE, Schweighofer N, Papaxanthis C (2010) Motor learning without doing: trial-by-trial improvement in motor performance during mental training. J Neurophysiol 104(2):774–783. doi:10.1152/jn.00257.2010

    Article  Google Scholar 

  19. Wolpert DM, Miall RC (1996) Forward models for physiological motor control. Neural Netw 9(8):1265–1279

    Article  MATH  Google Scholar 

  20. Wolpert DM, Doya K, Kawato M (2003) A unifying computational framework for motor control and social interaction. Philos Trans R Soc Lond B 358(1431):593–602. doi:10.1098/rstb.2002.1238

    Article  Google Scholar 

  21. Jeannerod M (2001) Neural simulation of action: a unifying mechanism for motor cognition. Neuro Image 14(1 Pt 2):S103–109. doi:10.1006/nimg.2001.0832

  22. Guillot A, Di Rienzo F, Macintyre T, Moran A, Collet C (2012) Imagining is not doing but involves specific motor commands: a review of experimental data related to motor inhibition. Front Hum Neurosci 6(247):2012. doi:10.3389/fnhum.2012.00247.eCollection

  23. Munzert J, Lorey B, Zentgraf K (2009) Cognitive motor processes: the role of motor imagery in the study of motor representations. Brain Res Rev 60(2):306–326. doi:10.1016/j.brainresrev.2008.12.024

    Article  Google Scholar 

  24. Bakker M, de Lange FP, Stevens JA, Toni I, Bloem BR (2007) Motor imagery of gait: a quantitative approach. Exp Brain Res 179(3):497–504. doi:10.1007/s00221-006-0807-x

    Article  Google Scholar 

  25. Decety J, Jeannerod M (1995) Mentally simulated movements in virtual reality: does Fitts’s law hold in motor imagery? Behav Brain Res 72(1–2):127–134

    Article  Google Scholar 

  26. Papaxanthis C, Pozzo T, Kasprinski R, Berthoz A (2003) Comparison of actual and imagined execution of whole-body movements after a long exposure to microgravity. Neurosci Lett 339(1):41–44

    Article  Google Scholar 

  27. Papaxanthis C, Schieppati M, Gentili R, Pozzo T (2002) Imagined and actual arm movements have similar durations when performed under different conditions of direction and mass. Exp Brain Res 143(4):447–452. doi:10.1007/s00221-002-1012-1

    Article  Google Scholar 

  28. Miall RC (2003) Connecting mirror neurons and forward models. Neuroreport 14(17):2135–2137. doi:10.1097/01.wnr.0000098751.87269.77

    Article  Google Scholar 

  29. Carr L, Iacoboni M, Dubeau M-C, Mazziotta JC, Lenzi GL (2003) Neural mechanisms of empathy in humans: a relay from neural systems for imitation to limbic areas. Proc Natl Acad Sci USA 100(9):5497–5502. doi:10.1073/pnas.0935845100

    Article  Google Scholar 

  30. Iacoboni M, Koski LM, Brass M, Bekkering H, Woods RP, Dubeau MC, Mazziotta JC, Rizzolatti, G. (2001) Reafferent copies of imitated actions in the right superior temporal cortex. Proc Natl Acad Sci USA 98(24):13995–13999. doi:10.1073/pnas.241474598

  31. Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the premotor cortex. Brain 119(Pt 2):593–609

    Article  Google Scholar 

  32. Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms underlying the understanding and imitation of action. Nat Rev Neurosci 2(9):661–670. doi:10.1038/35090060

    Article  Google Scholar 

  33. Pfeifer R, Lungarella M, Iida F (2012) The challenges ahead for bio-inspired ‘soft’ robotics, vol 55(11). ACM, New York, pp 76–87. doi:10.1145/2366316.236633534

  34. Diftler MA, Mehling JS, Abdallah ME, Radford NA, Bridgwater LB, Sanders AM, Askew RS, Linn DM, Yamokoski JD, Permenter FA, Hargrave BK, Platt R, Savely RT, Ambrose RO (2011) Robonaut 2—the first humanoid robot in space. In: IEEE international conference on robotics and automation, Shanghai, China, 9–13 May 2011, pp 2178–2183

  35. Bullock IM, Ma RR, Dollar AM (2013) A hand-centric classification of human and robot dexterous manipulation. IEEE Trans Haptics 6(2):129–144. doi:10.1109/TOH.2012.53

    Article  Google Scholar 

  36. Ruini F, Apel JS, Morse AF, Cangelosi A, Ellis R, Goslin J, Fische MH (2012) Towards a Biologically-inspired Cognitive Architecture for Short-Term Memory in Humanoid Robots. In: Advances in autonomous robotics. Lecture Notes in Computer Science, vol 7429. Springer, Berlin, pp 453–454. doi:10.1007/978-3-642-32527-4_55

  37. Diamond A, Holland OE (2014) Reaching control of a full-torso, modelled musculoskeletal robot using muscle synergies emergent under reinforcement learning. Bioinspir Biomim 9:016015. doi:10.1088/1748-3182/9/1/016015

  38. Pook PK, Ballard DH (1993) Recognizing teleoperated manipulations. IEEE Comput. Soc. Press, pp 578–585. doi:10.1109/ROBOT.1993.291896

  39. Nehaniv CL, Ab HA, Dautenhahn K (1999) Of hummingbirds and helicopters: an algebraic framework for interdisciplinary studies of imitation and its applications. In: Demiris J, Birk A (eds) Interdisciplinary approaches to robot learning. World Scientific Press, Singapore, pp 136–161

    Google Scholar 

  40. Tso SK, Liu KP (1996) Hidden Markov model for intelligent extraction of robot trajectory command from demonstrated trajectories. IEEE, pp 294–298. doi:10.1109/ICIT.1996.601593

  41. Shon AP, Grochow K, Rao RPN (2005) Robotic imitation for human motion capture using gaussian processes. IEEE, pp 129–134. doi:10.1109/ICHR.2005.1573557

  42. Aleotti J, Caselli S (2005) Trajectory clustering and stochastic approximation for robot programming by demonstration. IEEE, pp 1029–1034. doi:10.1109/IROS.2005.1545365

  43. Jäkel R, Schmidt-Rohr SR, Rühl SW, Kasper A, Xue Z, Dillmann R (2012) Learning of planning models for dexterous manipulation based on human demonstrations. Int J Soc Robot 4(4):437–448. doi:10.1007/s12369-012-0162-y

    Article  Google Scholar 

  44. Mellmann H, Cotugno G (2011) Dynamic motion control: adaptive bimanual grasping for a humanoid robot. Fundam Inf 112:89–101

  45. Lee J, Chang P, Jamisola R (2013) Relative Impedance control for dual-arm robots performing asymmetric bimanual tasks. In: IEEE transactions on industrial electronics 1-1. doi:10.1109/TIE.2013.2266079

  46. Hersch M, Billard AG (2008) Reaching with multi-referential dynamical systems. Auton Robots 25(1–2):71–83. doi:10.1007/s10514-007-9070-7

    Article  Google Scholar 

  47. Mohan V, Morasso P, Metta G, Sandini G (2009) A biomimetic, force-field based computational model for motion planning and bimanual coordination in humanoid robots. Auton Robots 27(3):291–307. doi:10.1007/s10514-009-9127-x

    Article  Google Scholar 

  48. Oztop E, Lin L, Kawato M, Cheng G (2006) Dexterous skills transfer by extending human body schema to a robotic hand. IEEE, pp 82–87. doi:10.1109/ICHR.2006.321367

  49. Oztop E, Kawato M, Arbib M (2006) Mirror neurons and imitation: a computationally guided review. Neural Netw 19(3):254–271. doi:10.1016/j.neunet.2006.02.002

    Article  MATH  Google Scholar 

  50. Demiris Y, Hayes G (2002) Imitation as a dual-route process featuring predictive and learning components: a biologically-plausible computational model. In: Dautenhahn K, Nehaniv C (eds) Imitation in animals and artifacts. MIT, Cambrige, pp 327–361

    Google Scholar 

  51. Demiris Y, Khadhouri B (2006) Hierarchical attentive multiple models for execution and recognition of actions. Robot Auton Syst 54(5):361–369. doi:10.1016/j.robot.2006.02.003

    Article  Google Scholar 

  52. Sauser E, Billard A (2006) Biologically inspired multimodal integration: interferences in a human–robot interaction game. IEEE, pp 5619–5624. doi:10.1109/IROS.2006.282283

  53. Toussaint M (2004) Learning a world model and planning with a self-organizing dynamic neural system. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems 16. MIT, Cambridge, pp 929–936

    Google Scholar 

  54. Bullock D, Grossberg S, Guenther FH (1993) A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. J Cogn Neurosci 5(4):408–435. doi:10.1162/jocn.1993.5.4.408

    Article  Google Scholar 

  55. Guenther FH, Micci Barreca D (1997) Neural models for flexible control of redundant systems. In: Morasso PG, Sanguineti V (eds) Self-organization, computational maps, and motor control. Elsevier, North Holland, pp 383–421

    Chapter  Google Scholar 

  56. Fiala JC (1995) Neural network models of motor timing and coordination. PhD dissertation, Cognitive & Neural Systems Department, Boston University, Boston

  57. Gentili RJ, Papaxanthis C, Ebadzadeh M, Eskiizmirliler S, Ouanezar S, Darlot C (2009) Integration of gravitational torques in cerebellar pathways allows for the dynamic inverse computation of vertical pointing movements of a robot arm. PLoS One 4(4):e5176. doi:10.1371/journal.pone.0005176

    Article  Google Scholar 

  58. Bonaiuto J, Arbib MA (2010) Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons. Biol Cybern 102:341–359

    Article  MathSciNet  MATH  Google Scholar 

  59. Bonaiuto J, Rosta E, Arbib MA (2007) Extending the mirror neuron system model, I. Audible actions and invisible grasps. Biol Cybern 96:9–38

    Article  MathSciNet  MATH  Google Scholar 

  60. Billard A, Mataric MJ (2001) Learning human arm movements by imitation: evaluation of a biologically inspired connectionist architecture. Robot Auton Syst 37:145–160

    Article  MATH  Google Scholar 

  61. Demiris Y, Johnson M (2003) Distributed, predictive perception of actions: a biologically inspired robotics architecture for imitation and learning. Connect Sci 15(4):231–243

    Article  Google Scholar 

  62. Lopes M, Santos-Victor J (2005) Visual learning by imitation with motor representations. IEEE Trans Syst Man Cybern B 35(3):438–449

    Article  Google Scholar 

  63. Arie H, Arakaki T, Sugano S, Tani J (2012) Imitating others by composition of primitive actions: a neuro-dynamic model. Robot Auton Syst 60:729–741. doi:10.1016/j.robot.2011.11.005

    Article  Google Scholar 

  64. Roy D, Hsiao KY, Mavridis N (2004) Mental imagery for a conversational robot. IEEE Trans Syst Man Cybern B 34(3):1374–1383. doi:10.1109/TSMCB.2004.823327

  65. Nehaniv CL, Dautenhahn K (2002) The correspondence problem. Imitation in animals and artifacts. MIT, Cambridge, pp 41–61

    Google Scholar 

  66. Caggiano V, Fogassi L, Rizzolatti G, Pomper JK, Thier P, Giese MA, Casile A (2011) View-based encoding of actions in mirror neurons of area f5 in macaque premotor cortex. Curr Biol 21(2):144–148. doi:10.1016/j.cub.2010.12.022

    Article  Google Scholar 

  67. Chrisley RL (1990) Cognitive map construction and use: a parallel distributed processing approach. In: Touretzky D, Hinton G, Sejnowski T (eds) The proceedings of the 1990 connectionist models summer school. Morgan Kaufmann, San Mateo

  68. Tani J (1996) Model-based learning for mobile robot navigation from the dynamical systems perspective. IEEE Trans Syst Man Cybern B 26(3):421–436. doi:1083-4419(96)03240-2

  69. Cassimatisa NL, Trafton JG, Bugajska MD, Schultz AC (2004) Integrating cognition, perception and action through mental simulation in robots. Robot Auton Syst 49:13–23. doi:10.1016/j.robot.2004.07.014

    Article  Google Scholar 

  70. Ziemke T, Jirenhed DA, Hesslow G (2005) Internal simulation of perception: a minimal neuro-robotic model. Neurocomputing 68:85–104

    Article  Google Scholar 

  71. Hauber W (1998) Involvement of basal ganglia transmitter systems in movement initiation. Prog Neurobiol 56(5):507–540

    Article  Google Scholar 

  72. Burgess PW, Dumontheil I, Gilbert SJ (2007) The gateway hypothesis of rostral prefrontal cortex (area 10) function. Trends Cogn Sci 11(7):290–298. doi:10.1016/j.tics.2007.05.004

    Article  Google Scholar 

  73. Wolpert DM, Goodbody SJ, Husain M (1998) Maintaining internal representations: the role of the human superior parietal lobe. Nat Neurosci 1(6):529–533. doi:10.1038/2245

    Article  Google Scholar 

  74. Blakemore S-J, Sirigu A (2003) Action prediction in the cerebellum and in the parietal lobe. Exp Brain Res 153(2):239–245. doi:10.1007/s00221-003-1597-z

    Article  Google Scholar 

  75. Gentili R, Papaxanthis C, Pozzo T (2006) Improvement and generalization of arm motor performance through motor imagery practice. Neuroscience 137(3):761–772. doi:10.1016/j.neuroscience.2005.10.013

    Article  Google Scholar 

  76. Buneo CA, Andersen RA (2006) The posterior parietal cortex: sensorimotor interface for the planning and online control of visually guided movements. Neuropsychologia 44(13):2594–2606. doi:10.1016/j.neuropsychologia.2005.10.011

    Article  Google Scholar 

  77. Grefkes C, Fink GR (2005) The functional organization of the intraparietal sulcus in humans and monkeys. J Anat 207(1):3–17. doi:10.1111/j.1469-7580.2005.00426.x

    Article  Google Scholar 

  78. Oh H, Gentili RJ, Reggia JA, Contreras-Vidal JL (2011) Learning of spatial relationships between observed and imitated actions allows invariant inverse computation in the frontal mirror neuron system. In: Proceedings of annual international conference of the IEEE Engineering in Medicine and Biology Society, pp 4183–4186. doi:10.1109/IEMBS.2011.6091038

  79. Oh H, Gentili RJ, Reggia JA, Contreras-Vidal JL (2012) Modeling of visuospatial perspectives processing and modulation of the fronto-parietal network activity during action imitation. In: Proceedings of annual international conference of the IEEE Engineering in Medicine and Biology Society, pp 2551–2554. doi:10.1109/EMBC.2012.6346484

  80. Marconi B, Genovesio A, Battaglia-Mayer A, Ferraina S, Squatrito S, Molinari M, ... Caminiti R (2001) Eye-hand coordination during reaching. I. Anatomical relationships between parietal and frontal cortex. Cereb Cortex 11(6):513–527

  81. Gharbawie OA, Stepniewska I, Kaas JH (2011) Cortical connections of functional zones in posterior parietal cortex and frontal cortex motor regions in new world monkeys. Cereb Cortex 21(9):1981–2002. doi:10.1093/cercor/bhq260

    Article  Google Scholar 

  82. Georgopoulos AP, Schwartz AB, Kettner RE (1986) Neuronal population coding of movement direction. Science 233(4771):1416–1419

    Article  Google Scholar 

  83. Ito M (2013) Error detection and representation in the olivo-cerebellar system. Front Neural Circuits 7:1. doi:10.3389/fncir.2013.00001 eCollection 2013

    Article  Google Scholar 

  84. Liégeois A (1977) Automatic supervisory control of the configuration and behavior of multibody mechanisms. IEEE Trans Syst Man Cybern 7(12):868–871. doi:10.1109/TSMC.1977.4309644

    Article  MATH  Google Scholar 

  85. Baillieul J, Hollerbach, Brockett RW (1984) Programming and control of kinematically redundant manipulators. In: IEEE conference on decision and control, December 1984, pp 768–774. doi:10.1109/CDC.1984.272110

  86. Molina-Vilaplana J, Feliu-Batlle J, López-Coronado J (2007) A modular neural network architecture for step-wise learning of grasping tasks. Neural Netw 20(5):631–645. doi:10.1016/j.neunet.2007.02.003

    Article  Google Scholar 

  87. Poggio T, Girosi F (1989) A theory of networks for approximation and learning. AI Memo No. 1140. MIT, Cambridge

  88. Hartenberg RS, Denavit J (1964) Kinematic synthesis of linkages. McGraw-Hill, New York

    Google Scholar 

  89. Morrey BF, Chao EY (1976) Passive motion of the elbow joint. J Bone Jt Surg Am 58:501–508

    Google Scholar 

  90. Jeannerod M (1995) Mental imagery in the motor context. Neuropsychologia 33(I1):1419–1432. doi:10.1016/0028-3932(95)00073-C

    Article  Google Scholar 

  91. Collet C, Guillot A (2010) Autonomic nervous system activities during imagined movements. In: Guillot A, Collet C (eds) The neurophysiological foundations of mental and motor imagery. Oxford University Press, New York, pp 95–108

    Chapter  Google Scholar 

  92. Contreras-Vidal JL, Stelmach GE (1995) A neural model of basal ganglia-thalamocortieal relations in normal and parkinsonian movement. Biol Cybern 73:467–476

    Article  MATH  Google Scholar 

  93. Frak V, Cohen H, Pourcher E (2004) A dissociation between real and simulated movements in Parkinson’s disease. Neuroreport 15:1489–1492

    Article  Google Scholar 

  94. Hétu S, Grégoire M, Saimpont A, Coll MP, Eugène F, Michon PE, Jackson PL (2013) The neural network of motor imagery: an ALE meta-analysis. Neurosci Biobehav Rev 37:930–949

    Article  Google Scholar 

  95. Lotze M, Halsband U (2006) Motor imagery. J Physiol Paris 99:386–395

    Article  Google Scholar 

  96. Sauvage C, Jissendi P, Seignan S, Manto M, Habas C (2013) Brain areas involved in the control of speed during a motor sequence of the foot: real movement versus mental imagery. J Neuroradiol 40(4):267–80. doi:10.1016/j.neurad.2012.10.001

    Article  Google Scholar 

  97. Lau HC, Rogers RD, Haggard P, Passingham RE (2004) Attention to Intention. Science 20:303(5661): 1208–1210

  98. Herrero MT, Barcia C, Navarro JM (2002) Functional anatomy of thalamus and basal ganglia. Childs Nerv Syst 18:386–404. doi:10.1007/s00381-002-0604-1

    Article  Google Scholar 

  99. Jeannerod M (2006) Motor cognition. Oxford University Press, New York

    Book  Google Scholar 

  100. Lotze M, Montoya P, Erb M, Hulsmann E, Flor H, Klose U, Birbaumer N, Grodd W (1999) Activation of cortical and cerebellar motor areas during executed and imagined hand movements: an fMRI study. J Cogn Neurosci 11:491–501

    Article  Google Scholar 

  101. Aron AR, Durston S, Eagle DM, Logan GD, Stinear CM, Stuphorn V (2007) Converging evidence for a fronto-basal-ganglia network for inhibitory control of action and cognition. J Neurosci 27:11860–11864

    Article  Google Scholar 

  102. Bonnet M, Decety J, Jeannerod M, Requin J (1997) Mental simulation of an action modulates the excitability of spinal reflex pathways in man. Brain Res Cogn Brain Res 5:221–228

    Article  Google Scholar 

  103. Bradberry TJ, Gentili RJ, Contreras-Vidal JL (2010) Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals. J Neurosci 30(9):3432–3437. doi:10.1523/JNEUROSCI.6107-09.2010

    Article  Google Scholar 

  104. Visalberghi E, Limongelli L (1996) Action and understanding: tool use revisited through the mind of capuchin monkeys. In: Russon A, Bard K, Parker S (eds) Reaching into thought. The minds of the great apes. Cambridge University Press, Cambridge, pp 57–79

    Google Scholar 

  105. Gentili RJ, Oh H, Molina J, Reggia JA, Contreras-Vidal JL (2012) Cortex inspired model for inverse kinematics computation for a humanoid robotic finger. In: Proceedings of annual international conference of the IEEE Engineering in Medicine and Biology Society, pp 3052–3055. doi:10.1109/EMBC.2012.6346608

  106. Pedreño-Molina JL, Molina-Vilaplana J, López-Coronado J, Gorce P (2005) A modular neural network linking Hyper RBF and AVITE models for reaching moving objects. Robotica 23(05):625. doi:10.1017/S0263574704001055

    Article  Google Scholar 

  107. Nishimoto R, Tani J (2009) Development of hierarchical structures for actions and motor imagery: a constructivist view from synthetic neuro-robotics study. Psychol Res 73:545–558. doi:10.1007/s00426-009-0236-0

  108. deJong R, Coles MG, Logan GD, Gratton G (1990) In search of the point of no return: the control of response processes. J Exp Psychol Hum Percept Perform 16:164–182

    Article  Google Scholar 

  109. Sylvester JC, Reggia JA, Weems SA, Bunting MF (2013) Controlling working memory with learned instructions. Neural Netw 41:23–38. doi:10.1016/j.neunet.2013.01.010

    Article  Google Scholar 

  110. Jeannerod M (1994) The representing brain: neural correlates of motor intention and imagery. Behav Brain Sci 17:187–202

    Article  Google Scholar 

  111. Guillot A, Lebon F, Rouffet D, Champely S, Doyon J, Collet C (2007) Muscular responses during motor imagery as a function of muscle contraction types. Int J Psychophysiol 66:18–27

    Article  Google Scholar 

  112. Slade JM, Landers DM, Martin PE (2002) Muscular activity during real and imagined movements: a test of inflow explanations. J Sport Exerc Psychol 24:151–167

    Google Scholar 

  113. Lebon F, Rouffet D, Collet C, Guillot A (2008) Modulation of EMG power spectrum frequency during motor imagery. Neurosci Lett 435:181–185

    Article  Google Scholar 

  114. Oztop E, Bradley NS, Arbib MA (2004) Infant grasp learning: a computational model. Exp Brain Res 158:480–503. doi:10.1007/s00221-004-1914-1

    Article  Google Scholar 

  115. Oztop E, Kawato M, Arbib MA (2013) Mirror neurons: functions, mechanisms and models. Neurosci Lett 540:43–55. doi:10.1016/j.neulet.2012.10.005

  116. Schweighofer N, Arbib MA, Kawato M (1998) Role of the cerebellum in reaching movements in humans. I. Distributed inverse dynamics control. Eur J Neurosci 10(1):86–94

    Article  Google Scholar 

Download references

Acknowledgments

This research was supported by the Office of Naval Research (ONR; N000141310597), USA.

Author information

Authors and Affiliations

  1. Department of Kinesiology, School of Public Health, University of Maryland, College Park, MD, 20742, USA

    Rodolphe J. Gentili & Ross H. Miller

  2. Neuroscience and Cognitive Science Program, University of Maryland, College Park, MD, 20742, USA

    Hyuk Oh

  3. Department of Computer Science, University of Maryland, College Park, MD, 20742, USA

    Di-Wei Huang, Garrett E. Katz & James A. Reggia

Authors
  1. Rodolphe J. Gentili
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hyuk Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di-Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Garrett E. Katz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ross H. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. James A. Reggia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rodolphe J. Gentili.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (docx 329 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gentili, R.J., Oh, H., Huang, DW. et al. A Neural Architecture for Performing Actual and Mentally Simulated Movements During Self-Intended and Observed Bimanual Arm Reaching Movements. Int J of Soc Robotics 7, 371–392 (2015). https://doi.org/10.1007/s12369-014-0276-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-014-0276-5

Keywords