Abstract
This paper introduces a model for associative learning combining both linguistic and behavior modalities. The model consists of language and behavior modules both implemented by a hierarchical dynamic network model and interacting densely through hub-like neurons, the so-called parametric biases (PB). By implementing this model for a humanoid robot with the task of manipulating multiple objects, the robot was tutored to associate sentences of two different grammatical types with corresponding sensory-motor schemata. The first type was a verb followed by an objective noun such as “hold red” or “hit blue”; the second was a verb followed by an objective noun and further followed by an adverb phrase such as “Put red on blue”. Our analysis of the results of a learning experiment showed that two clusters corresponding to these two types of grammatical sentences appear in the PB activity space, such that a specific micro structure is organized for each cluster.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Evans, G.: Semantic theory and tacit knowledge. In: Holzman, S., Leich, C. (eds.) Wittgenstein: To Follow a Rule, pp. 118–137. Routledge and Kegan Paul, London (1981)
Arbib, M.: Perceptual structures and distributed motor control. In: Handbook of Physiology: The Nervous System, II. Motor Control, pp. 1448–1480. MIT Press, Cambridge (1981)
Hauk, O., Johnsrude, I., Pulvermuller, F.: Somatotopic representation of action words in human motor and premotor cortex. Neuron 41(2), 301–307 (2004)
Tettamanti, M., Buccino, G., Saccuman, M.C., Gallese, V., Danna, M., Scifo, P., Fazio, F., Rizzolatti, G., Cappa, S.F., Perani, D.: Listening to action-related sentences activates fronto-parietal motor circuits. Journal of Cognitive Neuroscience 17(2), 273–281 (2005)
Pulvermuller, F.: Brain mechanisms linking language and action. Nature Reviews Neuroscience 6, 576–582 (2005)
Tomasello, M.: Constructing a language: A usage-based theory of language acquisition. Harvard University Press, Cambridge (2003)
Sugita, Y., Tani, J.: Learning semantic combinatoriality from the interaction between linguistic and behavioral processes. Adaptive Behavior 13(1), 33–52 (2005)
Elman, J.: Finding structure in time. Cognitive Science 14, 179–211 (1990)
Hinoshita, W., Arie, H., Tani, J., Ogata, T., Okuno, H.G.: Emergence of hierarchical structure mirroring linguistic compositionality in recurrent neural network. Neural Networks (submitted)
Yamashita, Y., Tani, J.: Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Computational Biology 4(11) (2008)
Nishimoto, R., Namikawa, J., Tani, J.: Learning multiple goal-directed actions through self-organization of a dynamic neural network model: A humanoid robot experiment. Adaptive Behavior 16(2-3), 166–181 (2008)
Arie, H., Endo, T., Arakaki, T., Sugano, S., Tani, J.: Creating novel goal-directed actions at criticality: A neuro-robotic experiment. New Mathematics and Natural Computation 5(1), 307–334 (2009)
Tani, J., Nishimoto, R., Paine, R.: Achieving “organic compositionality” through self-organization: Reviews on brain-inspired robotics experiments. Neural Networks, 584–603 (2008)
Ehrsson, H., Fagergren, A., Johansson, R., Forssberg, H.: Evidence for the involvement of the posterior parietal cortex in coordination of fingertip forces for grasp stability in manipulation. Journal of Neurophysiology 90, 2978–2986 (2003)
Bruce, C.J., Goldberg, M.E.: Primate frontal eye fields. i. single neurons discharging before saccades. Journal of Neurophysiology 53(3), 603–635 (1985)
Colby, C.L., Goldberg, M.E.: Space and attention in parietal cortex. Annual Review of Neuroscience 22, 319–349 (1999)
Jeong, S., Lee, M., Arie, H., Tani, J.: Developmental learning of integrating visual attention shifts and bimanual behavior in object manipulation tasks. In: Proc. of 2010 IEEE 9th International Conference on Development and Learning (2010) (submitted)
Wilson, S., Saygin, A., Sereno, M., Iacoboni, M.: Listening to speech activates motor areas involved in speech production. Nature Neuroscience 7(7), 701–702 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Arie, H., Endo, T., Jeong, S., Lee, M., Sugano, S., Tani, J. (2010). Integrative Learning between Language and Action: A Neuro-Robotics Experiment. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds) Artificial Neural Networks – ICANN 2010. ICANN 2010. Lecture Notes in Computer Science, vol 6353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15822-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-15822-3_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15821-6
Online ISBN: 978-3-642-15822-3
eBook Packages: Computer ScienceComputer Science (R0)