Abstract
We describe a novel approach that allows humanoid robots to incrementally integrate motion primitives and language expressions, when there are underlying natural language and motion language modules. The natural language module represents sentence structure using word bigrams. The motion language module extracts the relations between motion primitives and the relevant words. Both the natural language module and the motion language module are expressed as probabilistic models and, therefore, they can be integrated so that the robots can both interpret observed motion in the form of sentences and generate the motion corresponding to a sentence command. Incremental learning is needed for a robot that develops these linguistic skills autonomously . The algorithm is derived from optimization of the natural language and motion language modules under constraints on their probabilistic variables such that the association between motion primitive and sentence in incrementally added training pairs is strengthened. A test based on interpreting observed motion in the forms of sentence demonstrates the validity of the incremental statistical learning algorithm.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Argall, B., Chernova, S., Veloso, M., & Browning, B. (2009). A survey of robot learning from demonstration. Robotics and Autonomous Systems, 57(5), 469–483.
Billard, A., Calinon, S., & Guenter, F. (2006). Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Systems, 54, 370–384.
Breazeal, C., & Scassellati, B. (2002). Robots that imitate humans. Trends in Cognitive Sciences, 6(11), 481–487.
Deacon, T. (1997). Symbolic species: The co-evolution of language and the brain. New York: W.W. Norton and Company Inc.
Gallese, V., & Goldman, A. (1998). Mirror neuron and the simulation theory of mind-reading. Trends in Cognitive Sciences, 2(12), 493–501.
Haruno, M., Wolpert, D., & Kawato, M. (2001). Mosaic model for sensorimotor learning and control. Neural Computation, 13, 2201–2220.
Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. (2004). Embodied symbol emergence based on mimesis theory. International Journal of Robotics Research, 23(4), 363–377.
Kadone, H., & Nakamura, Y. (2005). Symbolic memory for humanoid robots using hierarchical bifurcations of attractors in nonmonotonic neural networks. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 2900–2905).
Ogata, T., Matsumoto, S., Tani, J., Komatani, K., & Okuno, H. G. (2007). Human-robot cooperation using quasi-symbols generated by rnnpb model. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2156–2161).
Ogata, T., Murase, M., Tani, J., Komatani, K., & Okuno, H. G. (2007). Two-way translation of compound sentences and arm motions by recurrent neural networks. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, (pp. 1858–1863).
Peirce, C. S. (1931–1958). The collected papers of Charles Sanders Peirce. Cambridge, MA: Harvard University Press.
Rabiner, L., & Juang, B. H. (1993). Fundamentals of speech recognition. Prentice Hall Signal Processing Series. Englewood Cliffs, NJ: Prentice Hall.
Rizzolatti, G., Fogassi, L., & Gallese, V. (2001). Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Reviews, 2, 661–670.
Saussure, F. D. (1966). Course in general linguistics. New York: McGraw-Hill Book Company.
Sugita, Y., & Tani, J. (2005). Learning semantic combinatoriality from the interaction between linguistic and behavioral processes. Adaptive Behavior, 13(1), 33–52.
Takano, W., & Nakamura, Y. (2015a). Statistical mutual conversion between whole body motion primitives and linguistic sentences for human motions. International Journal of Robotics Research. doi:10.1177/0278364915587923.
Takano, W., & Nakamura, Y. (2015b). Construction of a space of motion labels from their mapping to full-body motion symbols. Advanced Robotics, 29(2), 115–126.
Tani, J., & Ito, M. (2003). Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man and Cybernetics Part A, 33(4), 481–488.
Taylor, G. W., Hinton, G. E., & Roweis, S. (2006). Modeling human motion using binary latent variables. In Proceedings of the Conference on Neural Information Processing Systems (pp. 1345–1352).
Acknowledgments
This research was partially supported by a Grant-in-Aid for Young Scientists (A) (26700021) from the Japan Society for the Promotion of Science, and the Strategic Information and Communications R&D Promotion Program (142103011) of the Ministry of Internal Affairs and Communications.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Takano, W., Nakamura, Y. Incremental statistical learning for integrating motion primitives and language in humanoid robots. Auton Robot 40, 657–667 (2016). https://doi.org/10.1007/s10514-015-9486-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10514-015-9486-4