Abstract
Behaviour learning by robots is an emerging research field. The robot behaviour learning in realistic environment requires the robot to be able to learn incrementally in order to adapt itself to the dynamic environment. The incremental learning system should be stable and acquire new information without corrupting previous knowledge. Probabilistic approach has become the prevalent epitome in robotic learning systems. In probabilistic learning models, such as Hidden Markov Model, estimating the optimal number of states is difficult which limits the models capabilities by limiting the number of motion patterns to be learned a priori. In this paper, we presented an approach for learning behaviour patterns through continuous observation. We have proposed a novel architecture for learning the spatio-temporal sequences using Topological Gaussian Adaptive Resonance Hidden Markov Model. The proposed model dynamically generates the graph-based structure for the observed patterns through a novel topological mapping architecture. The structure (number of states) of the probabilistic model is not fixed and is adjusted based on the topological map of the acquired motion features. The topological map consists of nodes connected with edges, where each node represents the encoded motion features to adaptively generalize the observed behaviour patterns. The model combines the self-organizing, self-stabilizing properties and time series processing features to learn the spatio-temporal sequences. To demonstrate the properties of the proposed algorithm, we have performed a set of experiments through simulation on DARwIn-OP humanoid robot.
Similar content being viewed by others
References
Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286
Inamura T, Toshima I, Tanie H, Nakamura Y (2004) Embodied symbol emergence based on mimesis theory. Int J Robot Res 23(4–5):363–377
Calinon S, Billard A (2004) Stochastic gesture production and recognition model for a humanoid robot. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, 2004 (IROS 2004), pp 2769–2774
Grossberg S (1987) Competitive learning: from interactive activation to adaptive resonance. Cogn Sci 11(1):23–63
Billard AG, Calinon S, Guenter F (2006) Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robot Auton Syst 54(5):370–384
Kulic D, Takano W, Nakamura Y (2007) Incremental on-line hierarchical clustering of whole body motion patterns. In: The 16th robot and human interactive communication 2007, RO–MAN 2007
Grossberg S (2013) Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw 37:1–47
Jain L, Seera M, Lim C, Balasubramaniam P (2014) A review of online learning in supervised neural networks. Neural Comput Appl 25(3–4):491–509
Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vision Graph Image Process 37(1):54–115
Carpenter GA, Grossberg S (1987) Art 2: self-organization of stable category recognition codes for analog input patterns. Appl Opt 26(23):4919–4930
Carpenter G, Grossberg S, Rosen D (1991) Art 2–A: an adaptive resonance algorithm for rapid category learning and recognition. In: International joint conference on neural networks, IJCNN–91, pp 151–156
Bradski G, Carpenter G, Grossberg S (1994) Store working memory networks for storage and recall of arbitrary temporal sequences. Biol Cybern 71(6):469–480
Furao S, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19(1):90–106
Furao S, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21(10):1537–1547
Botzheim J, Kubota N (2012) Growing neural gas for information extraction in gesture recognition and reproduction of robot partners. In: IEEE 2012 international symposium on micro-nanomechatronics and human science (MHS), pp 149–154
Seyhan SS, Alpaslan FN, Yava M (2013) Simple and complex behavior learning using behavior hidden markov model and cobart. Neurocomputing 103:121–131
Yavas M, Alpaslan FN (2012) Hierarchical behavior categorization using correlation based adaptive resonance theory. Neurocomputing 77(1):71–81
Kulic D, Lee D, Ott C, Nakamura Y (2008) Incremental learning of full body motion primitives for humanoid robots. In: IEEE–RAS 8th international conference on humanoid robots, humanoids 2008, pp 326–332
Kulic D, Ott C, Lee D, Ishikawa J, Nakamura Y (2012) Incremental learning of full body motion primitives and their sequencing through human motion observation. Int J Robot Res 31:330–345
Kulic D, Takano W, Nakamura Y (2011) Towards lifelong learning and organization of whole body motion patterns. In: Kaneko M, Nakamura Y (eds) Robotics research, springer tracts in advanced robotics, vol 66. Springer, Berlin, pp 87–97
Calinon S, Billard A (2007) Incremental learning of gestures by imitation in a humanoid robot. In: Proceedings of the ACM/IEEE international conference on Human-robot interaction, ACM, New York, NY, USA, HRI ’07, pp 255–262
Cho S, Jo S (2011) Kinesthetic learning of behaviors in a humanoid robot. In: 2011 11th international conference on control, automation and systems (ICCAS), pp 1108–1112
Cho S, Jo S (2012) Incremental motion learning through kinesthetic teachings and new motion production from learned motions by a humanoid robot. Int J Control Autom Syst 10(1):126–135
Vasquez D, Fraichard T, Laugier C (2009) Incremental learning of statistical motion patterns with growing hidden markov models. IEEE Trans Intell Transp Syst 10(3):403–416
Tscherepanow M, Kortkamp M, Kammer M (2011) A hierarchical art network for the stable incremental learning of topological structures and associations from noisy data. Neural Netw 24(8):906–916
Wk Fung, Liu YH (2003) Adaptive categorization of art networks in robot behavior learning using game-theoretic formulation. Neural Netw 16(10):1403–1420
Jun L, Duckett T (2003) Robot behavior learning with a dynamically adaptive RBF network: experiments in offline and online learning. In: Proceedings of the second international conference on computational intelligence, robotics and autonomous system, Singapore
Martinetz T, Schulten K (1994) Topology representing networks. Neural Netw 7(3):507–522
Neal RM, Hinton GE (1999) A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan MI (ed) Learning in graphical models. MIT Press, Cambridge, pp 355–368
Haa I, Tamuraa Y, Asamaa H (2013) Development of open platform humanoid robot darwin-op. Adv Robot 27(3):223–232
Acknowledgments
This research is supported by High Impact Research Grant UM.C/625/1/HIR/MoE/FCSIT/10 from the Ministry of Education Malaysia.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dawood, F., Loo, C.K. Robot behaviour learning using Topological Gaussian Adaptive Resonance Hidden Markov Model. Neural Comput & Applic 27, 2509–2522 (2016). https://doi.org/10.1007/s00521-015-2021-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-015-2021-x