Skip to main content
Log in

Robot behaviour learning using Topological Gaussian Adaptive Resonance Hidden Markov Model

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Behaviour learning by robots is an emerging research field. The robot behaviour learning in realistic environment requires the robot to be able to learn incrementally in order to adapt itself to the dynamic environment. The incremental learning system should be stable and acquire new information without corrupting previous knowledge. Probabilistic approach has become the prevalent epitome in robotic learning systems. In probabilistic learning models, such as Hidden Markov Model, estimating the optimal number of states is difficult which limits the models capabilities by limiting the number of motion patterns to be learned a priori. In this paper, we presented an approach for learning behaviour patterns through continuous observation. We have proposed a novel architecture for learning the spatio-temporal sequences using Topological Gaussian Adaptive Resonance Hidden Markov Model. The proposed model dynamically generates the graph-based structure for the observed patterns through a novel topological mapping architecture. The structure (number of states) of the probabilistic model is not fixed and is adjusted based on the topological map of the acquired motion features. The topological map consists of nodes connected with edges, where each node represents the encoded motion features to adaptively generalize the observed behaviour patterns. The model combines the self-organizing, self-stabilizing properties and time series processing features to learn the spatio-temporal sequences. To demonstrate the properties of the proposed algorithm, we have performed a set of experiments through simulation on DARwIn-OP humanoid robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  2. Inamura T, Toshima I, Tanie H, Nakamura Y (2004) Embodied symbol emergence based on mimesis theory. Int J Robot Res 23(4–5):363–377

    Article  Google Scholar 

  3. Calinon S, Billard A (2004) Stochastic gesture production and recognition model for a humanoid robot. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems, 2004 (IROS 2004), pp 2769–2774

  4. Grossberg S (1987) Competitive learning: from interactive activation to adaptive resonance. Cogn Sci 11(1):23–63

    Article  MathSciNet  Google Scholar 

  5. Billard AG, Calinon S, Guenter F (2006) Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robot Auton Syst 54(5):370–384

    Article  Google Scholar 

  6. Kulic D, Takano W, Nakamura Y (2007) Incremental on-line hierarchical clustering of whole body motion patterns. In: The 16th robot and human interactive communication 2007, RO–MAN 2007

  7. Grossberg S (2013) Adaptive resonance theory: how a brain learns to consciously attend, learn, and recognize a changing world. Neural Netw 37:1–47

    Article  Google Scholar 

  8. Jain L, Seera M, Lim C, Balasubramaniam P (2014) A review of online learning in supervised neural networks. Neural Comput Appl 25(3–4):491–509

    Article  Google Scholar 

  9. Carpenter GA, Grossberg S (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput Vision Graph Image Process 37(1):54–115

    Article  MATH  Google Scholar 

  10. Carpenter GA, Grossberg S (1987) Art 2: self-organization of stable category recognition codes for analog input patterns. Appl Opt 26(23):4919–4930

    Article  Google Scholar 

  11. Carpenter G, Grossberg S, Rosen D (1991) Art 2–A: an adaptive resonance algorithm for rapid category learning and recognition. In: International joint conference on neural networks, IJCNN–91, pp 151–156

  12. Bradski G, Carpenter G, Grossberg S (1994) Store working memory networks for storage and recall of arbitrary temporal sequences. Biol Cybern 71(6):469–480

    Article  Google Scholar 

  13. Furao S, Hasegawa O (2006) An incremental network for on-line unsupervised classification and topology learning. Neural Netw 19(1):90–106

    Article  MATH  Google Scholar 

  14. Furao S, Hasegawa O (2008) A fast nearest neighbor classifier based on self-organizing incremental neural network. Neural Netw 21(10):1537–1547

    Article  MATH  Google Scholar 

  15. Botzheim J, Kubota N (2012) Growing neural gas for information extraction in gesture recognition and reproduction of robot partners. In: IEEE 2012 international symposium on micro-nanomechatronics and human science (MHS), pp 149–154

  16. Seyhan SS, Alpaslan FN, Yava M (2013) Simple and complex behavior learning using behavior hidden markov model and cobart. Neurocomputing 103:121–131

    Article  Google Scholar 

  17. Yavas M, Alpaslan FN (2012) Hierarchical behavior categorization using correlation based adaptive resonance theory. Neurocomputing 77(1):71–81

    Article  Google Scholar 

  18. Kulic D, Lee D, Ott C, Nakamura Y (2008) Incremental learning of full body motion primitives for humanoid robots. In: IEEE–RAS 8th international conference on humanoid robots, humanoids 2008, pp 326–332

  19. Kulic D, Ott C, Lee D, Ishikawa J, Nakamura Y (2012) Incremental learning of full body motion primitives and their sequencing through human motion observation. Int J Robot Res 31:330–345

    Article  Google Scholar 

  20. Kulic D, Takano W, Nakamura Y (2011) Towards lifelong learning and organization of whole body motion patterns. In: Kaneko M, Nakamura Y (eds) Robotics research, springer tracts in advanced robotics, vol 66. Springer, Berlin, pp 87–97

    Google Scholar 

  21. Calinon S, Billard A (2007) Incremental learning of gestures by imitation in a humanoid robot. In: Proceedings of the ACM/IEEE international conference on Human-robot interaction, ACM, New York, NY, USA, HRI ’07, pp 255–262

  22. Cho S, Jo S (2011) Kinesthetic learning of behaviors in a humanoid robot. In: 2011 11th international conference on control, automation and systems (ICCAS), pp 1108–1112

  23. Cho S, Jo S (2012) Incremental motion learning through kinesthetic teachings and new motion production from learned motions by a humanoid robot. Int J Control Autom Syst 10(1):126–135

    Article  Google Scholar 

  24. Vasquez D, Fraichard T, Laugier C (2009) Incremental learning of statistical motion patterns with growing hidden markov models. IEEE Trans Intell Transp Syst 10(3):403–416

    Article  Google Scholar 

  25. Tscherepanow M, Kortkamp M, Kammer M (2011) A hierarchical art network for the stable incremental learning of topological structures and associations from noisy data. Neural Netw 24(8):906–916

    Google Scholar 

  26. Wk Fung, Liu YH (2003) Adaptive categorization of art networks in robot behavior learning using game-theoretic formulation. Neural Netw 16(10):1403–1420

    Article  Google Scholar 

  27. Jun L, Duckett T (2003) Robot behavior learning with a dynamically adaptive RBF network: experiments in offline and online learning. In: Proceedings of the second international conference on computational intelligence, robotics and autonomous system, Singapore

  28. Martinetz T, Schulten K (1994) Topology representing networks. Neural Netw 7(3):507–522

    Article  Google Scholar 

  29. Neal RM, Hinton GE (1999) A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan MI (ed) Learning in graphical models. MIT Press, Cambridge, pp 355–368

    Google Scholar 

  30. Haa I, Tamuraa Y, Asamaa H (2013) Development of open platform humanoid robot darwin-op. Adv Robot 27(3):223–232

    Article  Google Scholar 

Download references

Acknowledgments

This research is supported by High Impact Research Grant UM.C/625/1/HIR/MoE/FCSIT/10 from the Ministry of Education Malaysia.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chu Kiong Loo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dawood, F., Loo, C.K. Robot behaviour learning using Topological Gaussian Adaptive Resonance Hidden Markov Model. Neural Comput & Applic 27, 2509–2522 (2016). https://doi.org/10.1007/s00521-015-2021-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2021-x

Keywords

Navigation