Skip to main content
SpringerLink
Log in

Application of emotion affected associative memory based on mood congruency effects for a humanoid

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Emotional factor plays an important role in communication. In the field of psychology, it is known that memory and emotions are closely related to each other. In this paper, we present the significance of emotional factors to associative memory in communication and apply it on human–robot interaction problems. Emotional models for the robot partner are developed, and an interactive robot system with a complex-valued multi-directional associative memory model is proposed. We utilize multi-modal information such as object, gesture, voice, and facial expressions to associate the relationships in associative memory, and generate the emotional information for the robot partner. As a result, the robot partner is able to perform various actions depending on the emotional factors. Results from the interactive experiments indicate possibility of suitable information for communication space being provided from the robot partner.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Pfeifer R, Scheier C (1999) Understanding intelligence. The MIT Press, Cambridge

    Google Scholar 

  2. Gregory RL (1998) The mind. Oxford Univ Press, Oxford

    Google Scholar 

  3. Sperber D, Wilson D (1995) Relevance: communication and cognition. Oxford Univ Press, Oxford

    Google Scholar 

  4. Minsky M (1986) The society of mind. Simon and Schuster, New York

    Google Scholar 

  5. Yorita A, Kubota N (2011) Cognitive development in partner robots for information support to elderly people. IEEE Trans Auton Ment Dev 3(1):64–73

    Article  Google Scholar 

  6. Zheng K, Glas DF, Kanda T, Ishiguro H, Hagita N (2013) Designing and implementing a human–robot team for social interactions. IEEE Trans Syst Man Cybern Syst 43(4):843–859

    Article  Google Scholar 

  7. Toris R, Kent D, Chernova S (2014) The robot management system: A framework for conducting human–robot interaction studies through crowdsourcing. J Hum Robot Interact 3(2):25–49

    Article  Google Scholar 

  8. Bechara A, Damasio H, Damasio AR (2000) Emotion, decision making and the orbitofrontal cortex. Cereb Cortex 10(3):295–307

    Article  Google Scholar 

  9. Reisberg DE, Hertel PE (2004) Memory and emotion. Oxford University Press, Oxford

    Book  Google Scholar 

  10. Ekkekakis P (2013) The measurement of affect, mood, and emotion: a guide for health-behavioral research. Cambridge University Press, Cambridge

    Book  Google Scholar 

  11. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    Article  MathSciNet  MATH  Google Scholar 

  12. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386

    Article  Google Scholar 

  13. Aizenberg N, Ivaskiv YL, Pospelov D, Hudiakov G (1973) Multiple-valued threshold functions. II. Synthesis of the multi-valued threshold elements. Kibernetika (Cybernetics) 1:53–66

    Google Scholar 

  14. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci 79(8):2554–2558

    Article  MathSciNet  Google Scholar 

  15. Kosko B (1987) Constructing an associative memory. Byte 12(10):137–144

    Google Scholar 

  16. Hagiwara M (1990) Multidirectional associative memory. In: International joint conference on neural networks, vol 1, pp 3–6

  17. Noest A (1988) Discrete-state phasor neural networks. Phys Rev A 38(4):2196–2199

    Article  Google Scholar 

  18. Jankowski S, Lozowski A, Zurada J (1996) Complex-valued multistate neural associative memory. IEEE Trans Neural Netw 7(6):1491–1496

    Article  Google Scholar 

  19. Donq-Liang L, Wen-June W (1998) A multivalued bidirectional associative memory operating on a complex domain. Neural Netw 11(9):1623–1635

    Article  Google Scholar 

  20. Kobayashi M, Yamazaki H (2005) Complex-valued multidirectional associative memory. IEEJ Trans Electron Inf Syst 125:1290–1295

    Google Scholar 

  21. Kohonen T (1972) Correlation matrix memories. IEEE Trans Comput 100(4):353–359

    Article  MATH  Google Scholar 

  22. Kitahara M, Kobayashi M (2014) Projection rule for rotor hopfield neural networks. IEEE Trans Neural Netw Learn Syst 25(7):1298–1307

    Article  Google Scholar 

  23. Oh H, Kothari SC (1994) Adaptation of the relaxation method for learning in bidirectional associative memory. IEEE Trans Neural Netw 5(4):576–583

    Article  Google Scholar 

  24. Shimizu Y, Osana Y (2010) Chaotic complex-valued multidirectional associative memory. In: IASTED artificial intelligence and applications

  25. Yoshida A, Osana Y (2011) Chaotic complex-valued multidirectional associative memory with variable scaling factor. In: Artificial neural networks and machine learning—ICANN 2011. Springer, pp 266–274

  26. Gandhi V, Prasad G, Coyle D, Behera L, McGinnity T (2014) Quantum neural network-based eeg filtering for a brain–computer interface. IEEE Trans Neural Netw Learn Syst 25(2):278–288

    Article  Google Scholar 

  27. Chen C, Dong D, Li H, Chu J, Tarn T (2014) Fidelity-based probabilistic q-learning for control of quantum systems. IEEE Trans Neural Netw Learn Syst 25(5):920–933

    Article  Google Scholar 

  28. Rigatos GG, Tzafestas SG (2006) Quantum learning for neural associative memories. Fuzzy Sets Syst 157(13):1797–1813

    Article  MathSciNet  MATH  Google Scholar 

  29. Ekman P (1992) An argument for basic emotions. Cognit Emot 6(3–4):169–200

    Article  Google Scholar 

  30. Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39(6):1161

    Article  Google Scholar 

  31. Han MJ, Lin CH, Song KT (2013) Robotic emotional expression generation based on mood transition and personality model. IEEE Trans Cybern 43(4):1290–1303

    Article  Google Scholar 

  32. Schneider M, Adamy J (2014) Towards modelling affect and emotions in autonomous agents with recurrent fuzzy systems. In: 2014 IEEE international conference on systems, man and cybernetics (SMC). IEEE, pp 31–38

  33. Russell JA, Barrett LF (1999) Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. J Personal Soc Psychol 76(5):805

    Article  Google Scholar 

  34. Ortony A (1990) The cognitive structure of emotions. Cambridge University Press, Cambridge

    Google Scholar 

  35. Mehrabian A (1971) Silent messages

  36. LeDoux JE (1994) Emotion, memory and the brain. Sci Am 270(6):50–57

    Article  Google Scholar 

  37. Ekman P (1993) Facial expression and emotion. Am Psychol 48(4):384

    Article  Google Scholar 

  38. Friedman HS, Riggio RE (1981) Effect of individual differences in nonverbal expressiveness on transmission of emotion. J Nonverbal Behav 6(2):96–104

    Article  Google Scholar 

  39. Cristinacce D, Cootes T (2008) Automatic feature localisation with constrained local models. Pattern Recognit 41(10):3054–3067

    Article  MATH  Google Scholar 

  40. Nuevo J, Bergasa LM, Jim’enez P (2010) RSMAT: robust simultaneous modeling and tracking. Pattern Recognit Lett 31(16):2455–2463

    Article  Google Scholar 

  41. Dawood F, Loo CK, Chin WH (2013) Incremental on-line learning of human motion using gaussian adaptive resonance hidden markov model. In: The 2013 international joint conference on neural networks (IJCNN). IEEE, pp 1–7

  42. Smith SM, Petty RE (1995) Personality moderators of mood congruency effects on cognition: the role of self-esteem and negative mood regulation. J Personal Soc Psychol 68(6):1092

    Article  Google Scholar 

  43. Kubota N, Toda Y (2012) Multimodal communication for human-friendly robot partners in informationally structured space. IEEE Trans Syst Man Cybern Part C Appl Rev 42(6):1142–1151

    Article  Google Scholar 

  44. Lee A, Kawahara T (2009) Recent development of open-source speech recognition engine Julius. In: Proceedings: APSIPA ASC 2009, pp 131–137

  45. Webb J, Ashley J (2012) Beginning Kinect Programming with the Microsoft Kinect SDK. Apress, New York

    Book  Google Scholar 

  46. Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215

    Article  MathSciNet  MATH  Google Scholar 

  47. Cootes TF, Taylor CJ (1992) Active shape models—‘smart snakes’. In: BMVC92. Springer, pp 266–275

  48. Di Stefano L, Mattoccia S, Tombari F (2005) ZNCC-based template matching using bounded partial correlation. Pattern Recognit Lett 26(14):2129–2134

    Article  Google Scholar 

  49. Kanaujia A, Huang Y, Metaxas D (2006) Tracking facial features using mixture of point distribution models. In: Computer vision, graphics and image processing. Springer, pp 492–503

  50. Rychlak JF (1973) Introduction to personality and psychotherapy: A theory-construction approach. Houghton Mifflin, Boston

    Google Scholar 

  51. Mischel W (1993) Introduction to personality, vol 250. London

Download references

Acknowledgments

This research is supported by Collaborative Research in Engineering, Science & Technology Grant P05C2-14 and University of Malaya Grant UM.C/625/1/HIR /MoE/FCSIT/10.

Author information

Authors and Affiliations

  1. Advanced Robotics Lab, University of Malaya, Lembah Pantai, 50603, Kuala Lumpur, Malaysia

    Naoki Masuyama, Md. Nazrul Islam & Chu Kiong Loo

  2. Swinburne University of Technology (Sarawak Campus), 93350, Kuching, Malaysia

    Manjeevan Seera

Authors
  1. Naoki Masuyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Md. Nazrul Islam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manjeevan Seera
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chu Kiong Loo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manjeevan Seera.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Masuyama, N., Islam, M.N., Seera, M. et al. Application of emotion affected associative memory based on mood congruency effects for a humanoid. Neural Comput & Applic 28, 737–752 (2017). https://doi.org/10.1007/s00521-015-2102-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-015-2102-x

Keywords

Search

Navigation