Skip to main content
SpringerLink
Log in

Emotion Recognition Using Facial Expression Images for a Robotic Companion

  • Conference paper
  • First Online:
Engineering Applications of Neural Networks (EANN 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 629))

Abstract

Social robots are gradually becoming part of society. However, social robots lack the ability to adequately interact with users in a natural manner and are in need of more human-like abilities. In this paper we present experimental results on emotion recognition through the use of facial expression images obtained from the KDEF database, a fundamental first step towards the development of an empathic social robot. We compare the performance of Support Vector Machines (SVM) and a Multilayer Perceptron Network (MLP) on facial expression classification. We employ Gabor filters as an image pre-processing step before classification. Our SVM model achieves an accuracy rate of 97.08 %, whereas our MLP achieves 93.5 %. These experiments serve as benchmark for our current research project in the area of social robotics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Castellano, G., Paiva, A., Kappas, A., Aylett, R., Hastie, H., Barendregt, W., Nabais, F., Bull, S.: Towards empathic virtual and robotic tutors. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 733–736. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  2. Dautenhahn, K., Campbell, A., Syrdal, D.: Does anyone want to talk to me? Reflections on the use of assistance and companion robots in care homes. In: 4th International Symposium on New Frontiers in Human-Robot Interaction (2015)

    Google Scholar 

  3. Cameron, D., Fernando, S., Collins, E., Millings, A., Moore, R., Sharkey, A., Evers, V.: Presence of life-like robot expressions influences childrens enjoyment of human-robot interactions in the field. In: 4th International Symposium on New Frontiers in Human-Robot Interaction (2015)

    Google Scholar 

  4. Darling, K.: Who’s Johnny? Anthropomorphic framing in human-robot interaction, integration, and policy. SSRN Electron. J. (2015)

    Google Scholar 

  5. Wada, K., Asada, T., Musha, T.: Robot therapy for prevention of dementia at home results of preliminary experiment. J. Robot. Mechatron. 19, 691–697 (2007)

    Article  Google Scholar 

  6. GeriJoy: Care and Companionship for Seniors-GeriJoy (2016). http://www.gerijoy.com/

  7. KSERA: Knowledgeable Service Robots for Aging (2016). http://www.aat.tuwien.ac.at/index_en.html/

  8. Wainer, J., Robins, B., Amirabdollahian, F., Dautenhahn, K.: Using the humanoid robot KASPAR to autonomously play triadic games and facilitate collaborative play among children with autism. IEEE Trans. Auton. Ment. Dev. 6, 183–199 (2014)

    Article  Google Scholar 

  9. Leite, I., Pereira, A., Mascarenhas, S., Martinho, C., Prada, R., Paiva, A.: The influence of empathy in human robot relations. Int. J. Hum. Comput. Stud. 71, 250–260 (2013)

    Article  Google Scholar 

  10. Lamm, C., Silani, G.: The neural underpinnings of empathy and their relevance for collective emotions. In: Scheve, C., Salmella, M. (eds.) Collective Emotions. Oxford University Press, Oxford (2014)

    Google Scholar 

  11. Shamay-Tsoory, S.: The neural bases for empathy. Neuroscientist 17(1), 18–24 (2010)

    Article  Google Scholar 

  12. Bernhardt, B., Singer, T.: The neural basis of empathy. Annu. Rev. Neurosci. 35, 1–23 (2012)

    Article  Google Scholar 

  13. Kilner, J., Lemon, R.: What we know currently about mirror neurons. Curr. Biol. 23, R1057–R1062 (2013)

    Article  Google Scholar 

  14. Rizzolatti, G., Fadiga, L., Gallese, V., Fogassi, L.: Premotor cortex and the recognition of motor actions. Cogn. Brain Res. 3, 131–141 (1996)

    Article  Google Scholar 

  15. Singer, T., Seymour, B., Doherty, J., Kaube, H., Dolan, R., Frith, C.: Empathy for pain involves the affective but not sensory components of pain. Science 303, 1157–1162 (2004)

    Article  Google Scholar 

  16. Lamm, C., Majdand, J.: The role of shared neural activations, mirror neurons, and morality in empathy A critical comment. Neurosci. Res. 90, 15–24 (2015)

    Article  Google Scholar 

  17. Agnew, Z., Bhakoo, K., Puri, B.: The human mirror system: a motor resonance theory of mind-reading. Brain Res. Rev. 54, 286–293 (2007)

    Article  Google Scholar 

  18. Duffy, B.R.: Fundamental issues in social robotics. Int. Rev. Inf. Ethics 6, 31 (2006)

    Google Scholar 

  19. Boughrara, H., Chtourou, M., Ben Amar, C., Chen, L.: Facial expres-sion recognition based on a MLP neural network using constructive training algorithm. Multimed. Tools Appl. 75, 709–731 (2014)

    Article  Google Scholar 

  20. Kahou, S., Michalski, V., Konda, K., Memisevic, R., Pal, C.: Recurrent neural networks for emotion recognition in video. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI 2015), pp. 467–474 (2015)

    Google Scholar 

  21. Lawrence, S., Giles, C., Tsoi, A.C., Back, A.: Face recognition: a convolutional neural network approach. IEEE Trans. Neural Netw. 8, 98–113 (1997)

    Article  Google Scholar 

  22. Gupta, A., Garg, M.: A human emotion recognition system using supervised self-organising maps. In: 2014 International Conference on Computing for Sustainable Global Development (INDIACom), pp. 654–659 (2016)

    Google Scholar 

  23. Sarnarawickrame, K., Mindya, S.: Facial expression recognition using active shape models and support vector machines. In: 2013 International Conference on Advances in ICT for Emerging Regions (ICTer), pp. 51–55 (2013)

    Google Scholar 

  24. Sohail, A., Bhattacharya, P.: Classifying facial expressions using level set method based lip contour detection and multi-class support vector machines. Int. J. Pattern Recogn. Artif. Intell. 25, 835–862 (2011)

    Article  MathSciNet  Google Scholar 

  25. Hewahi, N., Baraka, A.: Impact of ethnic group on human emotion recognition using backpropagation neural network. Broad Res. Artif. Intell. Neurosci. 2, 20 (2011)

    Google Scholar 

  26. Khashman, A.: Application of an emotional neural network to facial recognition. Neural Comput. Applic. 18, 309–320 (2008)

    Article  Google Scholar 

  27. Ouellet, S.: Realtime emotion recognition for gaming using deep convolutional network features (2014)

    Google Scholar 

  28. Burkert, P., Trier, F., Afzal, M., Dengel, A., Liwicki, M.: DeXpression: deep convolutional neural network for expression recognition (2015)

    Google Scholar 

  29. Levi, G., Hassner, T.: Emotion recognition in the wild via convolutional neural networks and mapped binary patterns. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (ICMI 2015), pp. 503–510 (2015)

    Google Scholar 

  30. Ahsan, T., Jabid, T., Chong, U.: Facial expression recognition using local transitional pattern on gabor filtered facial images. IETE Tech. Rev. 30, 47 (2013)

    Article  Google Scholar 

  31. Chelali, F., Djeradi, A.: Face recognition using MLP and RBF neural network with Gabor and discrete wavelet transform characterization: a comparative study. Math. Probl. Eng. 2015, 116 (2015)

    Article  Google Scholar 

  32. Beaudry, O., Roy-Charland, A., Perron, M., Cormier, I., Tapp, R.: Featural processing in recognition of emotional facial expressions. Cogn. Emot. 28, 416–432 (2013)

    Article  Google Scholar 

  33. Lundqvist, D., Flykt, A., Öhman, A.: The Karolinska Directed Emotional Faces - KDEF. CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet (1998). ISBN: 91-630-7164-9

    Google Scholar 

  34. Wang, J., Cheng, J.: Face recognition based on fusion of Gabor and 2DPCA features. In: International Symposium on Intelligent Signal Processing and Communication Systems (2010)

    Google Scholar 

  35. Kuhn, H., Tucker, A.: Nonlinear programming. In: Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, pp. 481–492 (1951)

    Google Scholar 

  36. Riedmiller, M., Braun, H.: A direct adaptive method for faster backpropagation learning: the RPROP algorithm. Neural Netw. 1, 586–591 (1993)

    Google Scholar 

  37. Altahhan, A.: Navigating a robot through big visual sensory data. Procedia Comput. Sci. 53, 478–485 (2015)

    Article  Google Scholar 

  38. Aldebaran.: Who is NAO? https://www.aldebaran.com/en/cool-robots/nao

Download references

Author information

Authors and Affiliations

  1. Faculty of Engineering, Environment and Computing, School of Computing, Electronics and Mathematics, Coventry University, Priory Street, Coventry, CV1 5FB, UK

    Ariel Ruiz-Garcia, Mark Elshaw, Abdulrahman Altahhan & Vasile Palade

Authors
  1. Ariel Ruiz-Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mark Elshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Abdulrahman Altahhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vasile Palade
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ariel Ruiz-Garcia .

Editor information

Editors and Affiliations

  1. Robert Gordon University, Aberdeen, United Kingdom

    Chrisina Jayne

  2. Lab of Forest Informatics (FiLAB), Democritus University of Thrace Lab of Forest Informatics (FiLAB), Orestiada, Greece

    Lazaros Iliadis

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Ruiz-Garcia, A., Elshaw, M., Altahhan, A., Palade, V. (2016). Emotion Recognition Using Facial Expression Images for a Robotic Companion. In: Jayne, C., Iliadis, L. (eds) Engineering Applications of Neural Networks. EANN 2016. Communications in Computer and Information Science, vol 629. Springer, Cham. https://doi.org/10.1007/978-3-319-44188-7_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-44188-7_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44187-0

  • Online ISBN: 978-3-319-44188-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation