Skip to main content
Springer Nature Link
Log in

Spontaneous Human-Robot Emotional Interaction Through Facial Expressions

  • Conference paper
  • First Online:
Social Robotics (ICSR 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9979))

Included in the following conference series:

Abstract

One of the main issues in the field of social and cognitive robotics is the robot’s ability to recognize emotional states and emotional interaction between robots and humans. Through effective emotional interaction, robots will be able to perform many tasks in human society. In this research, we have developed a robotic platform and a vision system to recognize the emotional state of the user through its facial expressions, which leads to a more realistic human-robot interaction (HRI). First, a number of features are extracted according to points detected by a vision system from the face of the user. Then, the emotional state of the user is analyzed with the help of these features. For the decision making unit, a state machine is designed that utilizes the results obtained from the emotional state analysis to generate the robot’s response. Finally, a fuzzy algorithm is used to improve the quality of emotional interaction and the results are implemented on a commercial humanoid robot platform which has the ability of producing facial expressions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Tadesse, Y., Hong, D., Priya, S.: Twelve degree of freedom baby humanoid head using shape memory alloy actuators. J. Mech. Robot. 3, 211–226 (2011)

    Article  Google Scholar 

  2. Saffari, E., Meghdari, A., Vazirnezhad, B., Alemi, M.: Ava (A Social Robot): design and performance of a robotic hearing apparatus. In: Tapus, A., André, E., Martin, J.-C., Ferland, F., Ammi, M. (eds.) ICSR 2015. LNCS, vol. 9388, pp. 440–450. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25554-5_44

    Chapter  Google Scholar 

  3. Hanumara, N.C., Slocum, A.H., Mitamura, T.: Design of a spherically actuated human interaction robot head. J. Mech. Design 134(5), 055001 (2012)

    Article  Google Scholar 

  4. Asfour, T., Welke, K., Azad, P., Ude, A., Dillmann, R.: The Karlsruhe humanoid head. In: 8th IEEE-RAS International Conference on Humanoid Robots, pp. 447–453, December 2008

    Google Scholar 

  5. Meghdari, A., Alemi, M., Ghazisaedy, M., Taheri, A.R., Karimian, A., Zandvakili, M.: Applying robots as teaching assistant in EFL classes at Iranian Middle-Schools. In: Proceeding International Conference on Education & Modern Educational Technologies (EMET-2013), 28–30 September 2013, Venice, Italy (2013)

    Google Scholar 

  6. Alemi, M., Meghdari, A., Ghazisaedy, M.: The impact of social robotics on L2 Learners’ anxiety and attitude in English vocabulary acquisition. Int. J. Soc. Robot. 7(4), 523–535 (2015)

    Article  Google Scholar 

  7. Alemi, M., Ghanbarzadeh, A., Meghdari, A., Moghaddam, L.J.: Clinical application of a humanoid robot in pediatric cancer interventions. Int. J. Soc. Robot. (2015)

    Google Scholar 

  8. Trinh, T.Q., Schroeter, C., Kessler, J., Gross, H.-M.: “Go Ahead, Please”: recognition and resolution of conflict situations in narrow passages for polite mobile robot navigation. In: Tapus, A., André, E., Martin, J.-C., Ferland, F., Ammi, M. (eds.) ICSR 2015. LNCS, vol. 9388, pp. 643–653. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25554-5_64

    Chapter  Google Scholar 

  9. Taheri, A.R., Alemi, M., Meghdari, A., PourEtemad, H.R., Basiri, N.M.: Social robots as assistants for autism therapy in Iran: research in progress. In: Second RSI/ISM International Conference Robotics and Mechatronics (ICRoM), pp. 760–766. IEEE (2014)

    Google Scholar 

  10. McColl, D., Nejat, G.: A socially assistive robot that can monitor affect of the elderly during mealtime assistance. J. Med. Dev. 8(3), 030941 (2014)

    Article  Google Scholar 

  11. Zacharatos, H., Gatzoulis, C., Chrysanthou, Y.L.: Automatic emotion recognition based on body movement analysis: a survey. IEEE Comput. Graph. Appl. 34(6), 35–45 (2014)

    Article  Google Scholar 

  12. Halder, A., Konar, A., Mandal, R., Chakraborty, A., Bhowmik, P., Pal, N.R., Nagar, A.K.: General and interval type-2 fuzzy face-space approach to emotion recognition. IEEE Trans. Syst. Man Cybern. Syst. 43(3), 587–605 (2013)

    Article  Google Scholar 

  13. Mavridis, N.: A review of verbal and non-verbal human-robot interactive communication. Robot. Auton. Syst. 63, 22–35 (2014)

    Article  Google Scholar 

  14. Aly, A., Tapus, A.: Multimodal adapted robot behavior synthesis within a narrative human-robot interaction. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2986–2993 (2015)

    Google Scholar 

  15. Yashaswi Alva, M., Nachamai, M., Paulose, J.: A comprehensive survey on features and methods for speech emotion detection. In: IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT) (2015)

    Google Scholar 

  16. Xiao, Y., Zhang, Z., Beck, A., Yuan, J., Thalmann, D.: Human–robot interaction by understanding upper body gestures. Presence 23(2), 133–154 (2014)

    Article  Google Scholar 

  17. Chakraborty, A., Konar, A., Chakraborty, U.K., Chatterjee, A.: Emotion recognition from facial expressions and its control using fuzzy logic. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 39(4), 726–743 (2009)

    Article  Google Scholar 

  18. Kotsia, I., Pitas, I.: Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Trans. Image Process. 16(1), 172–187 (2007)

    Article  MathSciNet  Google Scholar 

  19. Dahmane, M., Meunier, J.: Prototype-based modeling for facial expression analysis. IEEE Trans. Multimedia 16(6), 1574–1584 (2014)

    Article  Google Scholar 

  20. Li, Y., Mavadati, S.M., Mahoor, M.H., Zhao, Y., Ji, Q.: Measuring the intensity of spontaneous facial action units with dynamic Bayesian network. Pattern Recogn. 48(11), 3417–3427 (2015)

    Article  Google Scholar 

  21. Li, Y., Wang, S., Zhao, Y., Ji, Q.: Simultaneous facial feature tracking and facial expression recognition. IEEE Trans. Image Process. 22(7), 2559–2573 (2013)

    Article  Google Scholar 

  22. http://www.robokindrobots.com/

  23. Kinect for Windows SDK (2016). https://msdn.microsoft.com/en-us/library/

  24. Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting psychologists press, Palo alto (1978)

    Google Scholar 

  25. Popescu, M., Keller, J., Bezdek, J.C., Zare, A.: Random projections fuzzy c-means (RPFCM) for big data clustering. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), August 2015

    Google Scholar 

  26. Yan, J., Ryan, M., Power, J.: Using Fuzzy Logic: Towards Intelligent Systems, vol. 1. Prentice Hall, London (1994)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Social and Cognitive Robotics Laboratory, Center of Excellence in Design, Robotics and Automation (CEDRA), Sharif University of Technology, Tehran, Iran

    Ali Meghdari, Minoo Alemi, Ali Ghorbandaei Pour & Alireza Taheri

  2. Islamic Azad University, Tehran-West Branch, Tehran, Iran

    Minoo Alemi

Authors
  1. Ali Meghdari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Minoo Alemi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ali Ghorbandaei Pour
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alireza Taheri
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Meghdari .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, The University of Kansas, Lawrence, Indiana, USA

    Arvin Agah

  2. Department of Mechanical Engineering, Qatar University, Doha, Qatar

    John-John Cabibihan

  3. School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, USA

    Ayanna M. Howard

  4. Department of Systems Engineering and Automation, University Carlos III de Madrid, madrid, Spain

    Miguel A. Salichs

  5. Department of Mechanical, Aerospace and Biomedical Engineering, University of Tennessee, Knoxville, Tennessee, USA

    Hongsheng He

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Meghdari, A., Alemi, M., Pour, A.G., Taheri, A. (2016). Spontaneous Human-Robot Emotional Interaction Through Facial Expressions. In: Agah, A., Cabibihan, JJ., Howard, A., Salichs, M., He, H. (eds) Social Robotics. ICSR 2016. Lecture Notes in Computer Science(), vol 9979. Springer, Cham. https://doi.org/10.1007/978-3-319-47437-3_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47437-3_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47436-6

  • Online ISBN: 978-3-319-47437-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics