Skip to main content
SpringerLink
Log in

An experimental design for facial and color emotion expression of a social robot

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In this paper, we investigate the relationship between emotions and colors by showing robot animated emotion faces and colors to the participants through a series of surveys. We focused on representing a visualized emotion through a robot's facial expression and background colors. To complete the emotion design with animated faces and color background, we gave an experimental design for surveying the users' thoughts. We took an example of a robot animated face by using the ASUS Zenbo. We selected 11 colors as our color background and 24 facial expressions from Zenbo. To analyze our results from questionnaires, we used histograms to show the basic data situation and the multiple logistic regression analysis (MLRA) to see the marginal relationships. We separated our questionnaires into positive and negative questionnaires and divided the dataset into three cases to discuss the different relationships between color and emotion. Results showed that people preferred the blue color no matter whether the face was showing positive or negative emotion. The MLRA also showed the correct percentage is outstanding in case 2, either positive emotion or negative emotion. Participants thought Zenbo's robotic animated face was the same as they thought. Through our experimental design, we hope that people can consider more colors with emotion to design the human–robot interface that will be closer to the users' thoughts and make life more colorful with comfortable reactions with robots.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Mori M, MacDorman KF, Kageki N (2012) The uncanny valley. Robot Autom Mag IEEE 19(2):98–100

    Article  Google Scholar 

  2. Herse S, Vitale J, Tonkin M, Ebrahimian D, Ojha S, Johnston B, Judge W, and Williams M (2018). Do you trust me, blindly? Factors influencing trust to-wards a robot recommender system, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China, 7–14.

  3. Kanda T, Ishiguro H, Imai M, Ono T (2004) Development and evaluation of interactive humanoid robots. Proc IEEE 92(11):1839–1850

    Article  Google Scholar 

  4. Bartneck C, Belpaeme T, Eyssel F, Kanda T, Keijsers M, Sabanovic S (2020) Human-robot interaction – an introduction. Cambridge University Press, Cambridge

    Book  Google Scholar 

  5. Lin P-C, Mettrick D, Hung PCK, and Iqbal F (2018). Towards a music visualization on robot (MVR) prototype, IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR2018), 256–257, Taichung, Taiwan, December 10–12

  6. Lin P-C, Mettrick D, Hung PCK, Iqbal F (2019) Robot computing for music visualization, theory and applications of models of computation. TAMC 2019. Lect Notes Comput Sci 11436:438–447

    Article  MATH  Google Scholar 

  7. Lin, P-C, Hung PCK (2021) A case study of facial expression and color for social robot’s emotion expression, The 23rd International conference on artificial intelligence, (ICAI'21) in the 2021 World Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE'21), Virtual and live at Luxor (MGM), Las Vegas, USA, July 26–29

  8. Schütte STW, Eklund J, Axelsson JRC, Nagamachi M (2004) Concepts, methods and tools in Kansei engineering. Theor Issues Ergon Sci Taylor Francis 5(3):214–231

    Article  Google Scholar 

  9. Park C, Ryu J, Sohn J, and Cho H (2007) An Emotion Expression System for the Emotional Robot, 2007 IEEE International Symposium on Consumer Electronics, Irving, Texas, USA, 1–6

  10. Kwon D et al. (2007). Emotion Interaction System for a Service Robot, the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2007), Jeju, Korea, 351–356

  11. Takeno J, Mori K, Naito Y (2009) A robot with complex facial expressions. Sens Transducers Spec Issue 6:18–26

    Google Scholar 

  12. Saldien J, Goris K, Vanderborght B, Vanderfaeillie J, Lefeber D (2010) Expressing emotions with the social robot probo. Int J Soc Robot 2(4):377–389

    Article  Google Scholar 

  13. Cohen I, Looije R, and Neerincx MA (2011) Child's recognition of emotions in robot's face and body, the 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Lausanne, 123–124

  14. Read R and Belpaeme T (2012) How to use non-linguistic utterances to convey emotion in child-robot interaction, the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boston, MA, USA, 219–220

  15. Kędzierski J, Muszyński R, Zoll C, Oleksy A, Frontkiewicz M (2013) EMYS—Emotive head of a social robot. Int J Soc Robot 5(2):237–249

    Article  Google Scholar 

  16. Mirnig N, Strasser E, Weiss A, Kühnlenz B, Wollherr D, Tscheligi M (2014) Can you read my face?: a methodological variation for assessing facial expressions of robotic heads. Int J Soc Robot 7(1):63–76

    Article  Google Scholar 

  17. Ali F, Jun GW (2015). Design of robot head for expression of human emotion. applied mechanics and materials, 773–774, 893–897. Shih W, Naruse K, and Wu S (2017). Implement human-robot interaction via robot's emotion model, the IEEE 8th International Conference on Awareness Science and Technology (iCAST), Taichung, Taiwan, 580–585

  18. Shih W, Naruse K, Wu S (2017) Implement human-robot interaction via robot's emotion model, the IEEE 8th International Conference on Awareness Science and Technology (iCAST), Taichung, Taiwan, 580–585

  19. Reuten A, van Dam M, Naber M (2018) Pupillary responses to robotic and human emotions: the uncanny valley and media equation confirmed. Front Psychol 9:774–774

    Article  Google Scholar 

  20. Chew E, Chua XN (2020) Robotic Chinese language tutor: personalising progress assessment and feedback or taking over your job? Horizon 28(3):113–124

    Article  Google Scholar 

  21. Kita S, Mita A (2015) Emotion identification method using RGB information of human face, In: Proceedings of SPIE—The International Society for Optical Engineering, vol 9435, p 6

  22. Tielman M, Neerincx M, Meyer JJ, Looije R (2014) Adaptive emotional expression in robot-child interaction, In: the ACM/IEEE International Conference on Human-Robot Interaction, p 407–414

  23. Arora P Chaspari T (2018) Exploring Siamese neural network architectures for preserving speaker identity in speech emotion classification, In: Proceedings of the 4th Workshop on Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction, MA3HMI 2018 - In conjunction with ICMI 2018, p 15–18

  24. Latif S, Khalifa S, Rana R, Jurdak R (2020) Federated learning for speech emotion recognition applications, In: Proceedings - 2020 19th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2020, art. no. 9111050, p 341–342

  25. Hsieh WF, Sato-Shimokawara E, Yamaguchi T (2020) Investigation of robot expression style in human-robot interaction. J Robot Mechatron 32(1):224–235

    Article  Google Scholar 

  26. Chuah SHW, Yu J (2021) The future of service: the power of emotion in human-robot interaction. J Retail Consum Serv 61:102551

    Article  Google Scholar 

  27. Kaya N, Epps HH (2004) Relationship between color and emotion: a study of college students. Coll Stud J 38(3):396–405

    Google Scholar 

  28. Hanada. (2018) Correspondence analysis of color–emotion associations. Color Res Appl 43(2):224–237

    Article  Google Scholar 

  29. Demir. (2020) Investigation of color-emotion associations of the university students. Color Res Appl 45(5):871–884

    Article  MathSciNet  Google Scholar 

  30. Takei A, Imaizumi S (2022) Effects of color–emotion association on facial expression judgments. Heliyon 8(1):e08804–e08804

    Article  Google Scholar 

  31. Terada K, Yamauchi A, Ito A (2012). Artificial emotion expression for a robot by dynamic color change. 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, 314–321

  32. Li J, Chen L, Pan D, Wang F, Yang Y (2019) Red or blue? Effects of background color in distraction tasks on global processing in unconscious thought. Curr Psychol (New Brunswick, NJ) 40(5):2153–2160

    Article  Google Scholar 

  33. Wang T, Shu S, Mo L (2014) Blue or red? The effects of colour on the emotions of Chinese people. Asian J Soc Psychol 17(2):152–158

    Article  Google Scholar 

  34. Bekhtereva V, Müller MM (2017) Bringing color to emotion: the influence of color on attentional bias to briefly presented emotional images. Cognit Affect Behav Neurosci 17(5):1028–1047

    Article  Google Scholar 

  35. Wilms, & Oberfeld, D. (2017) Color and emotion: effects of hue, saturation, and brightness. Psychol Res 82(5):896–914

    Article  Google Scholar 

  36. Wan J, Zhou Y, Li Y, Su Y, Cao Y, Zhang L, Ying L, Deng W (2020) Research on color space perceptions and restorative effects of blue space based on color psychology: examination of the Yijie District of Dujiangyan City as an Example. Int J Environ Res Public Health 17(9):3137

    Article  Google Scholar 

  37. Johnson DO, Cuijpers RH, van der Pol D (2013) Imitating human emotions with artificial facial expressions. Int J Soc Robot 5(4):503–513

    Article  Google Scholar 

  38. Sixseconds (2020). Plutchik's wheel of emotions: exploring the emotion wheel. Online: https://www.6seconds.org/2017/04/27/plutchiks-model-of-emotions/

  39. Birren F (2013) Color Psychology and color therapy: a factual study of the influence of color on human life. Martino Fine Books, USA

    Google Scholar 

  40. Madden TJ, Hewett K, Roth MS (2000) Managing images in different cultures: a cross-national study of color meanings and preferences. J Int Market (East Lansing, Mich) 8(4):90–107

    Article  Google Scholar 

  41. Affects C (2008) Psychological properties of colours. Online: http://www.colour-affects.co.uk/psychological-properties-of-colours

  42. Nezlek JB, Kuppens P (2008) Regulating positive and negative emotions in daily. Life, J Person 76(3):561–580

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the Ministry of Education, R.O.C., under the grant TEEP@AsiaPlus. The work in this paper was also supported by the Ministry of Science and Technology under Grant No. MOST 109-2221-E-035-063-MY2. The data that support the findings of this study are not openly available due to the research topic still continuous with a MOST funding project and are available from the corresponding author P.-C. Lin upon reasonable request. The authors declare that there is no conflict of interest.

Author information

Authors and Affiliations

  1. Faculty of Information Engineering and Computer Science, Feng Chia University, Taichung, Taiwan

    Pei-Chun Lin

  2. Faculty of Business and Information Technology, Ontario Tech University, Oshawa, Canada

    Patrick C. K. Hung, Ying Jiang, Carolina Padilla Velasco & Marco Antonio Martínez Cano

Authors
  1. Pei-Chun Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick C. K. Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carolina Padilla Velasco
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco Antonio Martínez Cano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pei-Chun Lin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, PC., Hung, P.C.K., Jiang, Y. et al. An experimental design for facial and color emotion expression of a social robot. J Supercomput 79, 1980–2009 (2023). https://doi.org/10.1007/s11227-022-04734-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04734-7

Keywords

Search

Navigation