Skip to main content
SpringerLink
Log in

Real and fake emotion detection using enhanced boosted support vector machine algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Differentiating real and fake emotions becomes a new challenge in facial expression recognition and emotion detection. Real and fake emotions should be taken into account when developing an application. Otherwise, a fake emotion can be categorized as real emotion thereby rendering the model as futile. Very limited research has dealt with identifying fake emotions with accuracy as results are in a range of 51–76%. Performance of the available methods in detecting fake emotions is not encouraging. Thus, in this paper, we have proposed Enhanced Boosted Support Vector Machine (EBSVM) algorithm. EBSVM is a novel technique to determine important thresholds required to understand fake emotions. We have created a new dataset named FED comprising both real and fake emotion images of 50 subjects and used them with experiments along with SASE-FE. EBSVM considers the entire data for classification at each iteration using the ensemble classifier. The EBSVM algorithm achieved 98.08% as classification accuracy for different K-fold validations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Asthana A, Zafeiriou S, Cheng S, Pantic M (2014) Incremental face alignment in the wild. In proceedings of the IEEE conference on computer vision and pattern recognition(pp. 1859-1866)

  2. Bartlett MS, Littlewort GC, Frank MG, Lee K (2014) Automatic decoding of facial movements reveals deceptive pain expressions. Curr Biol 24(7):738–743

    Article  Google Scholar 

  3. Besold TR (2016) On cognitive aspects of human-level artificial intelligence. KI-Künstliche Intelligenz 30(3–4):343–346

    Article  Google Scholar 

  4. Besold TR (2018) "Pre-Proceedings of the Cognitive Computation Symposium: Thinking Beyond Deep Learning (CoCoSym 2018): Extended Abstracts/Speakers' Positions."

  5. Besold TR, Uckelman SL (2018) "The What, the Why, and the How of Artificial Explanations in Automated Decision-Making." arXiv preprint arXiv:1808.07074

  6. Besold T, Hernández-Orallo J, Schmid U (2015) Can machine intelligence be measured in the same way as human intelligence? KI-Künstliche Intelligenz 29(3):291–297

    Article  Google Scholar 

  7. Bhakt N, Joshi P, Dhyani P (2018) A novel framework for real and fake smile detection from videos. In 2018 second international conference on electronics, communication and aerospace technology (ICECA) (pp. 1327-1330). IEEE

  8. BVLC (2021) “Deeplerning model”. https://github.com/BVLC/caffe/tree/master/models/bvlc_“deep learning model”

  9. Calvo MG, Fernández-Martín A (2013) Can the eyes reveal a person’s emotions? Biasing role of the mouth expression. Motiv Emot 37(1):202–211

    Article  Google Scholar 

  10. Confalonieri R, Pease A, Schorlemmer M, Besold TR, Kutz O, Maclean E, Kaliakatsos-Papakostas M, eds. (2018) Concept invention: Foundations, implementation, social aspects and applications. Springer

  11. Confalonieri R, del Prado FM, Agramunt S, Malagarriga D, Faggion D, Weyde T, Besold TR (2019a) An ontology-based approach to explaining artificial neural networks. arXiv preprint arXiv:1906.08362

  12. Confalonieri R, Besold TR, Weyde T, Creel K, Lombrozo T, Mueller S, Shafto P (2019b) "What makes a good explanation? Cognitive dimensions of explaining intelligent machines." CogSci 2019: Creativity+ Cognition+ Computation

  13. Cote S, Hideg I, Van Kleef GA (2013) The consequences of faking anger in negotiations. J Exp Soc Psychol 49(3):453–463

    Article  Google Scholar 

  14. Dawel A, Wright L, Irons J, Dumbleton R, Palermo R, O’Kearney R, McKone E (2017) Perceived emotion genuineness: normative ratings for popular facial expression stimuli and the development of perceived-as-genuine and perceived-as-fake sets. Behav Res Methods 49(4):1539–1562

    Article  Google Scholar 

  15. Dibeklioglu H, Salah AA, Gevers T (2015) Recognition of genuine smiles. IEEE Transactions on Multimedia 17(3):279–294

    Article  Google Scholar 

  16. Doran D, Schulz S, Besold TR (2017) "What does explainable AI really mean? A new conceptualization of perspectives." arXiv preprint arXiv:1710.00794

  17. Duchenne GB, de Boulogne GBD (1990) The mechanism of human facial expression. Cambridge university press

  18. Ekman R (1997) What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS). Oxford University Press, USA

    Google Scholar 

  19. Ekman P (2004) Emotions revealed. BMJ 328(Suppl S5):0405184

    Article  Google Scholar 

  20. Ghayoumi M, Bansal AK (2014) An integrated approach for efficient analysis of facial expressions. In 2014 international conference on signal processing and multimedia applications (SIGMAP) (pp. 211-219). IEEE

  21. Ghayoumi M, Bansal AK (2016) Real emotion recognition by detecting symmetry patterns with dihedral group. In 2016 third international conference on mathematics and computers in sciences and in industry (MCSI) (pp. 178-184). IEEE

  22. Gunadi IGA, Harjoko A, Wardoyo R, Ramdhani N (2015) Fake smile detection using linear support vector machine. In 2015 international conference on data and software engineering (ICoDSE) (pp. 103-107). IEEE

  23. Huynh XP, Kim YG (2017) Discrimination between genuine versus fake emotion using long-short term memory with parametric bias and facial landmarks. In proceedings of the IEEE international conference on computer vision (pp. 3065-3072)

  24. Karat J (2002) Beyond task completion: evaluation of affective components of use. In the human-computer interaction handbook (pp. 1152-1164). L. Erlbaum associates Inc

  25. Krizhevsky A, Sutskever I, Hinton GE (2012) "ImageNet Classification with Deep Convolutional Neural Networks." Advances in neural information processing systems

  26. Kulkarni K, Corneanu C, Ofodile I, Escalera S, Baro X, Hyniewska S, Anbarjafari G (2018) Automatic recognition of facial displays of unfelt emotions. IEEE transactions on affective computing

  27. Kumar GR, Kumar RK, Sanyal G (2017) Discriminating real from fake smile using convolution neural network. In 2017 international conference on computational intelligence in data science (ICCIDS) (pp. 1-6). IEEE

  28. Leng L et al (2010) Dynamic weighted discrimination power analysis: a novel approach for face and palmprint recognition in DCT domain. International journal of physical sciences 5(17):2543–2554

    Google Scholar 

  29. Leng L, Li M, Teoh ABJ (2013) "Conjugate 2DPalmHash code for secure palm-print-vein verification." 2013 6th International congress on image and signal processing (CISP). Vol. 3. IEEE

  30. Leng L et al (2017) Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed Tools Appl 76(1):333–354

    Article  Google Scholar 

  31. Li L, Baltrusaitis T, Sun B, Morency LP (2017) Combining sequential geometry and texture features for distinguishing genuine and deceptive emotions. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3147–3153)

  32. Littlewort GC, Bartlett MS, Lee K (2009) Automatic coding of facial expressions displayed during posed and genuine pain. Image Vis Comput 27(12):1797–1803

    Article  Google Scholar 

  33. Lopatovska I, Arapakis I (2011) Theories, methods and current research on emotions in library and information science, information retrieval and human–computer interaction. Inf Process Manag 47(4):575–592

    Article  Google Scholar 

  34. Lüsi I, Junior JCJ, Gorbova J, Baró X, Escalera S, Demirel H, Allik J, Ozcinar C, Anbarjafari G (2017) Joint challenge on dominant and complementary emotion recognition using micro emotion features and head-pose estimation: Databases. In: Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference. IEEE, pp 809–813

  35. Matlab (2020) Classification learner toolbox: https://in.mathworks.com/help/stats/examples.html?category=getting-started-12&s_tid=CRUX_topnav

  36. Matlab (n.d.-a) transfer learning toolbox: https://in.mathworks.com/help/deeplearning/ug/transfer-learning-using-alexnet.html

  37. Matlab (n.d.-c) deep learning toolbox: https://in.mathworks.com/help/deeplearning/index.html

  38. MATLAB and Statistics Toolbox Release (2020b) The MathWorks, Inc., Natick, Massachusetts

  39. McLellan T, Johnston L, Dalrymple-Alford J, Porter R (2010) Sensitivity to genuine versus posed emotion specified in facial displays. Cognit Emot 24(8):1277–1292

    Article  Google Scholar 

  40. Mehrabian A (1968) Communication without words. Psychol Today 2(4)

  41. Nakano M, Mitsukura Y, Fukumi M, Akamatsu N (2002) True smile recognition system using neural networks. In proceedings of the 9th international conference on neural information processing, 2002. ICONIP'02. (Vol. 2, pp. 650-654). IEEE

  42. Picard RW (2001) Building HAL: computers that sense, recognize, and respond to human emotion. In human vision and electronic imaging VI (Vol. 4299, pp. 518-523). International Society for Optics and Photonics

  43. Russell JA, Bachorowski JA, Fernández-Dols JM (2003) Facial and vocal expressions of emotion. Annu Rev Psychol 54(1):329–349

    Article  Google Scholar 

  44. Sander D, Grandjean D, Scherer KR (2005) A systems approach to appraisal mechanisms in emotion. Neural Netw 18(4):317–352

    Article  Google Scholar 

  45. Saxen F, Werner P, Al-Hamadi A (2017). Real vs. fake emotion challenge: learning to rank authenticity from facial activity descriptors. In proceedings of the IEEE international conference on computer vision (pp. 3073-3078)

  46. Adolphs R, Tranel D, Damasio H, Damasio A (1994) Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala. Nature 372(6507):669–672

  47. Scherer KR (2003) Vocal communication of emotion: a review of research paradigms. Speech Comm 40(1–2):227–256

    Article  MATH  Google Scholar 

  48. Sidera F, Amadó A, Serrat E (2013) Are you really happy? Children’s understanding of real vs pretend emotions Current Psychology 32(1):18–31

  49. Song R, Over H, Carpenter M (2016) Young children discriminate genuine from fake smiles and expect people displaying genuine smiles to be more prosocial. Evol Hum Behav 37(6):490–501

    Article  Google Scholar 

  50. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) "Going deeper with convolutions." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9

  51. Taxer JL, Frenzel AC (2015) Facets of teachers' emotional lives: a quantitative investigation of teachers' genuine, faked, and hidden emotions. Teach Teach Educ 49:78–88

    Article  Google Scholar 

  52. Tian YI, Kanade T, Cohn JF (2001) Recognizing action units for facial expression analysis. IEEE Trans Pattern Anal Mach Intell 23(2):97–115

    Article  Google Scholar 

  53. Ugail H, Al-dahoud A (2019) A genuine smile is indeed in the eyes–the computer aided non-invasive analysis of the exact weight distribution of human smiles across the face. Adv Eng Inform 42:100967

    Article  Google Scholar 

  54. Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154

    Article  Google Scholar 

  55. Wan J, Escalera S, Anbarjafari G, Jair Escalante H, Baró X, Guyon I, ..., Xie Y (2017) Results and analysis of chalearn lap multi-modal isolated and continuous gesture recognition, and real versus fake expressed emotions challenges. In Proceedings of the IEEE International Conference on Computer Vision (pp. 3189–3197)

  56. Wang BX, Japkowicz N (2010) Boosting support vector machines for imbalanced data sets. Knowl Inf Syst 25(1):1–20

    Article  Google Scholar 

  57. Zaadnoordijk L, Besold TR (2019) "Artificial phenomenology for human-level artificial intelligence." In AAAI Spring Symposium: Towards Conscious AI Systems

  58. Zhang Z, Singh V, Slowe TE, Tulyakov S, Govindaraju V (2007) Real-time automatic deceit detection from involuntary facial expressions. In 2007 IEEE conference on computer vision and pattern recognition (pp. 1-6). IEEE

Download references

Author information

Authors and Affiliations

  1. School of Computer Science & Engineering, Vellore Institute of Technology, Chennai, Tamilnadu, 6200127, India

    Swaminathan Annadurai

  2. Department of Computer Applications, National Institute of Technology, Tiruchirappalli, Tamilnadu, 6200015, India

    Michael Arock

  3. Department of Computer Science & Engineering, GITAM School of Technology, Bengaluru, GITAM University, Bengaluru, Karnataka, 561203, India

    A. Vadivel

Authors
  1. Swaminathan Annadurai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Arock
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. Vadivel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Swaminathan Annadurai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Annadurai, S., Arock, M. & Vadivel, A. Real and fake emotion detection using enhanced boosted support vector machine algorithm. Multimed Tools Appl 82, 1333–1353 (2023). https://doi.org/10.1007/s11042-022-13210-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13210-6

Keywords

Search

Navigation