Skip to main content
SpringerLink
Log in
Book cover

International Conference on Ubiquitous Computing and Ambient Intelligence

UCAmI 2016: Ubiquitous Computing and Ambient Intelligence pp 341–352Cite as

Sensing Affective States Using Facial Expression Analysis

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10069))

Abstract

An important factor for the next generation of Human Computer Interaction is the implementation of an interaction model that automatically reasons in context of the users goals, attitudes, affective characteristics and capabilities, and adapts the system accordingly. Although various techniques have been proposed for automatically detecting affective states using facial expression, this is still a research challenge in terms of classification accuracy. This paper investigates an extensible automatic affective state detection approach via the analysis of facial expressions from digital photographs. The main contribution of this study can be summarised in two points. Firstly, utilising facial point distance vectors within the representation of facial expressions is shown to be more accurate and robust in comparison to using standard Cartesian coordinates. Secondly, employing a two-stage Support Vector Machine-based classification model, entitled Hierarchical Parallelised Binary Support Vector Machines (HPBSVM), is shown to improve classification performance over other machine learning techniques. The resulting classification model has been evaluated using two different facial expression datasets (namely CKPLUS and KDEF), yielding accuracy rates of 96.9 % and 96.2 % over each dataset respectively.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Afzal, S., Sezgin, T.M., Gao, Y., Robinson, P.: Perception of emotional expressions in different representations using facial feature points. In: Proceedings - 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009 (2009)

    Google Scholar 

  2. Ahn, H.I., Picard, R.W.: Measuring affective-cognitive experience and predicting market success. IEEE Trans. Affect. Comput. 5(2), 173–186 (2014)

    Article  Google Scholar 

  3. Çınar Akakın, H., Sankur, B.: Spatiotemporal-boosted DCT features for head and face gesture analysis. In: Salah, A.A., Gevers, T., Sebe, N., Vinciarelli, A. (eds.) HBU 2010. LNCS, vol. 6219, pp. 64–74. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  4. Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: Incremental face alignment in the wild. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1859–1866 (2014)

    Google Scholar 

  5. Busso, C., Deng, Z., Yildirim, S., Bulut, M.: Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th International Conference on Multimodal interfaces, pp. 205–211 (2004)

    Google Scholar 

  6. Chang, C.C., Lin, C.J.: LIBSVM - A Library for Support Vector Machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm/

  7. Den Uyl, M.J., Van Kuilenburg, H.: The FaceReader: online facial expression recognition. Proc. Measuring Behav. 30, 589–590 (2005)

    Google Scholar 

  8. Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17(2), 124–129 (1971)

    Article  Google Scholar 

  9. Ekman, P.: What scientists who study emotion agree about. Perspect. Psychol. Sci. 11(1), 31–34 (2016)

    Article  Google Scholar 

  10. Ekman, P., Friesen, W.V., Hager, J.C.: Facial Action Coding System - Investigator’s Guide (2002)

    Google Scholar 

  11. EL-Manzalawy, Y.: WLSVM (2005), http://www.cs.iastate.edu/~yasser/wlsvm/

  12. Fragopanagos, N., Taylor, J.G.: Emotion recognition in human-computer interaction. Neural Netw. Official J. Int. Neural Netw. Soc. 18(4), 389–405 (2005)

    Article  Google Scholar 

  13. Ghimire, D., Lee, J.: Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors (Basel, Switzerland) 13, 7714–7734 (2013)

    Article  Google Scholar 

  14. Jiang, B., Valstar, M.F., Pantic, M.: Action unit detection using sparse appearance descriptors in space-time video volumes. In: 2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011, pp. 314–321 (2011)

    Google Scholar 

  15. Joho, H., Jose, J.M., Valenti, R., Sebe, N.: Exploiting facial expressions for affective video summarisation. In: CIVR 2009, p. 1 (2009)

    Google Scholar 

  16. Kapoor, A., Burleson, W., Picard, R.W.: Automatic prediction of frustration. Int. J. Hum. Comput. Stud. 65(8), 724–736 (2007)

    Article  Google Scholar 

  17. Koller, D., Sahami, M.: Hierarchically classifying documents using very few words. In: Proceedings of the Fourteenth International Conference on Machine Learning, pp. 170–178 (1997)

    Google Scholar 

  18. Landwehr, N., Hall, M., Frank, E.: Logistic model trees. Mach. Learn. 59(1–2), 161–205 (2005)

    Article  MATH  Google Scholar 

  19. Liew, C.F., Yairi, T.: Facial expression recognition and analysis: a comparison study of feature descriptors. IPSJ Trans. Comput. Vis. Appl. 7, 104–120 (2015)

    Article  Google Scholar 

  20. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, CVPRW 2010, pp. 94–101 (2010)

    Google Scholar 

  21. Lundqvist, D., Flykt, A., Ohman, A.: The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, pp. 91–630 (1998)

    Google Scholar 

  22. Martinez, A.M.: Deciphering the face. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (2011)

    Google Scholar 

  23. McDuff, D., Gontarek, S., Picard, R.: Remote measurement of cognitive stress via heart rate variability. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 2957–2960. IEEE (2014)

    Google Scholar 

  24. Oliver, N.: LAFTER: a real-time face and lips tracker with facial expression recognition. In: CVPR, vol. 33, pp. 1369–1382 (1997)

    Google Scholar 

  25. Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)

    Google Scholar 

  26. Platt, J.C.: Fast training of support vector machines using sequential minimal optimization. In: Advances in Kernel Methods, pp. 185–208 (1998)

    Google Scholar 

  27. Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17, 715–734 (2005)

    Article  Google Scholar 

  28. Salah, a., Sebe, N., Gevers, T.: Communication and automatic interpretation of affect from facial expressions. Affective Computing and Interaction, pp. 157–183 (2009)

    Google Scholar 

  29. Shan, C., Gong, S., McOwan, P.W.: Beyond facial expressions: learning human emotion from body gestures. In: Procedings of the British Machine Vision Conference 2007, pp. 43.1-43.10 (2007)

    Google Scholar 

  30. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on Local Binary Patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  31. Soleymani, M., Pantic, M., Pun, T.: Multimodal emotion recognition in response to videos. IEEE Trans. Affect. Comput. 3, 211–223 (2012)

    Article  Google Scholar 

  32. Sullivan, J.W., Tyler, S.W.: Intelligent User Interfaces. ACM Press Series. ACM Press, New York (1991)

    MATH  Google Scholar 

  33. Vega, K., Arrieta, A., Esteves, F., Fuks, H.: FX e-Makeup for muscle based interaction. In: Marcus, A. (ed.) DUXU 2014. LNCS, vol. 8519, pp. 643–652. Springer, Heidelberg (2014). doi:10.1007/978-3-319-07635-5_61

    Chapter  Google Scholar 

  34. Whitehill, J., Bartlett, M., Movellan, J.: Automatic facial expression recognition for intelligent tutoring systems. In: 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 1–6 (2008)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Mathematics, Ulster University, Belfast, BT37 0QB, UK

    Anas Samara, Leo Galway, Raymond Bond & Hui Wang

Authors
  1. Anas Samara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leo Galway
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raymond Bond
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anas Samara .

Editor information

Editors and Affiliations

  1. University of Las Palmas de Gran Canaria, Las Palmas, Spain

    Carmelo R. García

  2. Departamento de Estadistica, Universidad La Laguna, La Laguna, Spain

    Pino Caballero-Gil

  3. Florida State University, Tallahassee, Florida, USA

    Mike Burmester

  4. University of Las Palmas de Gran Canaria, Las Palmas, Spain

    Alexis Quesada-Arencibia

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Samara, A., Galway, L., Bond, R., Wang, H. (2016). Sensing Affective States Using Facial Expression Analysis. In: García, C., Caballero-Gil, P., Burmester, M., Quesada-Arencibia, A. (eds) Ubiquitous Computing and Ambient Intelligence. UCAmI 2016. Lecture Notes in Computer Science(), vol 10069. Springer, Cham. https://doi.org/10.1007/978-3-319-48746-5_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48746-5_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48745-8

  • Online ISBN: 978-3-319-48746-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation