Skip to main content
Springer Nature Link
Log in

Facial expression recognition in ageing adults: from lab to ambient assisted living

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

This paper introduces an application that uses a webcam and aims to recognise emotions of ageing adults from their facial expression. Six basic emotions (Happiness, Sadness, Anger, Fear, Disgust and Surprise) as well as a Neutral state are distinguished. Active shape models are applied for feature extraction, the Cohn–Kanade, JAFFE and MMI databases are used for training, and support vector machines (\(\nu\)-SVM) are employed for facial expression classification. These six basic emotions are classified into three categories (Happiness, Negative-Emotion and Surprise), as Sadness, Anger, Fear and Disgust have been grouped into a single expression. The new three categories are found to be relevant for the purpose of detecting abnormalities in the ageing adult’s mood state. In the future, we will transfer the experience in a lab/research setup to a real ambient assisted living environment, which is the elderly person’s home. The application is devised to be the starting point to enhance the mood of the elderly people living alone at their homes by external stimuli.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • AT&T Laboratories Cambridge (1994). http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html. Accessed 21 Sept 2016

  • Alugupally N, Samal A, Marx D, Bhatia S (2011) Analysis of landmarks in recognition of face expressions. Pattern Recognit Image Anal 21:681–693

    Article  Google Scholar 

  • Castillo JC, Castro-González Á, Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A, Salichs MA (2016) Software architecture for smart emotion recognition and regulation of the ageing adult. Cognit Comput 8:357–367

    Article  Google Scholar 

  • Castillo JC, Fernández-Caballero A, Castro-González Á, Salichs MA, López MT (2014) A framework for recognizing and regulating emotions in the elderly. Lect Notes Comput Sci 8868:320–327

    Article  Google Scholar 

  • Chan CH (2000) The XM2VTS Database. http://www.ee.surrey.ac.uk/CVSSP/xm2vtsdb/. Accessed 21 Sept 2016

  • Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:1–27

    Article  Google Scholar 

  • Chang Y, Hu C, Feris R, Turk M (2006) Manifold based analysis of facial expression. Image Vis Comput 24:605–614

    Article  Google Scholar 

  • Chittora A, Mishra O (2012) Face recognition using RBF kernel based support vector machine. Int J Future Comput Commun 1:280–283

    Article  Google Scholar 

  • Cootes TF, Cristinacce D, Babalola K (2005) BioID face database. http://www.bioid.com/index.php?q=downloads/software/bioid-face-database.html. Accessed 21 Sept 2016

  • Cootes TF, Taylor CJ, Coper DH, Graham J (1996) Active shape models their training and application. Comput Vis Image Underst 61:38–59

    Article  Google Scholar 

  • Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297

    MATH  Google Scholar 

  • Cruz A, Bhanu B (2012) A biologically inspired approach for fusing facial expression and appearance for emotion recognition. In: Proceedings of the 19th IEEE international conference on image processing, Lake Buena Vista, FL, USA, October 2012, pp 2625–2628

  • Ekman P, Friesen WV, Hager JC (2002) Facial Action Coding System (FACS). http://face-and-emotion.com/dataface/facs/new_version.jsp. Accessed 21 Sept 2016

  • Fernández-Caballero A, Martínez-Rodrigo A, Pastor JM, Castillo JC, Lozano-Monasor E, López MT, Zangróniz R, Latorre JM, Fernández-Sotos A (2016) Smart environment architecture for emotion recognition and regulation. J Biomed Inform 64:55–73

    Article  Google Scholar 

  • Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A (2014) Improvement of the elderly quality of life and care through smart emotion regulation. Lect Notes Comput Sci 8868:348–355

    Article  Google Scholar 

  • Fernández-Caballero A, López MT, Serrano-Cuerda J (2014) Thermal-infrared pedestrian ROI extraction through thermal and motion information fusion. Sensors 14:6666–6676

    Article  Google Scholar 

  • Gang L, Xiao-hua L, Ji-liu Z, Xiao-gang G (2009) Geometric feature based facial expression recognition using multiclass support vector machines. In: Proceedings of the 2009 IEEE international conference on granular computing, Nanchang, China, August 2009, pp 318–321

  • Gascueña JM, Castillo JC, Navarro E, Fernández-Caballero A (2014) Engineering the development of systems for multisensory monitoring and activity interpretation. Int J Syst Sci 45:728–740

    Article  MATH  Google Scholar 

  • Ghimire D, Lee J (2013) Geometric feature-based facial expression recognition in image sequences using multi-class AdaBoost and support vector machines. Sensors 13:7714–7734

    Article  Google Scholar 

  • Hsieh C-C, Jiang M-K (2011) A facial expression classification system based on active shape model and support vector machine. In: Proceedings of the 2011 international symposium on computer science and society, Kota Kinabalu, Malaysia, July 2011, pp 311–314

  • Kanade T, Cohn J, Tian YL (2000) Comprehensive database for facial expression analysis. In: Proceedings of the fourth IEEE international conference on automatic face and gesture recognition, Grenoble, France, March 2000, pp 46–53

  • Lozano-Monasor E, López MT, Fernández-Caballero A, Vigo-Bustos F (2014) Facial expression recognition from webcam based on active shape models and support vector machines. Lect Notes Comput Sci 8868:147–154

    Article  Google Scholar 

  • Lucey P, Cohn JF, de Kana T, Saragih J, Ambadar Z, Matthews I (2010) The extended Cohn–Kanade Dataset (CK+): a complete facial expression dataset for action unit and emotion-specified expression. In: Proceedings of the 2010 IEEE Computer Society conference on computer vision and pattern recognition workshops, San Francisco, CA, USA, June 2010, pp 94–101

  • Lyons MJ, Kamachi M, Gyoba J (1997) Japanese Female Facial Expressions (JAFFE), Database of digital images. http://www.kasrl.org/jaffe_info.html. Accessed 21 Sept 2016

  • Maglogiannis I, Vouyioukas D, Aggelopoulos C (2009) Face detection and recognition of natural human emotion using Markov random fields. Pers Ubiquitous Comput 13:95–101

    Article  Google Scholar 

  • Pantic M, Bartlett M (2007) Machine analysis of facial expressions. In: Face recognition, chapter 20. I-Tech Education and Publishing, Vienna. Austria, pp 377–416

  • Setyati E, Suprapto YK, Purnomo MH (2012) Facial emotional expressions recognition based on active shape model and radial basis function network. In: Proceedings of the 2012 IEEE international conference on computational intelligence for measurement systems and applications (CIMSA), Tianjin, China, July 2012, pp 41–46

  • Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27:803–816

    Article  Google Scholar 

  • Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multi-modal affective database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3:42–55

    Article  Google Scholar 

  • Stegmann MB (2002) Analysis and segmentation of face images using point annotations and linear subspace techniques. Technical report IMM-REP-2002-22. Informatics and Mathematical Modelling, Technical University of Denmark

  • Suk M, Prabhakaran B (2014) Real-time mobile facial expression recognition system—a case study. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition workshops, Columbus, Ohio, USA, June 2014, pp 132–137

  • Tariq U, Lin K, Li Z, Zhou X, Wang Z, Le V, Huang TS, Lv X, Han TX (2011) Emotion Recognition from an ensemble of Features. In: Proceedings of the 2011 IEEE international conference on automatic face and gesture recognition and workshops, Santa Barbara, CA, USA, March 2011, pp 872–877

  • Valstar MF, Pantic M (2010) Induced disgust, happiness and surprise: an addition to the IMM facial expression database. In: Proceedings of the international conference on language resources and evaluation, workshop on emotion, Valletta, Malta, May 2010, pp 65–70

  • Wan C, Tian Y, Liu S (2012) Facial expression recognition in video sequences. In: Proceedings of the 10th World congress on intelligent control and automation, Beijing, China, July 2012, pp 4766–4770

  • Wang L, Gu X, Wang Y, Zhang L (2009) Happy-sad expression recognition using emotion geometry feature and support vector machine. Lect Notes Comput Sci 5507:535–542

    Article  Google Scholar 

  • Wei Y (2009) Research on facial expression recognition and synthesis. Master Thesis, Department of Computer Science and Technology, Nanjing University

  • Wu J, Mei L (2013) A face recognition algorithm based on ASM and Gabor features of key points. In: Proceedings of SPIE 2013, vol 8768, article number 87686L

  • Wu TF, Lin CJ, Wang RC (2004) Probability estimates for multi-class classification by pairwise coupling. J Mach Learn Res 5:975–1005

    MathSciNet  MATH  Google Scholar 

  • Zhang L, Tjondronegoro DW, Chandran V (2012) Discovering the best feature extraction and selection algorithms for spontaneous facial expression recognition. In: Proceedings of the 2012 IEEE conference on multimedia and expo, Melbourne, Australia, July 2012, pp 1027–1032

  • Zhao X, Zhang H, Xu Z (2012) Expression recognition by extracting facial features of shapes and textures. J Comput Inf Syst 8:3377–3384

    Google Scholar 

  • Zhou Q, Wang X (2013) Real-time facial expression recognition system based-on geometric features. Lect Notes Electr Eng 212:449–456

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Spanish Ministerio de Economía y Competitividad/European Regional Development Fund under TIN2015-72931-EXP and DPI2016-80894-R grants.

Author contributions

All authors have made substantial contributions in the definition of the research line, as well as in experimentation, data analysis, and manuscript preparation.

Author information

Authors and Affiliations

  1. Instituto de Investigación en Informática de Albacete, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Elena Lozano-Monasor

  2. Escuela Superior de Ingeniería Informática, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    María T. López & Francisco Vigo-Bustos

  3. Escuela de Ingenieros Industriales, Universidad de Castilla-La Mancha, 02071, Albacete, Spain

    Antonio Fernández-Caballero

Authors
  1. Elena Lozano-Monasor
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. María T. López
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Francisco Vigo-Bustos
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Antonio Fernández-Caballero
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Antonio Fernández-Caballero.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lozano-Monasor, E., López, M.T., Vigo-Bustos, F. et al. Facial expression recognition in ageing adults: from lab to ambient assisted living. J Ambient Intell Human Comput 8, 567–578 (2017). https://doi.org/10.1007/s12652-017-0464-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-017-0464-x

Keywords