Skip to main content
Springer Nature Link
Log in

Reliable facial expression recognition for multi-scale images using weber local binary image based cosine transform features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Accurate recognition of facial expression is a challenging problem especially from multi-scale and multi orientation face images. In this article, we propose a novel technique called Weber Local Binary Image Cosine Transform (WLBI-CT). WLBI-CT extracts and integrates the frequency components of images obtained through Weber local descriptor and local binary descriptor. These frequency components help in accurate classification of various facial expressions in the challenging domain of multi-scale and multi-orientation facial images. Identification of significant feature set plays a vital role in the success of any facial expression recognition system. Effect of multiple feature sets with varying block sizes has been investigated using different multi-scale images taken from well-known JAFEE, MMI and CK+ datasets. Extensive experimentation has been performed to demonstrate that the proposed technique outperforms the contemporary techniques in terms of recognition rate and computational time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24

Similar content being viewed by others

References

  1. Ahmed N, Natarajan T, Rao KR (1974) Discrete cosine transform. IEEE Trans Comput C-23:90–93

    Article  MathSciNet  MATH  Google Scholar 

  2. Ahonen T, Hadid A, Pietikäinen M (2004) Face recognition with local binary patterns. 8th European Conf Comput Vision 3021:469–481

    MATH  Google Scholar 

  3. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041

    Article  MATH  Google Scholar 

  4. Ajit Krisshnaa NL, Kadetotad Deepaka V, Manikantana K, Ramachandranb S (2014) Face recognition using transform domain feature extraction and PSO-based feature selection. Pattern Appl Soft Comput 22:141–161

    Article  Google Scholar 

  5. Bartlett MS, Littlewort G, Fasel I, Movellan JR (2003) Real time face detection and facial expression recognition: development and applications to human computer interaction, In: Conference on Computer Vision and Pattern Recognition Workshop, p 53

  6. Chao WL, Ding JJ, Liu JZ (2015) Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection. J Signal Process 2:552–561

    Google Scholar 

  7. Chen J, He C, Zhao G, Pietikainen M, Chen X, Gao W (2010) WLD: a robust local image descriptor. IEEE Trans Pattern Anal Mach Intell 32(9):1705–1720

    Article  Google Scholar 

  8. Chen W, Lu X, Du Y, Tian W (2013) Boosting local Gabor binary patterns for gender recognition. In: Proceedings of Ninth International Conference on Natural Computation, China, pp 34–38

  9. Chen J, Chen Z, Chi Z, Fu H (2016) Facial expression recognition in video with multiple feature fusion. IEEE Trans Affect Comput 2:1–16

    Google Scholar 

  10. Dabbaghchian S, Ghaemmaghami MP, Aghagolzadeh A (2010) Feature extraction using discrete cosine transform and discrimination power analysis with a face recognition technology. Pattern Recogn 43:1431–1440

    Article  MATH  Google Scholar 

  11. Dornaika F, Lazkano E, Sierra B (2011) Improving dynamic facial expression recognition with feature subset selection. Pattern Recogn Lett 32:740–748

    Article  Google Scholar 

  12. Du S, Martinez A (2011) The resolution of facial expressions of emotion. J Vis 11(13):1–13

    Article  Google Scholar 

  13. Ebenezer O, Yongzhao Z, Qi RM (2014) A neural-Adaboost based facial expression recognition system. Expert Syst Appl 41(7):3383–3390

    Article  Google Scholar 

  14. Eleftheriadis S, Rudovic O, Pantic M (2015) Discriminative shared Gaussian processes for Multiview and view-invariant facial expression recognition. IEEE Trans Image Process 24:189–204

    Article  MathSciNet  Google Scholar 

  15. Fan X, Tjahjadi T (2015) A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences. Pattern Recogn 48:3407–3416

    Article  Google Scholar 

  16. Goa T, Fenga XL, Lub H, Zhaia JH (2013) A novel face feature descriptor using adaptively weighted extended LBP pyramid. J Opt 124:6286–6291

    Google Scholar 

  17. Guo X, Zhang X, Deng C, Wei J (2013) Facial expression recognition based on independent component analysis. J Multimed 8(4):402–409

    Article  Google Scholar 

  18. Han J, Kamber M (2006) Data mining: concepts and techniques. Morgan Kaufmann pp 1–703

  19. Happy SL, Routray A (2015) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput 6:1–12

    Article  Google Scholar 

  20. Jing XY, Tang YY, Zhang D (2005) Rapid and brief communications: a Fourier-LDA approach for image recognition. Pattern Recogn 38:453–457

    Article  MATH  Google Scholar 

  21. Kamarol SKA, Jaward MH, Parkkinen J, Parthiban R (2016) Spatiotemporal feature extraction for facial expression recognition. IET Image Process 10:534–541

    Article  Google Scholar 

  22. Khan RA, Meyer A, Konik H, Bouakaz S (2012) Human vision inspired framework for facial expressions recognition. In: IEEE International Conference on Image Processing, pp 2593–2596

  23. Khan RA, Meyer A, Konik H, Bouakaz S (2013) Framework for reliable, real-time facial expression recognition for low resolution images. Pattern Recogn Lett 34:1159–1168

    Article  Google Scholar 

  24. Kumari J, Rajesh R, Pooja KM (2015) Facial expression recognition: a survey. Procedia Comput Sci 58:486–491

    Article  Google Scholar 

  25. Kumbhar M, Jadhav A, Patil M (2012) Facial expression recognition based on image feature. Int J Comput Commun Eng 1:117–119

    Article  Google Scholar 

  26. Li Y, Tao C, Tan Y, Shang K, Tian J (2016) Unsupervised multilayer feature learning for satellite image scene classification. IEEE Geosci Remote Sens Lett 13:157–161

    Article  Google Scholar 

  27. Liu H, Sun J, Liu L, Zhang H (2009) Feature selection with dynamic mutual information. J Pattern Recognit 42(7):1330–1339

    Article  MATH  Google Scholar 

  28. Long F, Bartlett MS (2016) Video-based facial expression recognition using learned spatiotemporal pyramid sparse coding features. Neurocomputing 173(3):2049–2054

    Article  Google Scholar 

  29. Lucey P, Cohn JF, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The Extended Cohn-Kanade Dataset (CK+) A complete dataset for action unit and emotion-specified expression, IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 94–101

  30. Lyons MJ, Kamachi M, Gyoba J (1997) Japanese female facial expressions (JAFFE), pp 200–2005

  31. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29:51–59

    Article  Google Scholar 

  32. Ojala T, Pietkainen M, Maenpaa T (2002) Multi resolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  33. Ouyang Y, Sang N, Huang R (2015) Accurate and robust facial expressions recognition by fusing multiple sparse representation based classifiers. J Neurocomputing 149:71–78

    Article  Google Scholar 

  34. Pantic M, Valstar M, Radermaker R, Maat L (2005) Web-based database for facial expression analysis. In: Proceedings of the 13th ACM International Conference on Multimedia, pp. 317–321

  35. Platt JC (1999) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges CJC, Smola AJ (eds) In advances in kernel methods. MIT Press, Cambridge, pp 185–208

    Google Scholar 

  36. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann pp. 235–240

  37. Shan C, Gong S, McOwan P (2009) Facial expression recognition based on local binary patterns: a comprehensive stud y. Image Vis Comput 27:803–816

    Article  Google Scholar 

  38. Song I, Kim HJ, Jeon PB (2014) Deep learning for real-time robust facial expression recognition on a smart phone. Proc. of IEEE International Conference on Consumer Electronics, pp 564–567

  39. Tian Y (2004) Evaluation of face resolution for expression analysis. In: CVPR Workshop on Face Processing in Video pp. 1–7

  40. Tian Y, Kanade T, Cohn J (2005) Handbook of face recognition, (Chapter 11. Facial Expression Analysis). Springer, pp. 1–40

  41. Tong Y, Chen R, Cheng Y (2014) Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle. Int J Light Electron Opt 125(16):4186–4189

    Article  Google Scholar 

  42. Ucar A, Demir Y, Guzelis C (2016) A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering. Neural Comput & Applic 27:131–142

    Article  Google Scholar 

  43. Valstar M, Pantic M (2006) Fully automatic facial action unit detection and temporal analysis. In: IEEE Conference on Computer Vision and Pattern Recognition Workshop, p 149

  44. Viola P, Jones M (2001) Rapid object detection using a boosted Cascade of simple features. IEEE Comput Soc Conf Comput Vision Pattern Recognit (CVPR'01) 1:511

    Google Scholar 

  45. Wang Z, Ruan Q, An G (2016) Facial expression recognition using sparse local fisher discriminant analysis. Neurocomputing 174:756–766

    Article  Google Scholar 

  46. Yankun Z, Chongqing L (2004) Efficient face recognition method based on DCT and LDA. J Syst Eng Electron 15(2):211–216

    Google Scholar 

  47. Yu K, Wanga Z, Zhuo L, Wangc J, Chi Z, Feng D (2013) Learning realistic facial expressions from web images. Pattern Recogn 46:2144–2155

    Article  Google Scholar 

  48. Zhang W, Zhang Y, Ma L, Guan J, Gong S (2015) Multimodal learning for facial expression recognition. Pattern Recogn 48:3191–3202

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Shaheed Zulfikar Ali Bhutto Institute of Science and Technology, House#13, Street 52, CAT B, G.10/3, Islamabad, 44000, Pakistan

    Sajid Ali Khan & Muhammad Usman

  2. Department of Computer Science and Software Engineering, International Islamic University, Islamabad, Pakistan

    Ayyaz Hussain

Authors
  1. Sajid Ali Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayyaz Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Usman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sajid Ali Khan.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, S.A., Hussain, A. & Usman, M. Reliable facial expression recognition for multi-scale images using weber local binary image based cosine transform features. Multimed Tools Appl 77, 1133–1165 (2018). https://doi.org/10.1007/s11042-016-4324-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4324-z

Keywords