Skip to main content
SpringerLink
Log in

Facial expression recognition via weighted group sparsity

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Considering the distinctiveness of different group features in the sparse representation, a novel joint multi-task and weighted group sparsity (JMT-WGS) method is proposed. By weighting popular group sparsity, not only the representation coefficients from the same class over their associate dictionaries may share some similarity, but also the representation coefficients from different classes have enough diversity. The proposed method is cast into a multi-task framework with two-stage iteration. In the first stage, representation coefficient can be optimized by accelerated proximal gradient method when the weights are fixed. In the second stage, the weights are computed via the prior information about their entropy. The experimental results on three facial expression databases show that the proposed algorithm outperforms other state-of-the-art algorithms and demonstrate the promising performance of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Darwin C. The Expression of the Emotions in Man and Animals. London: Penguin Classics, 1872

    Book  Google Scholar 

  2. Ekman P, Friesen W V. Pictures of facial affect. Technical Report, San Francisco: California Medical Center, 1976

    Google Scholar 

  3. Fasel B, Luettin J. Automatic facial expression analysis: a survey. Pattern Recognition, 2008, 36(1): 259–275

    Article  MATH  Google Scholar 

  4. Pantic M, Rothkrantz L J M. Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12): 1424–1445

    Article  Google Scholar 

  5. De La Torre F, Chu W, Xiong X, Cohn J. Intraface. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 2015, 1–8

    Google Scholar 

  6. Ekman P, Friesen W. Facial Action Coding System: Investigator’s Guide. Palo Alto, CA: Consulting Psychologists Press, 1993

    Google Scholar 

  7. Bartlett M S, Littlewort G, Fasel L, Movellan J R. Real time face detection and facial expression recognition: development and application to human-computer interaction. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. 2003

    Google Scholar 

  8. Bourel F, Chibelushi C C, Low A A. Robust facial expression recognition using a state-based model of spatiallylocalised facial dynamic. In: Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition. 2002, 113–118

    Chapter  Google Scholar 

  9. Tian Y I, Kanade T, Cohn J F. Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(2): 97–115

    Article  Google Scholar 

  10. Meng H Y, Bianchi-Berthouze N. Naturalistic affective expression classification by a multi-stage approach based on hidden Markov models. In: D’Mello S, Graesser A, Schuller B, et al., eds. Affective Computing and Intelligent Interaction, Lecture Notes in Computer Science, Vol 6975. Berlin: Springer, 2011, 378–387

    Chapter  Google Scholar 

  11. Cohen I, Sebe N, Garg A, Chen L S, Huang T S. Facial expression recognition from video sequences: temporal and static modeling. Computer Vision and Image Understanding, 2003, 91(1): 160–187

    Article  Google Scholar 

  12. Kotsia I, Pitas I. Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing, 2007, 16(1): 172–187

    Article  MathSciNet  Google Scholar 

  13. Sebe N, Lew M S, Cohen I, Sun Y, Gevers T, Huang T S. Authentic facial expression analysis. Image and Vision Computing, 2007, 25(12): 1856–1863

    Article  Google Scholar 

  14. Candès E J, Romberg J, Tao T. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006, 52(2): 489–509

    Article  MathSciNet  MATH  Google Scholar 

  15. Chen S S, Donoho D L, Saunders M A. Atomatic Decomposition by Basis Pursuit. SIAM Review, 2001, 43(1): 129–159

    Article  MathSciNet  MATH  Google Scholar 

  16. Mallat S G, Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 1993, 41(12): 3397–3415

    Article  MATH  Google Scholar 

  17. Wright J, Yang A Y, Ganesh A, Sastry S, Ma Y. Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(2): 210–227

    Article  Google Scholar 

  18. Yang M, Zhang L. Gabor feature based sparse representation for face recognition with gabor occlusion dictionary. In: Proceedings of European Conference on Computer Vision. 2010

    Google Scholar 

  19. Zheng H, Xie J C, Jin Z. Heteroscedastic sparse representation classification for face recognition. Neural Processing Letters, 2012, 35(3): 233–244

    Article  Google Scholar 

  20. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2006, 68(1): 49–67

    Article  MathSciNet  MATH  Google Scholar 

  21. Hinton G, Salakhutdinov R. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786): 504–507

    Article  MathSciNet  MATH  Google Scholar 

  22. Lai Z H, Wong W K, Xu Y, Yang J, Tang J, Zhang D. Approximate orthogonal sparse embedding for dimensionality reduction. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(4): 723–735

    Article  MathSciNet  Google Scholar 

  23. Lai Z H, Xu Y, Chen Q C, Yang J, Zhang D. Multilinear sparse principal component analysis. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(10): 1942–1950

    Article  Google Scholar 

  24. Lai Z H, Wong WH, Xu Y. Sparse alignment for robust tensor learning. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(10): 1779–1792

    Article  Google Scholar 

  25. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2006, 68(1): 49–67

    Article  MathSciNet  MATH  Google Scholar 

  26. Huang J Z, Zhang T. The benefit of group sparsity. The Annals of Statistics, 2010, 38(4): 1978–2004

    Article  MathSciNet  MATH  Google Scholar 

  27. Yuan X T, Liu X B, Yan S C. Visual classification with multitask joint sparse representation. IEEE Transactions on Image Processing, 2012, 21(10): 4349–4360

    Article  MathSciNet  Google Scholar 

  28. Lin Y Y, Liu T L, Fuh C S. Local ensemble kernel learning for object category recognition. In: Proceedings of IEEE Conference on Computer Vision (ICCV). 2007

    Google Scholar 

  29. Negahban S, Wainwright M J. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. The Annals of Statistics, 2011, 39(2): 1069–1097

    Article  MathSciNet  MATH  Google Scholar 

  30. Pong T K, Tseng P, Ji S, Ye J. Trace norm regularization: reformulations, algorithms, and multi-task learning. SIAM Journal on Optimization, 2010, 20(6): 3465–3489

    Article  MathSciNet  MATH  Google Scholar 

  31. He Z F, Yang M, Liu H D. Multi-task joint feature selection for multilabel classification. Chinese Journal of Electronics, 2015, 24(2): 281–287

    Article  Google Scholar 

  32. Cheng X, Li N J, Zhou T C, Wu Z Y, Zhou L. Multi-task object tracking with feature selection. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, 2015, (6): 1351–1354

    Article  Google Scholar 

  33. Zheng H, Geng X, Tao D C, Jin Z. A multi-task model for simultaneous face identification and facial expression recognition. Neurocomputing, 2016, 171(1): 515–523

    Article  Google Scholar 

  34. Bakker B, Heskes T. Task clustering and gating for bayesian multitask learning. Journal of Machine Learning Research, 2003, 4(4): 83–99

    MATH  Google Scholar 

  35. Evgeniou T, Pontil M. Regularized multi-task learning. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2004

    Google Scholar 

  36. Ando R K, Zhang T. A framework for learning predictive from multiple tasks and unlabeled data. Journal of Machine Learning Research, 2005, 6: 1817–1853

    MathSciNet  MATH  Google Scholar 

  37. Parameswaran S, Weinberger K Q. Large margin multi-task metric learning. In: Proceedings of Advances in Neural Information Processing Systems. 2010, 1867–1875

    Google Scholar 

  38. Zhang Y, Yeung D Y. Transfer metric learning by learning task relationships. In: Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2010

    Google Scholar 

  39. Chen X, Pan W K, Kwok J T, Garbonell J G. Accelerated gradient method for multi-task sparse learning problem. In: Proceedings of IEEE International Conference on Data Mining. 2009

    Google Scholar 

  40. Gehler P, Nowozin S. On feature combination for multiclass object classification. In: Proceedings of the 12th IEEE International Conference on Computer Vision. 2009, 221–228

    Google Scholar 

  41. Kanade T, Cohn J F, Tian Y. Comprehensive database for facial expression analysis. In: Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. 2000, 46–53

    Google Scholar 

  42. Wang J, Yin L J, Wei X Z, Sun Y. 3D facial expression recognition based on primitive surface feature distribution. In: Proceedings of IEEE Society Conference on Computer Vision and Pattern Recognition. 2006, 1399–1406

    Google Scholar 

  43. Dhall A, Goecke R, Joshi J, Wagner M, Gedeon T. Emotion recognitionin the wild challenge 2013. In: Proceedings of the 15th ACM International Conference on Multimodal Interaction. 2013, 509–516

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was partially supported by the Project funded by China Postdoctoral Science Foundation (2014M5615556), the National Natural Science Foundation of China (Grant Nos. 61273300, 61232007) and Jiangsu Natural Science Funds for Distinguished Young Scholar (BK20140022). Also it is partially supported by Jiangsu University Natural Science Funds (15KJB520024), the State Key Laboratory for Novel Software Technology from Nanjing University (KFKT2014B18), Jiangsu Key Laboratory of Image and Video Understanding for Social Safety (30920140122007). Finally, the authors would like to thank the anonymous reviewers for their constructive advice.

Author information

Authors and Affiliations

  1. Key Laboratory of Trusted Cloud Computing and Big Data Analysis, School of Information Engineering, Nanjing XiaoZhuang University, Nanjing, 211171, China

    Hao Zheng

  2. School of Computer Science and Engineering, Southeast University, Nanjing, 211189, China

    Hao Zheng & Xin Geng

Authors
  1. Hao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Geng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hao Zheng.

Additional information

Hao Zheng received the his BS degree from Southeast University (SEU), China in 1998, the MS degree from Nanjing University Posts and Telecommunications, China in 2005, and the PhD degree in pattern recognition and intelligence system from Nanjing University of Science and Technology, China in 2013. He is currently a postdoctor of SEU. His research interests include pattern recognition, image processing, face recognition, and computer vision.

Xin Geng received the BS (2001) and MS (2004) degrees in computer science from Nanjing University, China and the PhD (2008) degree in computer science from Deakin University, Australia. He has published over 25 refereed papers in these areas, including those published in prestigious journals and top international conferences. He has been a guest editor of several international journals, such as PRL and IJPRAI. He has served as a program committee member for a number of international conferences. He is also a frequent reviewer for various international journals and conferences. His research interests include computer vision, pattern recognition, and machine learning.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, H., Geng, X. Facial expression recognition via weighted group sparsity. Front. Comput. Sci. 11, 266–275 (2017). https://doi.org/10.1007/s11704-016-5204-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-016-5204-4

Keywords

Search

Navigation