Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Adaptively Weighted Facial Expression Recognition by Feature Fusion Under Intense Illumination Condition

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10639))

Included in the following conference series:

  • 3421 Accesses

Abstract

Accurate and robust facial expression recognition under complex environment is a challenging task. In this paper, we propose an adaptively weighted facial expression recognition approach to overcome the intense illumination difficulty by fusing diverse illumination invariant appearance features. First, a novel neural-network-based adaptive weight assignment strategy is designed to eliminate the adverse illumination variations efficiently and effectively. Then, a feature fusion strategy is developed to combine two of the most successful illumination invariant appearance descriptors, namely Gabor and Local Binary Patterns (LBP), for giving comprehensive and robust description of facial expressions. Extensive experiments demonstrate the superiority of the proposed approach on the common used CK+ dataset, especially the adaptive weight assignment for the significant improvement of recognition accuracy under extreme and intense illumination conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zou, X., Kittler, J., Messer, K.: Illumination invariant face recognition: a survey. In: IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–8 (2007)

    Google Scholar 

  2. Jobson, D.J., Rahman, Z., Woodell, G.A.: A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 6(7), 965–976 (1997)

    Article  Google Scholar 

  3. Wang, H., Li, S.Z., Wang, Y.: Face recognition under varying lighting conditions using self quotient image. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 819–824 (2004)

    Google Scholar 

  4. Ou, J., Bai, X.B., Pei, Y., Ma, L., Liu, W.: Automatic facial expression recognition using gabor filter and expression analysis. In: International Conference on Computer Modeling and Simulation, vol. 2, pp. 215–218 (2010)

    Google Scholar 

  5. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  6. Paysan, P., Knothe, R., Amberg, B., Romdhani, S., Vetter, T.: A 3D face model for pose and illumination invariant face recognition. In: IEEE International Conference on Advanced Video and Signal Based Surveillance, pp. 296–301 (2009)

    Google Scholar 

  7. Wang, S., Liu, Z., Lv, S., Lv, Y., Wu, G., Peng, P., Chen, F., Wang, X.: A natural visible and infrared facial expression database for expression recognition and emotion inference. IEEE Trans. Multimedia 12(7), 682–691 (2010)

    Article  Google Scholar 

  8. Di, W., Zhang, L., Zhang, D., Pan, Q.: Studies on hyperspectral face recognition in visible spectrum with feature band selection. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 40(6), 1354–1361 (2010)

    Article  Google Scholar 

  9. Ahonen, T., Hadid, A., Pietikäinen, M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Article  MATH  Google Scholar 

  10. Mohamed, A.A., Gavrilova, M.L., Yampolskiy, R.V.: Artificial face recognition using wavelet adaptive LBP with directional statistical features. In: International Conference on Cyberworlds, pp. 23–28 (2012)

    Google Scholar 

  11. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 511–518 (2001)

    Google Scholar 

  12. Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 971–987 (2002)

    Article  MATH  Google Scholar 

  13. Yang, J., Yang, J., Zhang, D., Lu, J.: Feature fusion: parallel strategy vs. serial strategy. Pattern Recogn. 36(6), 1369–1381 (2003)

    Article  MATH  Google Scholar 

  14. Ekman, P., Friesen, W.: Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists, San Francisco (1978)

    Google Scholar 

  15. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 94C–101 (2010)

    Google Scholar 

  16. Chen, J., Chen, Z., Chi, Z., Fu, H.: Facial expression recognition in video with multiple feature fusion. IEEE Trans. Affect. Comput. PP, 1 (2016)

    Google Scholar 

  17. Happy, S., Routray, A.: Automatic facial expression recognition using features of salient facial patches. IEEE Trans. Affect. Comput. 6(1), 1–12 (2015)

    Article  Google Scholar 

  18. Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: IEEE Winter Conference on Applications of Computer Vision, pp. 1–10 (2016)

    Google Scholar 

  19. Jung, H., Lee, S., Yim, J., Park, S., Kim, J.: Joint fine-tuning in deep neural networks for facial expression recognition. In: International Conference on Computer Vision, pp. 2983–2991 (2015)

    Google Scholar 

  20. Liu, M., Li, S., Shan, S., Wang, R., Chen, X.: Deeply learning deformable facial action parts model for dynamic expression analysis. In: Asian Conference on Computer Vision, pp. 143–157 (2014)

    Google Scholar 

  21. Zavaschi, T.H., Britto, A.S., Oliveira, L.E., Koerich, A.L.: Fusion of feature sets and classifiers for facial expression recognition. Expert Syst. Appl. 40(2), 646–655 (2013)

    Article  Google Scholar 

  22. Jain, S., Hu, C., Aggarwal, J.K.: Facial expression recognition with temporal modeling of shapes. In: International Conference on Computer Vision Workshops, pp. 1642–1649 (2011)

    Google Scholar 

  23. Liu, M., Li, S., Shan, S., Chen, X.: AU-aware deep networks for facial expression recognition. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2013)

    Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (61572450, 61303150), Anhui Provincial Natural Science Foundation (1708085QF138), the Fundamental Research Funds for the Central Universities (WK2350000002), the Open Funding Project of State Key Lab of Virtual Reality Technology and Systems, Beihang University (BUAA-VR-16KF-12), the Open Funding Project of State Key Lab of Novel Software Technology, Nanjing University (KFKT2016B08).

Author information

Authors and Affiliations

  1. Department of Automation, University of Science and Technology of China, Hefei, 230027, China

    Yuechuan Sun & Jun Yu

Authors
  1. Yuechuan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun Yu .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Sun, Y., Yu, J. (2017). Adaptively Weighted Facial Expression Recognition by Feature Fusion Under Intense Illumination Condition. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10639. Springer, Cham. https://doi.org/10.1007/978-3-319-70136-3_65

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70136-3_65

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70135-6

  • Online ISBN: 978-3-319-70136-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation