Skip to main content
SpringerLink
Log in

Gender recognition from visible and thermal infrared facial images

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Most present research of gender recognition focuses on visible facial images, which are sensitive to illumination changes. In this paper, we proposed hybrid methods for gender recognition by fusing visible and thermal infrared images. First, the active appearance model is used to extract features from visible images, as well as local binary pattern features and several statistical temperature features are extracted from thermal infrared images. Then, feature selection is performed by using the F-test statistic. Third, we propose using Bayesian Networks to perform explicit and implicit fusion of visible and thermal infrared image features. For explicit fusion, we propose two Bayesian Networks to perform decision-level and feature-level fusion. For implicit fusion, we propose using features from one modality as privileged information to improve gender recognition by another modality. Finally, we evaluate the proposed methods on the Natural Visible and Infrared facial Expression spontaneous database and the Equinox face database. Experimental results show that both feature-level and decision-level fusion improve the gender recognition performance, compared to that achieved from one modality. The proposed implicit fusion methods successfully capture the role of privileged information of one modality, thus enhance the gender recognition from another modality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Abate AF, Nappi M, Riccio D, Sabatino G (2007) 2d and 3d face recognition: a survey. Pattern Recog Lett 28(14):1885–1906. Image: information and control

    Article  Google Scholar 

  2. Aghajanian J, Warrell J, Prince SJD, Li P, Rohn JL, Baum B (2009) Patch-based within-object classification. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 1125–1132

  3. Alexandre LA (2010) Gender recognition: a multiscale decision fusion approach. Pattern Recog Lett 31(11):1422–1427

    Article  Google Scholar 

  4. Arandjelović O, Hammoud R, Cipolla R (2010) Thermal and reflectance based personal identification methodology under variable illumination. Pattern Recog 43(5):1801–1813

    Article  MATH  Google Scholar 

  5. Balci K, Atalay V (2002) Pca for gender estimation: which eigenvectors contribute? In: Proceedings. 16th international conference on pattern recognition, (ICPR 2002), vol 3. IEEE, pp 363–366

  6. Baluja S, Rowley HA (2007) Boosting sex identification performance. Int J Comput Vis 71(1):111–119

    Article  Google Scholar 

  7. Bebis G, Gyaourova A, Singh S, Pavlidis I (2006) Face recognition by fusing thermal infrared and visible imagery. Image Vis Computing 24(7):727–742

    Article  Google Scholar 

  8. Bekios-Calfa J, Buenaposada JM, Baumela L (2011) Revisiting linear discriminant techniques in gender recognition. IEEE Trans Pattern Anal Mach Intell 33(4):858–864

    Article  Google Scholar 

  9. Buchala S, Davey N, Gale TM, Frank RJ (2005) Analysis of linear and nonlinear dimensionality reduction methods for gender classification of face images. Int J Syst Sci 36(14):931–942

    Article  MathSciNet  MATH  Google Scholar 

  10. Chen C, Ross A (2011) Evaluation of gender classification methods on thermal and near-infrared face images. In: 2011 international joint conference on biometrics (IJCB). IEEE, pp 1–8

  11. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685

    Article  Google Scholar 

  12. Cootes T am_tools. http://www.isbe.man.ac.uk/bim/software/am_tools_doc/

  13. Cottrell GW, Metcalfe J (1990) Empath: face, emotion, and gender recognition using holons. In: Proceedings of the 1990 conference on Advances in neural information processing systems, vol 3. Morgan Kaufmann Publishers Inc. pp 564–571

  14. De Marsico M, Nappi M, Faro DR (2010) Face recognition against occlusions and expression variations. IEEE Trans Syst Man Cybern Part A: Syst Humans 40 (1):121–132

    Article  Google Scholar 

  15. Ding CHQ (2002) Analysis of gene expression profiles: class discovery and leaf ordering. In: Proceedings of the sixth annual international conference on computational biology. ACM, pp 127–136

  16. Fu X, Dai G, Wang C, Zhang L (2010) Centralized gabor gradient histogram for facial gender recognition. In: 2010 sixth international conference on natural computation (ICNC), vol 4. IEEE, pp 2070–2074

  17. Gao W, Ai H (2009) Face gender classification on consumer images in a multiethnic environment. Adv Biomet:169–178

  18. Ghiass RS, Arandjelovic O, Bendada H, Maldague X (2013) Infrared face recognition: a literature review. In: The 2013 international joint conference on neural networks (IJCNN). IEEE, pp 1–10

  19. Golomb BA, Lawrence DT, Sejnowski TJ (1991) Sexnet: a neural network identifies sex from human faces. Adv Neural Inf Process Syst 3:572–577

    Google Scholar 

  20. Gyaourova A, Bebis G, Pavlidis I (2004) Fusion of infrared and visible images for face recognition. Comput Vis-ECCV 2004:456–468

    MATH  Google Scholar 

  21. Hernández B, Olague G, Hammoud R, Trujillo L, Romero E (2007) Visual learning of texture descriptors for facial expression recognition in thermal imagery. Comput Vis Image Underst 106(2):258–269

    Article  Google Scholar 

  22. Jain A, Huang J, Fang S (2005) Gender identification using frontal facial images. In: IEEE international conference on multimedia and expo, 2005. (ICME 2005). IEEE, pp 4–pp

  23. Jain AK, Dass SC, Nandakumar K (2004) Can soft biometric traits assist user recognition? In: Proceedings of SPIE, vol 5404, pp 561–572

  24. Khan MM, Ward RD, Ingleby M (2009) Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature. ACM Transa Appl Perception (TAP) 6(1):6

    Google Scholar 

  25. Lee TS (1996) Image representation using 2d gabor wavelets. IEEE Trans Pattern Anal Mach Intell 18(10):959–971

    Article  Google Scholar 

  26. Makinen E, Raisamo R (2008) Evaluation of gender classification methods with automatically detected and aligned faces. IEEE Trans Pattern Anal Mach Intell 30 (3):541–547

    Article  Google Scholar 

  27. Mäkinen E, Raisamo R (2008) An experimental comparison of gender classification methods. Pattern Recog Lett 29(10):1544–1556

    Article  Google Scholar 

  28. Moghaddam B, Yang MH (2002) Learning gender with support faces. IEEE Trans Pattern Anal Mach Intell 24(5):707–711

    Article  Google Scholar 

  29. Murphy KP (1998) Inference and learning in hybrid Bayesian networks. University of California, Berkeley, Computer Science Division

  30. Ng C, Tay Y, Goi BM (2012) Recognizing human gender in computer vision: a survey. In: PRICAI 2012: trends in artificial intelligence, pp 335–346

  31. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern SMC-9:62–66

    Google Scholar 

  32. Raghavendra R, Dorizzi B, Rao A, Hemantha Kumar G (2011) Particle swarm optimization based fusion of near infrared and visible images for improved face verification. Pattern Recog 44(2):401–411

    Article  Google Scholar 

  33. Rojas-Bello RN, Lago-Fernandez LF, Martinez-Mufioz G, Sanchez-Montanes M A (2011) A comparison of techniques for robust gender recognition. In: 2011 18th IEEE international conference on image processing (ICIP). IEEE, pp 561–564

  34. Saatci Y, Town C (2006) Cascaded classification of gender and facial expression using active appearance models. In: 7th international conference on automatic face and gesture recognition, 2006. (FGR 2006). IEEE, pp 393–398

  35. Scalzo F, Bebis G, Nicolescu M, Loss L, Tavakkoli A (2008) Feature fusion hierarchies for gender classification. In: 19th international conference on pattern recognition, 2008. (ICPR 2008). IEEE, pp 1–4

  36. Selinger A, Socolinsky DA (2001) Appearance-based facial recognition using visible and thermal imagery: a comparative study. Technical report, Equinox Corporation

  37. Shih H-C (2013) Robust gender classification using a precise patch histogram. Pattern Recog 46(2):519–528

    Article  Google Scholar 

  38. Singh R, Vatsa M, Noore A (2008) Integrated multilevel image fusion and match score fusion of visible and infrared face images for robust face recognition. Pattern Recogn 41(3):880–893

    Article  MATH  Google Scholar 

  39. Sun Z, Bebis G, Yuan X, Louis SJ (2002) Genetic feature subset selection for gender classification: a comparison study. In: Proceedings. Sixth IEEE workshop on applications of computer vision, 2002.(WACV 2002). IEEE, pp 165–170

  40. Vezzetti E, Marcolin F (2012) 3d human face description: landmarks measures and geometrical features. Image Vis Comput 30(10):698–712

    Article  Google Scholar 

  41. Vapnik V, Vashist A et al (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5–6):544–557

    Article  MATH  Google Scholar 

  42. Wang S, Liu Z, Lv S, Lv Y, Wu G, Peng P, Chen F, Wang X (2010) Anaturalvisibleandinfraredfacialexpressiondatabaseforexpressionrecognition and emotion inference. IEEE Trans Multimed 12(7):682–691

    Article  Google Scholar 

  43. Wang JG, Li J, Yau WY, Sung E (2010) Boosting dense sift descriptors and shape contexts of face images for gender recognition. In: 2010 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE, pp 96–102

  44. Wang Z, Wang S (2011) Spontaneous facial expression recognition by using feature-level fusion of visible and thermal infrared images. In: 2011 IEEE international workshop on machine learning for signal processing (MLSP). IEEE, pp 1–6

  45. Wang C, Huang D, Wang Y, Zhang G (2012) Facial image-based gender classification using local circular patterns. In: 21th international conference on pattern recognition, 2012. (ICPR 2012). IEEE, pp 2432–2435

  46. Wang S, He S (2013) Spontaneous facial expression recognition by fusing thermal infrared and visible images. Intell Autonom Syst 12:263–272

    Article  Google Scholar 

  47. Wang S, Liu Z, Wang Z, Guobing W, Shen P, He S, Wang X (2013) Analyses of a multimodal spontaneous facial expression database. IEEE Trans Affect Comput 4(1):34–46

    Article  Google Scholar 

  48. Yang Z, Ai H (2007) Demographic classification with local binary patterns. Adv Biomet:464–473

  49. Yamaguchi MK, Hirukawa T, Kanazawa S et al (1995) Judgment of gender through facial parts. Perception 24:563–563

Download references

Acknowledgments

This work has been supported by the National Natural Science Foundation of China (61175037, 61228304, 61473270), the US-NSF (CNS-1205664 ), and Project from Anhui Science and Technology Agency(1508085SMF223).

Author information

Authors and Affiliations

  1. Key Lab of Computing and Communication Software of Anhui Province, School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, 230027, People’s Republic of China

    Shangfei Wang, Zhen Gao, Shan He & Menghua He

  2. Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY, 12180, USA

    Qiang Ji

Authors
  1. Shangfei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhen Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Menghua He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Qiang Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shangfei Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, S., Gao, Z., He, S. et al. Gender recognition from visible and thermal infrared facial images. Multimed Tools Appl 75, 8419–8442 (2016). https://doi.org/10.1007/s11042-015-2756-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2756-5

Keywords

Search

Navigation