Skip to main content
SpringerLink
Log in

Demographic attribute estimation in face videos combining local information and quality assessment

  • Special Issue Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Nowadays, video analysis applications are gaining popularity given the rise of CCTV systems and the availability of video cameras to the general public, such as cameras in mobile devices. Many image analysis and processing tasks have evolved toward video domain, with the advantage of redundant information obtained from several frames, which can help disambiguating many recognition outputs. In this context, there are also particular video problems to deal with, such as uncontrolled scenarios and poor image quality. Most existing works regarding facial demographic estimation are focused on still image datasets; therefore, we propose to address gender and age estimation in video scenarios. In order to handle known video problems such as low-quality image capture, occlusions and pose variations, we propose a threefold strategy to adapt current image-based attribute recognition algorithms. First, we employ a quality assessment step based on 12 metrics to select relevant good quality frames from a face video sequence. Second, we propose a component-based approach to determine the most discriminant local regions of the face for each specific attribute, under these varying conditions. Third, we evaluate different frame combination strategies to produce the final video prediction. In our experimental validation, conducted in 3 datasets (EURECOM Augmented, UvA-Nemo Smile and YouTube Faces datasets), we show the advantages of our proposed strategy for improving video-based demographic attribute classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and material

All the data employed for experimentation are under public domain.

Code availability

We cannot disclose the code.

Notes

  1. http://rgb-d.eurecom.fr/

  2. https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/.

References

  1. Becerra-Riera, F., Morales-González, A., Méndez-Vázquez, H.: A survey on facial soft biometrics for video surveillance and forensic applications. Artif. Intell. Rev. 52(2), 1155–1187 (2019). https://doi.org/10.1007/s10462-019-09689-5

    Article  Google Scholar 

  2. Park, U., Jain, A.K.: Face matching and retrieval using soft biometrics. Trans. Inf. Forens. Secur. 5(3), 406–415 (2010). https://doi.org/10.1109/TIFS.2010.2049842

    Article  Google Scholar 

  3. Tome, P., Fierrez, J., Vera-Rodriguez, R., Nixon, M.S.: Soft biometrics and their application in person recognition at a distance. Trans. Inf. Forens. Secur. 9(3), 464–475 (2014). https://doi.org/10.1109/TIFS.2014.2299975

    Article  Google Scholar 

  4. Ng, C.-B., Lo, W.-H.: Effect of image distortion on facial age and gender classification performance of convolutional neural networks”. IOP Conference Series: Materials Science and Engineering 495, 012029 (2019)

  5. Greco, A., Saggese, A., Vento, M., Vigilante, V.: Gender recognition in the wild: a robustness evaluation over corrupted images. J. Ambient Intell. Hum. Comput. (2020). https://doi.org/10.1007/s12652-020-02750-0

    Article  Google Scholar 

  6. Becerra-Riera, F., Morales-González, A., Mendez-Vazquez, H., Dugelay, J.-L.: Attribute-based quality assessment for demographic estimation in face videos. In: ICPR 2020, 25th International Conference on Pattern Recognition, 10-15 January 2021, Milan, Italy (Virtual Conference), IEEE, Ed., Milan (2021)

  7. Becerra-Riera, F., Morales-González, A., Méndez-Vázquez, H.: Exploring local deep representations for facial gender classification in videos. In: International Workshop on Artificial Intelligence and Pattern Recognition (IWAIPR), ser. Lecture Notes in Computer Science, vol. 11047. Springer, pp. 104–112 (2018)

  8. Zhang, Y., Shang, K., Wang, J., Li, N., Zhang, M.M.: Patch strategy for deep face recognition. IET Image Process. 12(5), 819–825 (2018)

    Article  Google Scholar 

  9. Méndez-Vázquez, H., Becerra-Riera, F., Morales-González, A., López-Avila, L., Tistarelli, M.: Local deep features for composite face sketch recognition. In: 7th International Workshop on Biometrics and Forensics, IWBF: Cancun, Mexico, May 2–3, 2019. IEEE, vol. 2019, pp. 1–6 (2019)

  10. Carcagnì, P., Coco, M.D., Cazzato, D., Leo, M., Distante, C.: A study on different experimental configurations for age, race, and gender estimation problems. EURASIP J. Image Video Process. 2015, 1–22 (2015)

    Article  Google Scholar 

  11. Gupta, R., Kumar, S., Yadav, P., Shrivastava, S.: Identification of age, gender and race smt (scare, marks, tattoos) from unconstrained facial images using statistical techniques. In: 2018 International Conference on Smart Computing and Electronic Enterprise (ICSCEE), pp. 1–8 (2018)

  12. Antipov, G., Baccouche, M., Berrani, S.-A., Dugelay, J.-L.: Effective training of convolutional neural networks for face-based gender and age prediction. Pattern Recognit. 72(C), 15–26 (2017)

    Article  Google Scholar 

  13. Afifi, M., Abdelhamed, A.: Afif4: Deep gender classification based on adaboost-based fusion of isolated facial features and foggy faces. J. Vis. Commun. Image Represent. 62, 77–86 (2019)

    Article  Google Scholar 

  14. Danişman, T.: Bagging ensemble for deep learning based gender recognition using test-time augmentation on large-scale datasets. Turkish J. Electr. Eng. Comput. Sci. 29(4), 2084–2100 (2021)

    Article  Google Scholar 

  15. Yang, T.-Y., Huang, Y.-H., Lin, Y.-Y., Hsiu, P.-C., Chuang, Y.-Y.: Ssr-net: A compact soft stagewise regression network for age estimation. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, 7, pp. 1078–1084 (2018). https://doi.org/10.24963/ijcai.2018/150

  16. Terhörst, P., Huber, M., Kolf, J. N., Zelch, I., Damer, N., Kirchbuchner, F., Kuijper, A.: Reliable age and gender estimation from face images: Stating the confidence of model predictions. In: 10th International Conference on BTAS. IEEE, 10 (2019)

  17. Althnian, A., Aloboud, N., Alkharashi, N., Alduwaish, F., Alrshoud, M., Kurdi, H.: Face gender recognition in the wild: an extensive performance comparison of deep-learned, hand-crafted, and fused features with deep and traditional models. Appl. Sci. 11(1), 89 (2021)

    Article  Google Scholar 

  18. Roxo, T., Proença, H.: Faces in the wild: efficient gender recognition in surveillance conditions (2021)

  19. Demirkus, M., Garg, K., Guler, S.: Automated person categorization for video surveillance using soft biometrics. In: Biometric Technology for Human Identification VII, vol. 7667. SPIE, pp. 236 – 247 (2010). https://doi.org/10.1117/12.851424

  20. Hadid, A., Pietikäinen, M.: Demographic classification from face videos using manifold learning. Neurocomputing 100, 197–205 (2013). (special issue: Behaviours in video)

    Article  Google Scholar 

  21. Wang, W.-C., Hsu, R.-Y., Huang, C.-R., Syu, L.-Y.: Video gender recognition using temporal coherent face descriptor. In: 16th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/ Distributed Computing, SNPD. IEE, pp. 113–118 (2015)

  22. Demirkus, M., Precup, D., Clark, J.J., Arbel, T.: Hierarchical spatio-temporal probabilistic graphical model with multiple feature fusion for binary facial attribute classification in real-world face videos. IEEE TPAMI 38, 1185–1203 (2016)

    Article  Google Scholar 

  23. Azzopardi, G., Greco, A., Saggese, A., Vento, M.: Fast gender recognition in videos using a novel descriptor based on the gradient magnitudes of facial landmarks. In: Advanced Video and Signal Based Surveillance, pp. 1–6 (2017)

  24. Dibeklioğlu, H., Gevers, T., Salah, A.A., Valenti, R.: A smile can reveal your age: enabling facial dynamics in age estimation. In: Proceedings of the 20th ACM International Conference on Multimedia, ser. MM’12. New York, NY, USA: Association for Computing Machinery, pp. 209–218. (2012). https://doi.org/10.1145/2393347.2393382

  25. Dantcheva, A., Brémond, F.: Gender estimation based on smile-dynamics. IEEE Trans. Inf. Forens. Secur. 12(3), 719–729 (2017). https://doi.org/10.1109/TIFS.2016.2632070

    Article  Google Scholar 

  26. Kharchevnikova, A., Savchenko, A.V.: Neural networks in video-based age and gender recognition on mobile platforms. Opt. Mem. Neural Netw. 27(4), 246–259 (2018)

    Article  Google Scholar 

  27. Oleksii, G., Olena, P.: Video stream gender classification using shallow cnn. Int. J. Pattern Recognit. Artif. Intell. (2020). https://doi.org/10.1142/S0218001421550016

    Article  Google Scholar 

  28. Roxo, T., Proença, H.: Is gender “in-the-wild” inference really a solved problem? IEEE Transactions on Biometrics, Behavior, and Identity Science, pp. 1–1 (2021)

  29. Levi, G., Hassner, T.: Age and gender classification using convolutional neural networks. In: Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. IEEE, pp. 34–42 (2015)

  30. Best-Rowden, L., Jain, A.K.: Learning face image quality from human assessments. IEEE Trans. Inf. Forens. Secur. 13(12), 3064–3077 (2018)

    Article  Google Scholar 

  31. Hernandez-Ortega, J., Galbally, J., Fierrez, J., Haraksim, R., Beslay, L.: Faceqnet: Quality assessment for face recognition based on deep learning. In: 2019 International Conference on Biometrics (ICB). IEEE, pp. 1–8 (2019)

  32. Xiang, X.: Beyond deep feature averaging: Sampling videos towards practical facial pain recognition. In: CVPR Workshops, pp. 37–42 (2019)

  33. Kharchevnikova, A., Savchenko, A.V.: Efficient video face recognition based on frame selection and quality assessment. PeerJ Comput. Sci. 7, e391 (2021)

    Article  Google Scholar 

  34. Selim, M., Sundararajan, S., Pagani, A., Stricker, D.: Image quality-aware deep networks ensemble for efficient gender recognition in the wild. In: Proceedings of the 13th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2018), vol. 5, pp. 351–358 (2018)

  35. Wang, X., Guo, R., Kambhamettu, C.: Deeply-learned feature for age estimation. Winter Conference on Applications of Computer Vision (WACV), pp. 534–541 (2015)

  36. Zhong, Y., Sullivan, J., Li, H.: Face attribute prediction using off-the-shelf deep learning networks. In: International Conference on Biometrics (ICB). IEEE (2016)

  37. Rodríguez, P., Cucurull, G., Gonfaus, J.M., Roca, F.X., González, J.: Age and gender recognition in the wild with deep attention. Pattern Recognit. 72(C), 563–571 (2017)

    Article  Google Scholar 

  38. Lee, B., Gilani, S.Z., Hassan, G.M., Mian, A., Facial gender classification-analysis using convolutional neural networks. In: Digital Image Computing: Techniques and Applications (DICTA). IEEE, vol. 2019, pp. 1–8 (2019)

  39. Min, R., Kose, N., Dugelay, J.-L.: KinectFaceDB: A Kinect database for face recognition. IEEE Trans. Syst. Man Cybern. Syst. 07 (2014). http://www.eurecom.fr/publication/4393

  40. Demirkus, M., Precup, D., Clark, J.J., Arbel, T.: Hierarchical temporal graphical model for head pose estimation and subsequent attribute classification in real-world videos. Comput. Vis. Image Underst. 136, 128–145 (2015)

    Article  Google Scholar 

  41. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324

    Article  MATH  Google Scholar 

  42. Méndez-Vázquez, H., Chang, L., Rizo-Rodríguez, D., Morales-González, A.: Evaluación de la calidad de las imágenes de rostros utilizadas para la identificación de las personas. Computación y Sistemas 16, 147–165, 06 (2012)

  43. Face Recognition Format Data Interchange, Version 2.0. InterNational Committee for Information Technology Standards (INCITS) Secretariat. Information Technology Industry Council, Standard (2006)

  44. Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: CVPR. IEEE Computer Society, pp. 529–534 (2011)

  45. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: A database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst, Tech. Rep. 07-49, October 2007

  46. Kumar, N., Berg, A., Belhumeur, P.N., Nayar, S.: Describable visual attributes for face verification and image search. IEEE TPAMI 33(10), 1962–1977 (2011)

    Article  Google Scholar 

  47. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR (2017). arXiv:1704.04861

  48. Rothe, R., Timofte, R., Gool, L.V.: Deep expectation of real and apparent age from a single image without facial landmarks. Int. J. Comput. Vis. (2016)

  49. Zhang, S.Y., Zhifei, Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)

  50. Openvino age and gender recognition retail 0013 (2019). https://docs.openvinotoolkit.org/2019_R1/_age_gender_recognition_retail_0013_description_age_gender_recognition_retail_0013.html

  51. Bilinski, P.T., Dantcheva, A., Brémond, F.: Can a smile reveal your gender? In: International Conference of the BIOSIG, ser. LNI, vol. P-260. GI/IEEE, vol. 2016, pp. 27–38 (2016)

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Advanced Technologies Application Center (CENATAV), 7A #21406 Siboney, Playa, P.C.12200, Havana, Cuba

    Fabiola Becerra-Riera, Annette Morales-González & Heydi Méndez-Vázquez

  2. Digital Security Department, EURECOM, Campus Sophia Tech, 450 route des Chappes, Biot Sophia Antipolis, 06410, France

    Jean-Luc Dugelay

Authors
  1. Fabiola Becerra-Riera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Annette Morales-González
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heydi Méndez-Vázquez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean-Luc Dugelay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabiola Becerra-Riera.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Becerra-Riera, F., Morales-González, A., Méndez-Vázquez, H. et al. Demographic attribute estimation in face videos combining local information and quality assessment. Machine Vision and Applications 33, 26 (2022). https://doi.org/10.1007/s00138-021-01269-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-021-01269-4

Keywords

Search

Navigation