Skip to main content
SpringerLink
Log in

Facial Beauty Prediction Using Transfer and Multi-task Learning Techniques

  • Conference paper
  • First Online:
Pattern Recognition and Artificial Intelligence (ICPRAI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12068))

Abstract

The objective of facial beauty prediction, which is a significant yet challenging problem in the domains of computer vision and machine learning, is to develop a human-like model that automatically evaluates facial attractiveness. Using deep learning methods to enhance facial beauty prediction is a promising and important area. This study provides a new framework for simultaneous facial attractiveness assessment, gender recognition as well as ethnicity identification using deep Convolutional Neural Networks (CNNs). Specifically, a deep residual network originally trained on massive face datasets is utilized which is capable of learning high-level and robust features from face images. Furthermore, a multi-task learning algorithm that operates on the effective features, exploits the synergy among the tasks. Said differently, a multi-task learning scheme is employed by our model to learn optimal shared features for these correlated tasks in an end-to-end manner. Interestingly, prediction correlation of 0.94 is achieved by our method for the SCUT-FBP5500 benchmark dataset (spanning 5500 facial images), which would certainly support the efficacy of our proposed model. This would also indicate significant improvement in accuracy over the other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Laurentini, A., Bottino, A.: Computer analysis of face beauty: a survey. Comput. Vis. Image Underst. 125, 184–199 (2014)

    Article  Google Scholar 

  2. Liu, S., Fan, Y.-Y., Samal, A., Guo, Z.: Advances in computational facial attractiveness methods. Multimedia Tools Appl. 75(23), 16633–16663 (2016). https://doi.org/10.1007/s11042-016-3830-3

    Article  Google Scholar 

  3. Zhang, D., Chen, F., Xu, Y.: Computer Models for Facial Beauty Analysis. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32598-9_14

    Book  Google Scholar 

  4. Lin, L., Liang, L., Jin, L.: R2-ResNeXt: a ResNeXt-based regression model with relative ranking for facial beauty prediction. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 85–90. IEEE, Beijing (2018)

    Google Scholar 

  5. Fan, Y.-Y., Liu, S., Li, B., Guo, Z., Samal, A., Wan, J.: Label distribution-based facial attractiveness computation by deep residual learning. IEEE Trans. Multimed. 20(8), 2196–2208 (2018)

    Article  Google Scholar 

  6. Xie, D., Liang, L., Jin, L., Xu, J., Li, M.: SCUT-FBP: a benchmark dataset for facial beauty perception. In: 2015 IEEE International Conference on Systems, Man, and Cybernetics, pp. 1821–1826. IEEE, Kowloon (2015)

    Google Scholar 

  7. Xu, J., Jin, L., Liang, L., Feng, Z., Xie, D.: A new humanlike facial attractiveness predictor with cascaded fine-tuning deep learning model (2015). arXiv preprint arXiv:1511.02465

  8. Liang, L., Lin, L., Jin, L., Xie, D., Li, M.: SCUT-FBP5500: a diverse benchmark dataset for multi-paradigm facial beauty prediction. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 1598–1603. IEEE, Beijing (2018)

    Google Scholar 

  9. Xu, L., Xiang, J., Yuan, X.: CRNet: classification and regression neural network for facial beauty prediction. In: Hong, R., Cheng, W.-H., Yamasaki, T., Wang, M., Ngo, C.-W. (eds.) PCM 2018. LNCS, vol. 11166, pp. 661–671. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00764-5_61

    Chapter  Google Scholar 

  10. Shi, S., Gao, F., Meng, X., Xu, X., Zhu, J.: Improving facial attractiveness prediction via co-attention learning. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4045–4049. IEEE, Brighton (2019)

    Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas (2016)

    Google Scholar 

  12. Guo, Y., Zhang, L., Hu, Y., He, X., Gao. J.: Ms-celeb-1 m: a dataset and benchmark for large-scale face recognition (2016). arXiv preprint arXiv:1607.08221

  13. Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 67–74. IEEE, Xi’an (2018)

    Google Scholar 

  14. Zhu, L., Wang, K., Lin, L., Zhang, L.: Learning a lightweight deep convolutional network for joint age and gender recognition. In: 23rd International Conference on Pattern Recognition (ICPR), pp. 3282–3287. IEEE, Cancún Center (2016)

    Google Scholar 

  15. Gao, L., Li, W., Huang, Z., Huang, D., Wang, Y.: Automatic facial attractiveness prediction by deep multi-task learning. In: IEEE 24th International Conference on Pattern Recognition (ICPR), pp. 3592–3597. IEEE, Beijing (2018)

    Google Scholar 

  16. Ranjan, R., Patel, V.M., Chellappa, R.: HyperFace: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans. Pattern Anal. Mach. Intell. 41(1), 121–135 (2019)

    Article  Google Scholar 

  17. Zhang, Z., Luo, P., Loy, C. C., Tang, X.: Facial landmark detection by deep multitask learning. In: 13th European Conference Computer Vision (ECCV 2014), Zurich, Switzerland, pp. 94–108 (2014)

    Google Scholar 

  18. Eisenthal, Y., Dror, G., Ruppin, E.: Facial attractiveness: beauty and the machine. Neural Comput. 18(1), 119–142 (2006)

    Article  Google Scholar 

  19. Kagian, A., Dror, G., Leyvand, T., Meilijson, I., Cohen-Or, D., Ruppin, E.: A machine learning predictor of facial attractiveness revealing human-like psychophysical biases. Vis. Res. 48, 235–243 (2008)

    Article  Google Scholar 

  20. Vahdati, E., Suen, C.Y.: A novel female facial beauty predictor. In: International Conference on Pattern Recognition and Artificial Intelligence (ICPRAI), Montreal, pp. 378–382 (2018)

    Google Scholar 

  21. Vahdati, E., Suen, C.Y.: Female facial beauty analysis using transfer learning and stacking ensemble model. In: Karray, F., Campilho, A., Yu, A. (eds.) ICIAR 2019. LNCS, vol. 11663, pp. 255–268. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27272-2_22

    Chapter  Google Scholar 

  22. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE, Miami (2009)

    Google Scholar 

  23. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105 (2012)

    Google Scholar 

  24. Xie, S., Girshick, R., Doll´ar, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: IEEE International Conference Computer Vision and Pattern Recognition, Honolulu, HI, USA, pp. 5987–5995 (2017)

    Google Scholar 

  25. Ehrlich, M., Shields, T.J., Almaev, T., Amer, M.R.: Facial attributes classification using multi-task representation learning: In: IEEE International Conference Computer Vision Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA, pp. 752–760 (2016)

    Google Scholar 

  26. Hand, E.M., Chellappa, R.: Attributes for improved attributes: a multi-task network utilizing implicit and explicit relationships for facial attribute classification. In: Thirty-First AAAI Conference Artificial Intelligence (AAAI-17), San Francisco, California, USA, pp. 4068–4074 (2017)

    Google Scholar 

  27. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  28. Paszke, A., et al.: Automatic differentiation in PyTorch. In: 31st Conference Neural Inform. Processing System (NIPS 2017), Long Beach, CA, USA (2017)

    Google Scholar 

  29. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L. C.: MobileNetV2: Inverted residuals and linear bottlenecks (2018). arXiv preprint arXiv:1801.04381v3

Download references

Acknowledgment

This research was supported by a research grant from NSERC, the Natural Sciences and Engineering Research Council of Canada.

Author information

Authors and Affiliations

  1. CENPARMI, Department of Computer Science and Software Engineering, Concordia University, Montreal, QC, H3G 1M8, Canada

    Elham Vahdati & Ching Y. Suen

Authors
  1. Elham Vahdati
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching Y. Suen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Elham Vahdati or Ching Y. Suen .

Editor information

Editors and Affiliations

  1. East China Normal University, Shanghai, China

    Yue Lu

  2. Paris Descartes University, Paris, France

    Nicole Vincent

  3. Hong Kong Baptist University, Kowloon, Hong Kong

    Pong Chi Yuen

  4. Sun Yat-sen University, Guangzhou, China

    Wei-Shi Zheng

  5. Polytechnique Montréal, Montreal, QC, Canada

    Farida Cheriet

  6. Concordia University, Montreal, QC, Canada

    Ching Y. Suen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vahdati, E., Suen, C.Y. (2020). Facial Beauty Prediction Using Transfer and Multi-task Learning Techniques. In: Lu, Y., Vincent, N., Yuen, P.C., Zheng, WS., Cheriet, F., Suen, C.Y. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2020. Lecture Notes in Computer Science(), vol 12068. Springer, Cham. https://doi.org/10.1007/978-3-030-59830-3_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59830-3_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59829-7

  • Online ISBN: 978-3-030-59830-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation