Skip to main content
SpringerLink
Log in

Efficient Group-Based Cohesion Prediction in Images Using Facial Descriptors

  • Conference paper
  • First Online:
Recent Trends in Analysis of Images, Social Networks and Texts (AIST 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1357))

Abstract

In this paper we study the problem of predicting the cohesiveness and emotion of a group of people in photo. We proposed a fast approach, consisting of face detection by using MTCNN, aggregation of facial features (age, gender and embeddings) extracted by multi-task MobileNet, prediction of group cohesion and classification of emotional background using multi-output convolution neural network. Experimental study on the Group Affect Dataset from EmotiW 2019 challenge demonstrated that our approach allows to achieve an improvement of quality and even to reduce the running time of an algorithm’s work when compared to known solutions. As a result, we obtained mean squared error 0.63 for cohesion prediction, which is 0.21 lower when compared to baseline CapsNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/gbcodes/GroupCohesiveness.

References

  1. Tarasov, A.V., Savchenko, A.V.: Emotion recognition of a group of people in video analytics using deep off-the-shelf image embeddings. In: van der Aalst, W.M.P., et al. (eds.) AIST 2018. LNCS, vol. 11179, pp. 191–198. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-11027-7_19

    Chapter  Google Scholar 

  2. Dhall, A., Goecke, R., Ghosh, S., Joshi, J., Hoey, J., Gedeon, T.: From individual to group-level emotion recognition: EmotiW 5.0. In: Proceedings of the 19th International Conference on Multimodal Interaction (ICMI), pp. 524–528. ACM (2017)

    Google Scholar 

  3. Ghosh, S., Dhall, A., Sebe, N., Gedeon, T.: Predicting group cohesiveness in images. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2019)

    Google Scholar 

  4. Zhu, B., Guo, X., Barner, K., Boncelet, C.: Automatic group cohesiveness detection with multi-modal features. In: Proceedings of the International Conference on Multimodal Interaction (ICMI), pp. 577–581. ACM (2019)

    Google Scholar 

  5. Zou, B., Lin, Z., Wang, H., Wang, Y., Lyu, X., Xie, H.: Joint prediction of group-level emotion and cohesiveness with multi-task loss. In: Proceedings of the 5th International Conference on Mathematics and Artificial Intelligence, pp. 24–28 (2020)

    Google Scholar 

  6. Xuan Dang, T., Kim, S.H., Yang, H.J., Lee, G.S., Vo, T.H.: Group-level cohesion prediction using deep learning models with a multi-stream hybrid network. In: Proceedings of the International Conference on Multimodal Interaction (ICMI), pp. 572–576. ACM (2019)

    Google Scholar 

  7. Guo, D., Wang, K., Yang, J., Zhang, K., Peng, X., Qiao, Y.: Exploring regularizations with face, body and image cues for group cohesion prediction. In: Proceedings of the International Conference on Multimodal Interaction (ICMI), pp. 557–561. ACM (2019)

    Google Scholar 

  8. Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  9. Savchenko, A.: Efficient statistical face recognition using trigonometric series and CNN features. In: Proceedings of the 24th International Conference on Pattern Recognition (ICPR), pp. 3262–3267. IEEE (2018)

    Google Scholar 

  10. Savchenko, A.V.: Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output convNet. PeerJ Comput. Sci. 5, e197 (2019)

    Article  Google Scholar 

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE (2016)

    Google Scholar 

  12. Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)

    Article  Google Scholar 

  13. Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: RetinaFace: single-shot multi-level face localisation in the wild. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5203–5212. IEEE (2020)

    Google Scholar 

  14. Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: Proceedings of 13th International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 67–74. IEEE (2018)

    Google Scholar 

  15. Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition (2015)

    Google Scholar 

  16. Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. In: Proceedings of International Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4690–4699. IEEE (2019)

    Google Scholar 

  17. Rassadin, A., Gruzdev, A., Savchenko, A.: Group-level emotion recognition using transfer learning from face identification. In: Proceedings of the 19th International Conference on Multimodal Interaction (ICMI), pp. 544–548. ACM (2017)

    Google Scholar 

  18. Pedregosa-Izquierdo, F.: Feature extraction and supervised learning on fMRI: from practice to theory. Ph.D. thesis (2015)

    Google Scholar 

  19. Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A.V., Gulin, A.: CatBoost: unbiased boosting with categorical features. In: Advances in Neural Information Processing Systems (NIPS), pp. 6638–6648 (2018)

    Google Scholar 

Download references

Acknowledgements

The work of A.V. Savchenko is supported by RSF (Russian Science Foundation) grant 20-71-10010.

Author information

Authors and Affiliations

  1. HSE University, Laboratory of Algorithms and Technologies for Network Analysis, Nizhny Novgorod, Russia

    Ilya Gavrikov & Andrey V. Savchenko

Authors
  1. Ilya Gavrikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrey V. Savchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ilya Gavrikov .

Editor information

Editors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Wil M. P. van der Aalst

  2. University of Ljubljana, Ljubljana, Slovenia

    Vladimir Batagelj

  3. National Research University Higher School of Economics, Perm, Russia

    Alexey Buzmakov

  4. National Research University Higher School of Economics, Moscow, Russia

    Dmitry I. Ignatov

  5. University of Melbourne, Melbourne, VIC, Australia

    Anna Kalenkova

  6. Krasovskii Institute of Mathematics and Mechanics of RAS, Ekaterinburg, Russia

    Michael Khachay

  7. National Research University Higher School of Economics, Saint-Petersburg, Russia

    Olessia Koltsova

  8. University of Oslo, Oslo, Norway

    Andrey Kutuzov

  9. National Research University Higher School of Economics, Moscow, Russia

    Sergei O. Kuznetsov

  10. National Research University Higher School of Economics, Moscow, Russia

    Irina A. Lomazova

  11. Lomonosov Moscow State University, Moscow, Russia

    Natalia Loukachevitch

  12. National Research University Higher School of Economics, Moscow, Russia

    Ilya Makarov

  13. LORIA, Vandœuvre-lès-Nancy, France

    Amedeo Napoli

  14. Skolkovo Institute of Science and Technology, Moscow, Russia

    Alexander Panchenko

  15. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  16. Università Ca’ Foscari Venezia, Venezia, Italy

    Marcello Pelillo

  17. National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey V. Savchenko

  18. Kazan Federal University, Kazan, Russia

    Elena Tutubalina

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gavrikov, I., Savchenko, A.V. (2021). Efficient Group-Based Cohesion Prediction in Images Using Facial Descriptors. In: van der Aalst, W.M.P., et al. Recent Trends in Analysis of Images, Social Networks and Texts. AIST 2020. Communications in Computer and Information Science, vol 1357. Springer, Cham. https://doi.org/10.1007/978-3-030-71214-3_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-71214-3_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-71213-6

  • Online ISBN: 978-3-030-71214-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation