Skip to main content
Springer Nature Link
Log in

Automatic Emotion Analysis in Movies: Matteo Garrone’s Dogman as a Case Study

  • Conference paper
  • First Online:
Deep Learning Theory and Applications (DeLTA 2024)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2172))

Included in the following conference series:

  • 295 Accesses

Abstract

This paper is the first step of an expansive ongoing initiative centered on automated film analysis through an ecocritical lens. Ecocriticism, an interdisciplinary field, delves into environmental themes within cultural works, broadening the scope of humanities’ focus on representation issues. Our objective is to pioneer a method for automated, dependable analysis of audiovisual narratives within fictional feature films, exploring the interplay between human emotions exhibited by characters and their surrounding environments. Using the acclaimed Italian crime/noir film, Dogman (2018), as a case study, we have constructed a modular pipeline integrating Facial Recognition and Emotion Detection technologies to scrutinize the emotional dynamics of the film’s two main characters. Our approach facilitates a comprehensive comparison over the film’s duration, enabling human analysts to future insights into the nuanced relationship between characters’ emotional states and the environmental contexts in which they unfold. Preliminary findings indicate promising outcomes from our pipeline, laying a solid foundation for subsequent film analyses. These results not only underscore the viability of automated methods in film studies but also offer a substantive starting point for deeper explorations into the complex interconnections between human emotions and cinematic environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.festival-cannes.com/en/f/dogman/.

References

  1. Anju Chandran, V.A.: Facial expression recognition using patch based Gabor features. Int. J. Appl. Inf. Syst. 10(7), 23–28 (2016)

    Google Scholar 

  2. Baracco, A., et al.: Film ecophilosophy and the question of the animal: Jacques Derrida’s cat and Matteo Garrone’s dogs. In: Italy and the Ecological Imagination. Ecocritical Theories and Practices, pp. 55–69. Vernon Press (2022)

    Google Scholar 

  3. Bengio, Y., Ducharme, R., Vincent, P.: A neural probabilistic language model. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Processing Systems, vol. 13. MIT Press (2000)

    Google Scholar 

  4. Brogi, D., et al.: Le verità dell’immaginazione. “dogman” (matteo garrone, 2018). DOPPIOZERO (2018)

    Google Scholar 

  5. Catolfi, A., et al.: Città e periferie nel cinema italiano contemporaneo: tra dogman e lo chiamavano jeeg robot. MIMESIS CINEMA, pp. 47–58 (2020)

    Google Scholar 

  6. Cerami, V.: Fattacci: il racconto di quattro delitti italiani. Garzanti (2020)

    Google Scholar 

  7. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the NAACL 2019, pp. 4171–4186 (2019)

    Google Scholar 

  8. Dosovitskiy, A., et al.: An image is worth \(16\times 16\) words: transformers for image recognition at scale. CoRR abs/2010.11929 (2020)

    Google Scholar 

  9. Ekman, P.: An argument for basic emotions. Cogn. Emot. 6, 169–200 (1992)

    Article  Google Scholar 

  10. Ekman, P., et al.: Basic emotions. In: Handbook of Cognition and Emotion, vol. 98, no. 45–60, p. 16 (1999)

    Google Scholar 

  11. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378–382 (1971)

    Article  Google Scholar 

  12. Goodfellow, I.J., et al.: Challenges in representation learning: a report on three machine learning contests. In: Lee, M., Hirose, A., Hou, Z.G., Kil, R.M. (eds.) ICONIP 2013. LNCS, vol. 8228, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-42051-1_16

    Chapter  Google Scholar 

  13. Gu, M., et al.: Hierarchical attention network for interpretable and fine-grained vulnerability detection, pp. 1–6 (2022)

    Google Scholar 

  14. Hao, Y., et al.: Language models are general-purpose interfaces. ArXiv abs/2206.06336 (2022)

    Google Scholar 

  15. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  16. Huang, S., et al.: Language is not all you need: aligning perception with language models. ArXiv abs/2302.14045 (2023)

    Google Scholar 

  17. Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics YOLO (2023). https://github.com/ultralytics/ultralytics

  18. Johannessen, R.: Space and reality in the cinematic city. Matteo Garrone’s early cinema and dogman. L’avventura 6(Speciale), 55–68 (2020)

    Google Scholar 

  19. Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–74 (1977)

    Article  Google Scholar 

  20. Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. CoRR abs/1910.13461 (2019)

    Google Scholar 

  21. Liu, B., Hu, M., Cheng, J.: Opinion observer: analyzing and comparing opinions on the web. In: Proceedings of the 14th International Conference on World Wide Web, WWW 2005, pp. 342–351. Association for Computing Machinery, New York (2005)

    Google Scholar 

  22. Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning (2023)

    Google Scholar 

  23. Liu, P., Han, S., Meng, Z., Tong, Y.: Facial expression recognition via a boosted deep belief network. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1805–1812 (2014)

    Google Scholar 

  24. Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019)

    Google Scholar 

  25. Lozier, L.M., Vanmeter, J.W., Marsh, A.A.: Impairments in facial affect recognition associated with autism spectrum disorders: a meta-analysis. Dev. Psychopathol. 26(4pt1), 933–945 (2014)

    Google Scholar 

  26. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, pp. 94–101 (2010)

    Google Scholar 

  27. Machajdik, J., Hanbury, A.: Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM International Conference on Multimedia (2010)

    Google Scholar 

  28. Mahmood, A., Hussain, S., Iqbal, K., Elkilani, W.S.: Recognition of facial expressions under varying conditions using dual-feature fusion. Math. Probl. Eng. 2019(1), 9185481 (2019)

    Article  Google Scholar 

  29. McDuff, D., El Kaliouby, R., Picard, R.W.: Crowdsourcing facial responses to online videos: extended abstract. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 512–518 (2015)

    Google Scholar 

  30. Meeker, J.W.: The comedy of survival: studies in literary ecology. Scribner (1974)

    Google Scholar 

  31. Mohammad, S.M., Turney, P.D.: Crowdsourcing a word-emotion association lexicon (2013)

    Google Scholar 

  32. Mollahosseini, A., Chan, D., Mahoor, M.H.: Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE (2016)

    Google Scholar 

  33. Mollahosseini, A., Hasani, B., Mahoor, M.H.: AffectNet: a database for facial expression, valence, and arousal computing in the wild. IEEE Trans. Affect. Comput. 10(1), 18–31 (2019)

    Article  Google Scholar 

  34. Moretti, L.: Il Canaro. Magliana 1988: storia di una vendetta. Red Star Press, Roma (2018)

    Google Scholar 

  35. Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(1–2), 1–135 (2008)

    Article  Google Scholar 

  36. Parrott, W.G. (ed.): Emotions in Social Psychology: Key Readings. Psychology Press (2000)

    Google Scholar 

  37. Peng, K.C., Chen, T., Sadovnik, A., Gallagher, A.C.: A mixed bag of emotions: model, predict, and transfer emotion distributions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 860–868 (2015)

    Google Scholar 

  38. Peng, Z., et al.: Kosmos-2: grounding multimodal large language models to the world. ArXiv abs/2306 (2023)

    Google Scholar 

  39. Pezzotti, B.: Towards a definition of mediterranean noir or crime in the mediterranean: mediterranean noir or mediterranean crime fiction? Belphégor, Littérature populaire et culture médiatique (2022)

    Google Scholar 

  40. Plutchik, R.: The Emotions. University Press of America (1991)

    Google Scholar 

  41. Radford, A., et al.: Learning transferable visual models from natural language supervision. CoRR abs/2103.00020 (2021)

    Google Scholar 

  42. Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)

    Google Scholar 

  43. Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21, 140:1–140:67 (2020)

    Google Scholar 

  44. Rueckert, W.: Literature and ecology: an experiment in ecocriticism. In: The Ecocriticism Reader: Landmarks in Literary Ecology (1996)

    Google Scholar 

  45. Rust, S., Monani, S., Cubitt, S.: Ecocinema Theory and Practice. Routledge (2013)

    Google Scholar 

  46. Salvatore, R.: L’abbraccio dell’altro come velo sul reale: la scrittura per immagini di matteo garrone. Ágalma: rivista di studi culturali e di estetica 37(1), 80–90 (2019)

    Google Scholar 

  47. Serengil, S.I., Ozpinar, A.: Hyperextended lightface: a facial attribute analysis framework. In: 2021 International Conference on Engineering and Emerging Technologies (ICEET), pp. 1–4. IEEE (2021)

    Google Scholar 

  48. Shriver-Rice, M., Vaughan, H.: What is environmental media studies? (2020)

    Google Scholar 

  49. Siersdorfer, S., Hare, J., Minack, E., Deng, F.: Analyzing and predicting sentiment of images on the social web. In: ACM Multimedia 2010, 25–29 October 2010, pp. 715–718 (2010)

    Google Scholar 

  50. Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45, 427–437 (2009)

    Article  Google Scholar 

  51. Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)

    Google Scholar 

  52. Tonguç, G., Ozkara, B.O.: Automatic recognition of student emotions from facial expressions during a lecture. Comput. Educ. 148, 103797 (2020)

    Article  Google Scholar 

  53. Touvron, H., et al.: LLaMA: open and efficient foundation language models (2023)

    Google Scholar 

  54. Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)

    Google Scholar 

  55. Yang, J., She, D., Sun, M.: Joint image emotion classification and distribution learning via deep convolutional neural network. In: International Joint Conference on Artificial Intelligence (2017)

    Google Scholar 

  56. Yang, J., Sun, M., Sun, X.: Learning visual sentiment distributions via augmented conditional probability neural network. In: AAAI Conference on Artificial Intelligence (2017)

    Google Scholar 

  57. Yu, Z., Zhang, C.: Image based static facial expression recognition with multiple deep network learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 435–442. Association for Computing Machinery, New York (2015)

    Google Scholar 

  58. Yuan, J., Mcdonough, S., You, Q., Luo, J.: Sentribute: image sentiment analysis from a mid-level perspective. In: WISDOM (2013)

    Google Scholar 

  59. Zheng, L., et al.: Judging LLM-as-a-judge with MT-bench and chatbot arena. In: Advances in Neural Information Processing Systems, vol. 36 (2024)

    Google Scholar 

Download references

Acknowledgments

The essay is the result of collective work and discussion. Each author participated in every phase of discussion, development, validation and writing of this work. In particular, Alessia Forciniti, Claudiu Daniel Hromei and Daniele Margiotta wrote Sections 1–6, Stefano Locati wrote Sections 1 and 6.

Claudiu Daniel Hromei is a Ph.D. student enrolled in the National Ph.D. in Artificial Intelligence, XXXVII cycle, course on Health and life sciences, organized by the Università Campus Bio-Medico di Roma.

Author information

Authors and Affiliations

  1. Department of Enterprise Engineering, University of Rome Tor Vergata, Rome, Italy

    Claudiu Daniel Hromei

  2. Reveal S.r.l. - Wecom Group, Rome, Italy

    Daniele Margiotta

  3. Department of Humanistic Studies, IULM University, Milan, Italy

    Alessia Forciniti

  4. Department of Communication, Arts and Media, IULM University, via Carlo Bo 1, 20143, Milan, Italy

    Stefano Locati

Authors
  1. Claudiu Daniel Hromei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessia Forciniti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniele Margiotta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefano Locati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Claudiu Daniel Hromei .

Editor information

Editors and Affiliations

  1. Instituto de Telecomunicações and University of Lisbon, Lisbon, Portugal

    Ana Fred

  2. LIAS, LIAS/ENSMA, Poitiers, France

    Allel Hadjali

  3. Ford Motor Company, Dearborn, MI, USA

    Oleg Gusikhin

  4. University of Naples Federico II, Naples, Italy

    Carlo Sansone

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hromei, C.D., Forciniti, A., Margiotta, D., Locati, S. (2024). Automatic Emotion Analysis in Movies: Matteo Garrone’s Dogman as a Case Study. In: Fred, A., Hadjali, A., Gusikhin, O., Sansone, C. (eds) Deep Learning Theory and Applications. DeLTA 2024. Communications in Computer and Information Science, vol 2172. Springer, Cham. https://doi.org/10.1007/978-3-031-66705-3_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-66705-3_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-66704-6

  • Online ISBN: 978-3-031-66705-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics