Skip to main content
SpringerLink
Log in

A movie box office revenue prediction model based on deep multimodal features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Demand forecasting a film’s opening weekend box office revenue is a difficult and complex task that decision-makers face due to a lack of historical data and various complex factors. We proposed a novel Deep Multimodal Feature Classifier Neural Network model (DMFCNN) for predicting a film’s opening weekend box office revenue using deep multimodal visual features extracted from movie posters and movie metadata. DMFCNN is an end-to-end predictive model that fuses two different feature classifiers’ predictive power in estimating the movie box office revenue. Initially, a pre-trained residual convolutional neural network (ResNet50) architecture using transfer learning techniques extracts visual, and object representations learned from movie posters. The movie posters’ discriminative and financial success-related features are combined with other movie metadata to classify the movie box office revenue. The proposed DMFCNN aided in developing a robust predictive model that jointly learns and defines useful revenue-related poster features and objects semantics, which strongly correlates with movie box office revenue and aesthetic appearance. Although our main task was classification, we also analyzed regressions between our exogenous variables as a regularizer to avoid the risk of overfitting. We evaluated DMFCNN’s performance and compared it to various state-of-the-art models on the Internet Movie Database by collecting 49,857 movies metadata and posters from 2006 to 2019. The learned information on movie posters and predicted outcomes outperformed existing models, achieving 59.30% prediction accuracy. The proposed fusion strategy outperformed the existing fusion schemes in precision, Area Under Cover, sensitivity, and specificity by achieving 80%, 81%, 79%, and 78%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Abadi M et al (2016) "TensorFlow: A system for large-scale machine learning", In: Proc 12th USENIX Sympo Operating Syst Design Implemen, OSDI 2016. https://doi.org/10.48550/arXiv.1605.08695

  2. Ahmed U, Waqas H, Afzal MT (2020) Pre-production box-office success quotient forecasting. Soft Comput 24(9):6635–6653. https://doi.org/10.1007/s00500-019-04303-w

    Article  Google Scholar 

  3. Barney G, Kaya K (2019) "Predicting genre from movie posters", Semant. Sch.

    Google Scholar 

  4. Beck J (2011) The sales effect of word of mouth: a model for creative goods and estimates for novels," SSRN Electron. J. https://doi.org/10.2139/ssrn.931382

    Book  Google Scholar 

  5. Chang CC, Lin CJ (2011) "LIBSVM: a library for support vector machines", ACM Trans. Intell. Syst. Technol. https://doi.org/10.1145/1961189.1961199

  6. Chen T, Guestrin C (2016) "XGBoost: a scalable tree boosting system", https://doi.org/10.1145/2939672.2939785

  7. Chu WT, Guo HJ "movie genre classification based on poster images with deep neural networks," MUSA2 2017 - proc. Work Multimodal Underst Soc Affect Subj Attrib co-located with MM 2017:39–45. https://doi.org/10.1145/3132515.3132516

  8. Delen D, Sharda R, Kumar P (2007) Movie forecast guru: a web-based DSS for Hollywood managers. Decis Support Syst 43(4):1151–1170. https://doi.org/10.1016/j.dss.2005.07.005

    Article  Google Scholar 

  9. Dellarocas CN, Awad N, Zhang XM (2011) "Using Online Reviews as a Proxy of Word-of-Mouth for Motion Picture Revenue Forecasting", SSRN Electron. J. https://doi.org/10.2139/ssrn.620821

  10. Ghiassi M, Lio D, Moon B (2015) Pre-production forecasting of movie revenues with a dynamic artificial neural network. Expert Syst Appl 42(6):3176–3193. https://doi.org/10.1016/j.eswa.2014.11.022

    Article  Google Scholar 

  11. He K, Zhang X, Ren S, Sun J (2016) "Deep residual learning for image recognition," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90

  12. Ho TK (1995) "Random decision forests," In: Proc Int Conf Doc Anal Recog, ICDAR. https://doi.org/10.1109/ICDAR.1995.598994

  13. Hur M, Kang P, Cho S (2016) "Box-office forecasting based on sentiments of movie reviews and Independent subspace method", Inf. Sci. (Ny)., vol. 372, pp. 608–624, https://doi.org/10.1016/j.ins.2016.08.027

  14. Ivasic-Kos M, Pobar M, Mikec L (2014) Movie posters classification into genres based on low-level features. 2014 37th Int Conv Inf Commun Technol Electron Microelectron MIPRO 2014 - Proc (i, May):1198–1203. https://doi.org/10.1109/MIPRO.2014.6859750

  15. Ivasic-Kos M, Pobar M, Ipsic I (2015) "Automatic movie posters classification into genres," In: Adv Intel Syst Comput. https://doi.org/10.1007/978-3-319-09879-1_32

  16. Kim T, Hong J, Kang P (2015) Box office forecasting using machine learning algorithms based on SNS data. Int J Forecast 31(2):364–390. https://doi.org/10.1016/j.ijforecast.2014.05.006

    Article  Google Scholar 

  17. Kim T, Hong J, Kang P (2017) "Box Office Forecasting considering Competitive Environment and Word-of-Mouth in Social Networks: A Case Study of Korean Film Market", Comput. Intell. Neurosci., vol. 2017. https://doi.org/10.1155/2017/4315419

  18. Krizhevsky A, Sutskever I, Hinton GE (2017) "ImageNet Classification with Deep Convolutional Neural Networks," in. In; Proc Adv Neural Inform Proc Syst. Commun. ACM 60, 6, 2017, pp. 84–90. https://doi.org/10.1145/3065386

  19. Kuznetsova A, Rom H, Alldrin N, Uijlings J, Krasin I, Pont-Tuset J, Kamali S, Popov S, Malloci M, Kolesnikov A, Duerig T, Ferrari V (2020) The open images dataset V4: unified image classification, object detection, and visual relationship detection at scale. Int J Comput Vis 128(7):1956–1981. https://doi.org/10.1007/s11263-020-01316-z

    Article  Google Scholar 

  20. Lecun Y, Bengio Y, Hinton G (2015) "Deep Learning," Nature. https://doi.org/10.1038/nature14539

  21. Lee KJ , Chang W (2009) "Bayesian belief network for box-office performance: a case study on Korean movies", Expert Syst Appl. https://doi.org/10.1016/j.eswa.2007.09.042

  22. Mangolin RB et al (2020) "A multimodal approach for multi-label movie genre classification", Multimed. Tools Appl. https://doi.org/10.1007/s11042-020-10086-2

  23. Matsuzaki Y et al (2017) "Could you guess an interesting movie from the posters?: An evaluation of vision-based features on movie poster database", Proc. 15th IAPR Int. Conf. Mach. Vis. Appl. MVA 2017, pp. 538–541. https://doi.org/10.23919/MVA.2017.7986919

  24. Moreno-Seco F, Iñesta JM, Ponce De León PJ, Micó L (2006) "Comparison of classifier fusion methods for classification in pattern recognition tasks," in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4109 LNCS. https://doi.org/10.1007/11815921_77

  25. Nambiar G, Roy P, Singh D (2020) "Multi modal genre classification of movies", 2020 IEEE Int. Conf Innov Technol INOCON pp. 1–6, 2020, https://doi.org/10.1109/INOCON50539.2020.9298385

  26. Ozkan K, Atak ON, Isik S (2018) "using movie posters for prediction of box-office revenue with deep learning approach," 26th IEEE signal process. Commun Appl Conf SIU 2018:1–4. https://doi.org/10.1109/SIU.2018.8404649

    Article  Google Scholar 

  27. Redmon J, Divvala S, Girshick R, Farhadi A (2016) "You only look once: Unified, real-time object detection", In: Proc IEEE Comput Soc Conf Comput Vision Pattern Recog. pp. 779–788 https://doi.org/10.1109/CVPR.2016.91.

  28. Ru Y, Li B, Liu J, Chai J (2018) An effective daily box office prediction model based on deep neural networks. Cogn Syst Res 52:182–191. https://doi.org/10.1016/j.cogsys.2018.06.018

    Article  Google Scholar 

  29. Sharda R, Delen D (2006) Predicting box-office success of motion pictures with neural networks. Expert Syst Appl 30(2):243–254. https://doi.org/10.1016/j.eswa.2005.07.018

    Article  Google Scholar 

  30. Sirattanajakarin S, Thusaranon P (2019) "Movie genre in multi-label classification using semantic extraction from only movie poster", in ACM Int Conf Proc Series pp. 23–27. https://doi.org/10.1145/3348445.3348475

  31. Tang Z, Dong S (2020) A total sales forecasting method for a new short life-cycle product in the pre-market period based on an improved evidence theory: application to the film industry. Int J Prod Res 0(0):1–15. https://doi.org/10.1080/00207543.2020.1825861

    Article  Google Scholar 

  32. Wang F, Zhang Y, Li X, Zhu H (2010) "Why do moviegoers go to the theater? The role of prerelease media publicity and online word of mouth in driving Moviegoing behavior", J. Interact. Advert. https://doi.org/10.1080/15252019.2010.10722177.

  33. Wang W, Xiu J, Yang Z, Liu C (2018) "A deep learning model for predicting movie box office based on deep belief network," in lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) https://doi.org/10.1007/978-3-319-93818-9_51

  34. Wang Z, Zhang J, Ji S, Meng C, Li T, Zheng Y (2020) Predicting and ranking box office revenue of movies based on big data. Inf Fusion 60(June 2019):25–40. https://doi.org/10.1016/j.inffus.2020.02.002

    Article  Google Scholar 

  35. Wehrmann J, Barros RC (2017) "Convolutions through time for multi-label movie genre classification," Proc. ACM Symp. Appl. Comput., vol. Part F1280, pp. 114–119. https://doi.org/10.1145/3019612.3019641

  36. Wehrmann J, Barros RC (2017) Movie genre classification: a multi-label approach based on convolutions through time. Appl Soft Comput J 61:973–982. https://doi.org/10.1016/j.asoc.2017.08.029

    Article  Google Scholar 

  37. Wi JA, Jang S, Kim Y (2020) Poster-based multiple movie genre classification using Inter-Channel features. IEEE Access 8:66615–66624. https://doi.org/10.1109/ACCESS.2020.2986055

    Article  Google Scholar 

  38. Zhang L, Luo J, Yang S (2009) Forecasting box office revenue of movies with BP neural network. Expert Syst Appl 36(3 PART 2):6580–6587. https://doi.org/10.1016/j.eswa.2008.07.064

    Article  Google Scholar 

  39. Zhou Y, Yen GG (2018) Evolving Deep Neural Networks for Movie Box-Office Revenues Prediction. IEEE Congr Evol Comput CEC 2018 - Proc 2018:1–8. https://doi.org/10.1109/CEC.2018.8477691

    Article  Google Scholar 

  40. Zhou H, Hermans T, Karandikar AV, Rehg JM (2010) "Movie genre classification via scene categorization," MM'10 - Proc. ACM Multimed. Int. Conf., pp. 747–750, 2010 https://doi.org/10.1145/1873951.1874068

  41. Zhou Y, Zhang L, Yi Z (2017) Predicting movie box-office revenues using deep neural networks. Neural Comput Appl 31(6):1855–1865. https://doi.org/10.1007/s00521-017-3162-x

    Article  Google Scholar 

Download references

Availability of data and material

The data that support the findings of this study are openly available in Mendeley Data V2 at https://doi.org/10.17632/xv9wtc9gdk.2 . Code has not been made available to the public.

Funding

This work is supported by the National Nature Science Foundation of China under Grant No. 71672004.

Author information

Authors and Affiliations

  1. Research Base of Beijing Modern Manufacturing Development, College of Economics and Administration, Beijing University of Technology, No. 100 Pingleyuan, Beijing, 100124, People’s Republic of China

    Canaan Tinotenda Madongo & Tang Zhongjun

Authors
  1. Canaan Tinotenda Madongo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tang Zhongjun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Canaan Tinotenda Madongo.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Madongo, C.T., Zhongjun, T. A movie box office revenue prediction model based on deep multimodal features. Multimed Tools Appl 82, 31981–32009 (2023). https://doi.org/10.1007/s11042-023-14456-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14456-4

Keywords

Search

Navigation