Skip to main content

Advertisement

SpringerLink
Log in

A survey on video content rating: taxonomy, challenges and open issues

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Rating a video based on its content is one of the most important solutions to classify videos for audience age groups. In this regard, Film content rating and TV programmes rating are the only two most common rating systems which have been accomplished by the professional committees. However, due to the huge number of short videos shared in social media, it is impossible to review and rate their contents manually by a committee. Therefore, a proper solution is by utilizing computer vision capabilities to analyze the video content and rate it. Automatic Video Content Rating (VCR) system rates a short video to classify it for audience age groups. Inspired by the current manually film and TV programmes rating systems, VCR depends on five main components that comprise violence, profanity language, nudity, pornography, and substance abuse. To date, several reviews and survey papers have addressed advancements and innovations in video content analysis such as violence, nudity, and pornography detection. However, the lack of a comprehensive survey paper to investigate a VCR system and explain taxonomy, challenges, and open issues is discovered; thus, this study is undertaken. In this paper, in addition, to fill this gap, we review deep learning studies related to the relevant subjects of VCR. Moreover, we have investigated recently published works related to VCR based on the audio, static visual and, motion visual aspects of a video. Furthermore, related current datasets are investigated as well as the performances of published models in these datasets are compared. Finally, the challenges and the future of VCR are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Villani S (2001) Impact of media on children and adolescents: a 10-year review of the research. J Am Acad Child Adolesc Psychiatry 40(4):392–401

    Article  Google Scholar 

  2. Short MB, Black L, Smith AH, Wetterneck CT, Wells DE (2012) A review of internet pornography use research: methodology and content from the past 10 years. Cyberpsychol Behav Soc Netw 15(1):13–23

    Article  Google Scholar 

  3. Ofcom (2018) Children and parents: Media use and attitudes report 2018.

  4. Akram W, Kumar R (2018) A study on positive and negative effects of social media on society. INTERNATIONAL JOURNAL OF COMPUTER SCIENCES AND ENGINEERING, 5(10)

  5. Lenhart A (2015) Teens, Social Media & Technology Overview. APRIL 9, 2015]; Available from: http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/

  6. YouTube (2018) YouTube content rating. [cited 2019; Available from: https://support.google.com/youtube/answer/146399

  7. Thrasher JF, Sargent JD, Vargas R, Braun S, Barrientos-Gutierrez T, Sevigny EL, Billings DL, Arillo-Santillan E, Navarro A, Hardin J (2014) Are movies with tobacco, alcohol, drugs, sex, and violence rated for youth? A comparison of rating systems in Argentina, Brazil, Mexico, and the United States. Int J Drug Policy 25(2):267–275

    Article  Google Scholar 

  8. Ali A, Senan N (2017) A Review on Violence Video Classification Using Convolutional Neural Networks, in Recent Advances on Soft Computing and Data Mining. p. 130-140

  9. Gangwar A, Fidalgo E, Alegre E, González-Castro V (2017) Pornography and Child Sexual Abuse Detection in Image and Video: A Comparative Evaluation, in 8th International Conference on Imaging for Crime Detection and Prevention (ICDP 2017): Madrid, Spain

  10. Zhang H-B, Zhang Y-X, Zhong B, Lei Q, Yang L, Du J-X, Chen D-S (2019) A comprehensive survey of vision-based human action recognition methods. Sensors (Basel), 19(5)

  11. Kaushal M, Khehra BS, Sharma A (2018) Soft computing based object detection and tracking approaches: state-of-the-art survey. Appl Soft Comput 70:423–464

    Article  Google Scholar 

  12. Subetha T, Chitrakala S (2016) A survey on human activity recognition from videos, in 2016 International Conference on Information Communication and Embedded Systems (ICICES). p. 1-7

  13. Yu H, Yang Z, Tan L, Wang Y, Sun W, Sun M, Tang Y (2018) Methods and datasets on semantic segmentation: a review. Neurocomputing 304:82–103

    Article  Google Scholar 

  14. Karamizadeh S, Arabsorkhi A (2018) Methods of Pornography Detection, in Proceedings of the 10th International Conference on Computer Modeling and Simulation - ICCMS 2018. p. 33-38

  15. Lateef F, Ruichek Y (2019) Survey on semantic segmentation using deep learning techniques. Neurocomputing 338:321–348

    Article  Google Scholar 

  16. MPAA, NATO (2010) Classification and rating rules.

  17. TV Parental Guidelines Monitoring Board (2018) TV Parental Guidelines. 03.02.2019 [cited 2019 03 Feb]; Available from: http://www.tvguidelines.org

  18. The British Board of Film Classification (2016) The BBFC's Classification Guidelines

  19. Amazon (2020) Internet Movie Database (IMDB). [cited 2020; IMDb (Internet Movie Database) is an online database of information related to films, television programs, home videos, video games, and streaming content online – including cast, production crew and personal biographies, plot summaries, trivia, ratings, and fan and critical reviews.]. Available from: https://www.imdb.com/

  20. Teeni-Harari T, Yadin S (2019) Regulatory Literacy: Rethinking Television Rating in the New Media Age 88(2)

  21. Tu Z, Xie W, Qin Q, Poppe R, Veltkamp RC, Li B, Yuan J (2018) Multi-stream CNN: learning representations based on human-related regions for action recognition. Pattern Recogn 79:32–43

    Article  Google Scholar 

  22. Caetano C, Avila S, Schwartz W R, Guimarães S J F, Araújo A d A (2016) A mid-level video representation based on binary descriptors: a case study for pornography detection. Neurocomputing, 213: p. 102–114.

  23. Nurhadiyatn A, Cahyadi S, Damatraseta F, Rianto Y (2017) Adult Content Classification Through Deep Convolution Neural Network. International conference on computer, control, informatics and its applications

  24. Wehrmann J, Simões GS, Barros RC, Cavalcante VF (2018) Adult content detection in videos with convolutional and recurrent neural networks. Neurocomputing 272:432–438

    Article  Google Scholar 

  25. Moreira D, Avila S, Perez M, Moraes D, Testoni V, Valle E, Goldenstein S, Rocha A (2016) Pornography classification: the hidden clues in video space-time. Forensic Sci Int 268:46–61

    Article  Google Scholar 

  26. Perez M, Avila S, Moreira D, Moraes D, Testoni V, Valle E, Goldenstein S, Rocha A (2017) Video pornography detection through deep learning techniques and motion information. Neurocomputing 230:279–293

    Article  Google Scholar 

  27. Vitorino P, Avila S, Perez M, Rocha A (2018) Leveraging deep neural networks to fight child pornography in the age of social media. J Vis Commun Image Represent 50:303–313

    Article  Google Scholar 

  28. Ribeiro PC, Audigier R, Pham QC (2016) RIMOC, a feature to discriminate unstructured motions: application to violence detection for video-surveillance. Comput Vis Image Underst 144:121–143

    Article  Google Scholar 

  29. Zhang T, Jia W, Gong C, Sun J, Song X (2018) Semi-supervised dictionary learning via local sparse constraints for violence detection. Pattern Recogn Lett 107:98–104

    Article  Google Scholar 

  30. Walsh J, Mahony N, Campbell S, Carvalho A (2019) Deep Learning vs. Traditional Computer Vision, in Computer Vision Conference (CVC): Las Vegas, Nevada, United States

  31. Gibert D, Mateu C, Planes J (2020) The rise of machine learning for detection and classification of malware: research developments, trends and challenges. J Netw Comput Appl 153:102526

    Article  Google Scholar 

  32. Tahaei H, Afifia F, Asemia A, Zaki F, Anuar NB (2020) The rise of traffic classification in IoT networks: a survey. J Netw Comput Appl 154:102538

    Article  Google Scholar 

  33. Nie L, Jiang D, Guo L, Yu S (2016) Traffic matrix prediction and estimation based on deep learning in large-scale IP backbone networks. J Netw Comput Appl 76:16–22

    Article  Google Scholar 

  34. Zhang W, Wulan G, Zhai J, Xu L, Zhao D, Liu X, Yang S, Zhoud J (2018) An intelligent power distribution service architecture using cloud computing and deep learning techniques. J Netw Comput Appl 103:239–248

    Article  Google Scholar 

  35. Chen F, Fu Z, Zhen L (2019) Thermal power generation fault diagnosis and prediction model based on deep learning and multimedia systems. Multimed Tools Appl 78(4):4673–4692

    Article  Google Scholar 

  36. Prates RM, Cruz R, Marotta AP, Ramos RP, Filho EFS, Cardoso JS (2019) Insulator visual non-conformity detection in overhead power distribution lines using deep learning. Computers & Electrical Engineering 78:343–355

    Article  Google Scholar 

  37. Sorin V, Barash Y, Konen E, Klang E (2020) Deep learning for natural language processing in radiology—fundamentals and a systematic review. J Am Coll Radiol 17:639–648

    Article  Google Scholar 

  38. Ayadi A, Samet A, FDB d B, Zanni-Merk C (2019) Ontology population with deep learning-based NLP: a case study on the biomolecular network ontology. Procedia Computer Science 159:572–581

    Article  Google Scholar 

  39. Hu T, Liu L, Si W (2020) Special issue on deep learning in biomedical signal and medical image processing [1128 T]. Multimed Tools Appl 79(13):9161–9161

    Article  Google Scholar 

  40. (2020) Deep learning algorithms for medical image processing. Multimedia Tools and Applications.

  41. Gupt O, Raskar R (2018) Distributed learning of deep neural network over multiple agents. J Netw Comput Appl 116:1–8

    Article  Google Scholar 

  42. Zhang Q, Bai C, Chen Z, Li P, Wang S, Gao H (2019) Smart Chinese medicine for hypertension treatment with a deep learning model. J Netw Comput Appl 129:1–8

    Article  Google Scholar 

  43. Zhang X, Han Y, Wei X, Wang Q (2019) HOBA: a novel feature engineering methodology for credit card fraud detection with a deep learning architecture. Inf Sci

  44. Kim E, Lee J, Shin H, Yang H, Cho S, S-k N, Song Y, J-a Y, J-i K (2019) Champion-challenger analysis for credit card fraud detection: hybrid ensemble and deep learning. Expert Syst Appl 128:214–224

    Article  Google Scholar 

  45. Li B, Zhao J, Fu H (2020) DLT-net: deep learning transmittance network for single image haze removal. Signal, Image and Video Processing

  46. Algarni AD (2020) Efficient object detection and classification of heat emitting objects from infrared images based on deep learning. Multimed Tools Appl 79:13403–13426

    Article  Google Scholar 

  47. Abbas Q, Ibrahim MEA, Jaffar MA (2018) Video scene analysis: an overview and challenges on deep learning algorithms. Multimed Tools Appl 77(16):20415–20453

    Article  Google Scholar 

  48. Soon FC, Khaw HY, Chuah JH, Kanesan J (2019) Vehicle logo recognition using whitening transformation and deep learning. SIViP 13(1):111–119

    Article  Google Scholar 

  49. Moustafa MN (2015) Applying deep learning to classify pornographic images and videos. arXiv:1511.08899v1 [cs.CV].

  50. Ou X, Ling H, Yu H, Li P, Zou F, Liu S (2017) Adult image and video recognition by a deep multicontext network and fine-to-coarse strategy. ACM Trans Intell Syst Technol 8(5):1–25

    Article  Google Scholar 

  51. Sernani P, Falcionelli N, Mekuria D N, Dragoni A F (2020) Violence detection in videos by combining 3D convolutional neural networks and support vector machines. Appl Artif Intell

  52. Ullah F U M, Ullah A, Muhammad K, Haq I U, Baik S W (2019) Violence detection using spatiotemporal features with 3D convolutional neural network. Sensors (Basel), 19(11).

  53. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  54. Davis S B, Mermelstein P (1980) Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Sentences 28(4): p. 357–366.

  55. Xu M, Duan L-Y, Cai J, Chia L-T, Xu C, Tian Q (2004) HMM-Based Audio Keyword Generation, in Advances in Multimedia Information Processing - PCM 2004, 5th Pacific Rim: Tokyo, Japan

  56. Wang H, Kläser A, Schmid C, Liu C-L (2013) Dense Trajectories and Motion Boundary Descriptors for Action Recognition. International Journal of computer vision, 103

  57. Sun Y, Wu X, Yu W, Yu F (2018) Action recognition with motion map 3D network. Neurocomputing 297:33–39

    Article  Google Scholar 

  58. Caetano C, Avila S, Guimar S, de a Ara'ujo a (2014) Pornography Detection using BossaNova Video Descriptor. 22nd European Signal Processing Conference (EUSIPCO)

  59. Liu Y, Yang Y, Xie H, Tang S (2014) Fusing audio vocabulary with visual features for pornographic video detection. Futur Gener Comput Syst 31:69–76

    Article  Google Scholar 

  60. Liu Y Z, Ouyang J L, Liu J X (2017) Bimodal Codebooks Based Adult Video Detection. 2017 Ieee global conference on signal and information processing (Globalsip 2017): p. 1397-1400

  61. Moreira D, Avila S, Perez M, Moraes D, Testoni V, Valle E, Goldenstein S, Rocha A (2019) Multimodal data fusion for sensitive scene localization. Information Fusion 45:307–323

    Article  Google Scholar 

  62. Gao Y, Liu H, Sun X, Wang C, Liu Y (2016) Violence detection using oriented VIolent flows. Image Vis Comput 48-49:37–41

    Article  Google Scholar 

  63. Ben Mabrouk A, Zagrouba E (2017) Spatio-temporal feature using optical flow based distribution for violence detection. Pattern Recogn Lett 92:62–67

    Article  Google Scholar 

  64. Mahmoodi J, Salajeghe A (2019) A classification method based on optical flow for violence detection. Expert Syst Appl 127:121–127

    Article  Google Scholar 

  65. Acar E, Hopfgartner F, Albayrak S (2016) Breaking down violence detection: combining divide-et-impera and coarse-to-fine strategies. Neurocomputing 208:225–237

    Article  Google Scholar 

  66. Avila S, Thome N, Cord M, Valle E, de A Araújo A (2013) Pooling in image representation: the visual codeword point of view. Comput Vis Image Underst 117(5):453–465

    Article  Google Scholar 

  67. Nievas EBAS, Deniz O, Garcia GB, Sukthankar R (2011) Hockey fight detection dataset. Computer Analysis of Images and Patterns: p:332–339

  68. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent Flows: Real-Time Detection of Violent Crowd Behavior, in 3rd IEEE International Workshop on Socially Intelligent Surveillance and Monitoring (SISM) at the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR): Rhode Island

  69. Demarty C-H, Penet C, Soleymani M, Gravier G (2014) VSD, a public dataset for the detection of violent scenes in movies: design, annotation, analysis and evaluation. Multimed Tools Appl, 74.

  70. Demarty C, Ionescu B, Jiang Y, Quang V L, Schedl M, Penet C (Year) Benchmarking Violent Scenes Detection in movies. in 2014 12th International Workshop on Content-Based Multimedia Indexing (CBMI). of Conference.

  71. Sjöberg M, Ionescu B, Jiang Y-G, Quang V L, Schedl M, Demarty C-H ( 2014) The MediaEval 2014 Affect Task: Violent Scenes Detection, in In Working Notes Proceedings of the MediaEval 2014 Workshop: Barcelona, Spain

  72. Demarty C-H, Penet C, Gravier G, Soleymani M (year) A Benchmarking Campaign for the Multimodal Detection of Violent Scenes in Movies. In Computer Vision – ECCV 2012. Workshops and Demonstrations. Of conference. Berlin, Heidelberg: springer Berlin Heidelberg

  73. Yeffet L, Wolf L (2009) Local Trinary Patterns for human action recognition, in 2009 IEEE 12th International Conference on Computer Vision. Kyoto, IEEE

    Google Scholar 

  74. Hassner T, Itcher Y, Kliper-Gross O (2012) Violent flows: Real-time detection of violent crowd behavior, in 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. IEEE: Providence, RI, USA.

  75. Keslassy E (2016) Conservative Group, Filmmakers at Odds Over Movie Ratings in France. MARCH 25, 2016]; Available from: https://variety.com/2016/film/global/france-movie-ratings-1201737635/.

  76. Steven (2014) Movie ratings in Germany. 17 Feb 2014; Available from: https://stevenglassman.com/2014/02/17/movie-ratings-in-germany/

  77. Kuehne H, Jhuang H, Garrote E, Poggio T, Serre T (2011) A Large Video Database for Human Motion Recognition, in Proceedings of the International Conference on Computer Vision (ICCV)

  78. Liu Y, Feng X, Zhou Z (2016) Multimodal video classification with stacked contractive autoencoders. Signal Process 120:761–766

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the Faculty of Computer Science & Information Technology, University of Malaya under project (Geran Penyelidikan Fakulti) GPF018D-2019.

Author information

Authors and Affiliations

  1. Faculty of Computer Science & Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Amin Khaksar Pour, Woo Chaw Seng, Shivakumara Palaiahnakote & Nor Badrul Anuar

  2. School of Electrical and Computer Technology, Xiamen University Malaysia, Sepang, Malaysia

    Hamid Tahaei

Authors
  1. Amin Khaksar Pour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Woo Chaw Seng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shivakumara Palaiahnakote
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamid Tahaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nor Badrul Anuar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Amin Khaksar Pour or Nor Badrul Anuar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khaksar Pour, A., Chaw Seng, W., Palaiahnakote, S. et al. A survey on video content rating: taxonomy, challenges and open issues. Multimed Tools Appl 80, 24121–24145 (2021). https://doi.org/10.1007/s11042-021-10838-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10838-8

Keywords

Search

Navigation