Skip to main content
SpringerLink
Log in

Identification of cyberbullying: A deep learning based multimodal approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Cyberbullying can be delineated as a purposive and recurrent act, which is aggressive in nature, done via different social media platforms such as Facebook, Twitter, Instagram and others. While existing approaches for detecting cyberbullying concentrate on unimodal approaches, e.g., text or visual based methods, we proposed a deep learning based early identification framework which is a multimodal (textual and visual) approach (inspired by the informal nature of social media data) and performed a broad analysis on vine dataset. Early identification framework predicts a post or a media session as bully or non-bully as early as possible as we have processed information for each of the modalities (both independently and fusion-based) chronologically. Our multimodal feature-fusion based experimental analysis achieved 0.75 F-measure using ResidualBiLSTM-RCNN architecture, which clearly reflects the effectiveness of our proposed framework. All the codes of this study are made publicly available on paper’s companion repository.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://cyberbullying.org/

  2. https://www.ncpc.org/wp-content/uploads/2017/11/

  3. https://tfhub.dev/google/universal-sentence-encoder/1

  4. https://nlp.stanford.edu/software/tokenizer.shtml

  5. https://ai.googleblog.com/2016/08/improving-inception-and-image.html

References

  1. Agrawal S, Awekar A (2018) Deep learning for detecting cyberbullying across multiple social media platforms. In: European Conference on Information Retrieval, Springer, pp 141–153

  2. Badjatiya P et al (2017) Deep learning for hate speech detection in tweets. In: Proceedings of the 26th international conference on world wide Web companion, pp 759–760

  3. Basu T, Murthy CA (2012) A feature selection method for improved document classification. In: International conference on advanced data mining and applications, Springer, pp 296–305

  4. Cambria E et al (2013) Sentic blending: Scalable multimodal fusion for the continuous inter- pretation of semantics and sentics. In: 2013 IEEE symposium on computational intelli- gence for human-like intelligence (CIHLI), IEEE, pp 108–117

  5. Cer D, et al (2018) Universal sentence encoder. In: arXiv:1803.11175

  6. Dietterich TG (1998) Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput 10.7:1895–1923

    Article  Google Scholar 

  7. Dinakar K, Reichart R, Lieberman H (2011) Modeling the detection of textual cyberbullying. In: Fifth international AAAI conference on weblogs and social media

  8. Djuric N et al (2015) Hate speech detection with comment embeddings. In: Proceedings of the 24th international conference on world wide Web, pp 29–30

  9. Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feed- forward neural networks. In: Proceedings of the Thirteenth international conference on artificial intelligence and statistics, pp 249–256

  10. Goodman LA (1961) Snowball sampling. In: The annals of mathematical statistics, pp 148–170

  11. Hosseinmardi H et al (2015) Analyzing labeled cyberbullying incidents on the instagram social network. In: International conference on social informatics, Springer, pp 49–66

  12. Kumari K, Singh JP (2020) Identification of cyberbullying on multi-modal social media posts using genetic algorithm. In: Transactions on Emerging Telecommunications Technologies, pp e3907

  13. Kumari K et al (2019) Towards Cyberbullying-free social media in smart cities: a unified multi-modal approach. In: Soft Computing, pp 1–12

  14. Kumari K et al (2019) Aggressive social media post detection system containing symbolic images. In: Conference on e-Business, e-Services and e-Society, Springer, pp 415–424

  15. Lipton ZC, Elkan C, Narayanaswamy B (2014) Thresholding classifiers to maximize f1 score. In:

  16. Meng TL, Khushi M (2019) Reinforcement learning in financial markets. Data 4.3 :110

    Article  Google Scholar 

  17. Nuzzo R (2014) Scientific method: statistical errors. Nature News 506.7487:150

    Article  Google Scholar 

  18. Paul Sayanta, Jandhyala SK, Basu T (2018) Early detection of signs of anorexia and depression over social media using effective machine learning frame- works . In: CLEF (Working notes)

  19. Peng Y, Qi J, Yuan Y (2018) Modality-specific cross-modal similarity mea- surement with recurrent attention network. IEEE Trans Image Process 27.11:5585–5599

    Article  Google Scholar 

  20. Pinheiro P, Collobert R (2014) Recurrent convolutional neural networks for scene labeling. In: International conference on machine learning, pp 82–90

  21. Poria S et al (2015) Towards an intelligent framework for multimodal affective data analysis. Neural Netw 63:104–116

    Article  Google Scholar 

  22. Poria S et al (2017) A review of affective computing: From unimodal analysis to multi- modal fusion. Inform Fusion 37:98–125

    Article  Google Scholar 

  23. Prakash A, et al (2016) Neural paraphrase generation with stacked residual lstm networks. In: arXiv:1610.03098

  24. Qureshi SA et al (2019) Multitask representation learning for multimodal estimation of depression level. IEEE Intel Syst 34.5:45–52

    Article  Google Scholar 

  25. Rafiq RI et al (2015) Careful what you share in six seconds: Detecting cyberbullying instances in Vine. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 617–622

  26. Rafiq RI et al (2016) Analysis and detection of labeled cyberbullying instances in Vine, a video-based social network. Soc Netw Anal Mining 6.1:88

    Article  Google Scholar 

  27. Reynolds K, Kontostathis A, Edwards L (2011) Using machine learning to detect cyberbullying. In: 2011 10th international conference on machine learning and applica- tions and workshops, Vol 2, IEEE, pp 241–244

  28. Smith PK et al (2008) Cyberbullying: Its nature and impact in secondary school pupils. J Child Psychol Psychiatry 49.4:376–385

    Article  Google Scholar 

  29. Sun S, Xie Z (2017) Bilstm-based models for metaphor detection. In: National CCF conference on natural language processing and chinese computing, Springer, pp 431–442

  30. Targ S, Almeida D, Lyman K (2016) Resnet in resnet: Generalizing residual architectures. In: arXiv:1603.08029

  31. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemo- Metrics and Intelligent Laboratory Systems 2.1-3:37–52

    Article  Google Scholar 

Download references

Acknowledgments

We would like to express our gratitude to Rahat et al. [25, 26] for sharing their labeled multimodal dataset, Vine. Dr. Sriparna Saha gratefully acknowledges the Young Faculty Research Fellowship (YFRF) Award, supported by Visvesvaraya Ph.D. Scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia) for carrying out this research.

Author information

Authors and Affiliations

  1. Indian Institute of Technology Patna, Bihta, India

    Sayanta Paul & Sriparna Saha

  2. Cork Institute of Technology Cork, Cork, Ireland

    Mohammed Hasanuzzaman

Authors
  1. Sayanta Paul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sriparna Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed Hasanuzzaman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sayanta Paul.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

https://github.com/sayantapaul/Multimodal-Cyberbullying-Identification

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Paul, S., Saha, S. & Hasanuzzaman, M. Identification of cyberbullying: A deep learning based multimodal approach. Multimed Tools Appl 81, 26989–27008 (2022). https://doi.org/10.1007/s11042-020-09631-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09631-w

Keywords

Search

Navigation