ToxVI: a Multimodal LLM-based Framework for Generating Intervention in Toxic Code-Mixed Videos
Pages 3937 - 3942
Abstract
While considerable research has delved into detecting toxic content in text-based data, the realm of video content, particularly in languages other than English, has received less attention. Prior studies have primarily focused on creating automated tools to identify online toxic speech but have often overlooked the crucial next steps of mitigating its impact and discouraging future use. We can discourage social media users from sharing such material by automatically generating interventions that explain why certain content is inappropriate. To bridge this research gap, we propose an innovative task: generating interventions for toxic videos in code-mixed languages which go beyond existing methods focusing on text and images to combat online toxicity. We are introducing a Toxic Code-Mixed Intervention Video benchmark dataset (ToxCMI), comprising 1697 code-mixed toxic video utterances sourced from YouTube. Each utterance in this dataset has been meticulously annotated for toxicity and severity, accompanied by interventions provided in Hindi-English code-mixed languages. We have developed an advanced multimodal framework ToxVI, specifically designed for the task of generating Toxic Video appropriate Interventions, leveraging Large Language Models (LLMs), which comprises three modules - Modality module, Cross-Modal Synchronization module and Generation module. Our experiments demonstrate that integrating multiple modalities from the videos significantly enhances the performance of the proposed task and outperforms all the baselines by a significant margin.
References
[1]
AI@Meta. 2024. Llama 3 Model Card. https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md.
[2]
Cleber Alcântara, Viviane Pereira Moreira, and Diego de Vargas Feijó. 2020. Offensive Video Detection: Dataset and Baseline Results. In Proceedings of The 12th Language Resources and Evaluation Conference, LREC 2020, May 11--16, 2020, Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asunción Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association, Marseille, France, 4309--4319. https://aclanthology.org/2020.lrec-1.531/
[3]
Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, June 29, 2005, Jade Goldstein, Alon Lavie, Chin-Yew Lin, and Clare R. Voss (Eds.). Association for Computational Linguistics, Ann Arbor, Michigan, USA, 65--72. https://aclanthology.org/W05-0909/
[4]
Yi-Ling Chung, Elizaveta Kuzmenko, Serra Sinem Tekiroglu, and Marco Guerini. 2019. CONAN-COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, Florence, Italy, 2819--2829. https://doi.org/10.18653/V1/P19--1271
[5]
Mithun Das, Rohit Raj, Punyajoy Saha, Binny Mathew, Manish Gupta, and Animesh Mukherjee. 2023. HateMM: A Multi-Modal Dataset for Hate Video Classification. In Proceedings of the Seventeenth International AAAI Conference on Web and Social Media, ICWSM 2023, June 5--8, 2023, Yu-Ru Lin, Meeyoung Cha, and Daniele Quercia (Eds.). AAAI Press, Limassol, Cyprus, 1014--1023. https://doi.org/10.1609/ICWSM.V17I1.22209
[6]
Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2018, February 02-03, 2018, Jason Furman, Gary E. Marchant, Huw Price, and Francesca Rossi (Eds.). ACM, New Orleans, LA, USA, 67--73. https://doi.org/10.1145/3278721.3278729
[7]
Maeve Duggan. 2017. Online harassment 2017. Pew Research Center (2017). https://www.pewresearch.org/internet/2017/07/11/online-harassment-2017
[8]
Antigoni-Maria Founta, Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Athena Vakali, and Ilias Leontiadis. 2019. A Unified Deep Learning Architecture for Abuse Detection. In Proceedings of the 11th ACM Conference on Web Science, WebSci 2019, June 30 - July 03, 2019, Paolo Boldi, Brooke Foucault Welles, Katharina Kinder-Kurlanda, Christo Wilson, Isabella Peters, and Wagner Meira Jr. (Eds.). ACM, Boston, MA, USA, 105--114. https://doi.org/10.1145/3292522.3326028
[9]
Raul Gomez, Jaume Gibert, Lluís Gómez, and Dimosthenis Karatzas. 2020. Exploring Hate Speech Detection in Multimodal Publications. In Proceedingds of the IEEE Winter Conference on Applications of Computer Vision, WACV 2020, March 1--5, 2020. IEEE, Snowmass Village, CO, USA, 1459--1467. https://doi.org/10.1109/WACV45572.2020.9093414
[10]
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. CoRR, Vol. abs/2310.06825 (2023). https://doi.org/10.48550/ARXIV.2310.06825 showeprint[arXiv]2310.06825
[11]
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020. Contextualizing Hate Speech Classifiers with Post-hoc Explanation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 5435--5442. https://doi.org/10.18653/v1/2020.acl-main.483
[12]
Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6--12, 2020, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). Virtual, 1--14.
[13]
Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. Association for Computational Linguistics, Barcelona, Spain, 74--81. https://aclanthology.org/W04--1013
[14]
Krishanu Maity, Raghav Jain, Prince Jha, Sriparna Saha, and Pushpak Bhattacharyya. 2023. GenEx: A Commonsense-aware Unified Generative Framework for Explainable Cyberbullying Detection. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, December 6--10, 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 16632--16645. https://doi.org/10.18653/V1/2023.EMNLP-MAIN.1035
[15]
Krishanu Maity, Prince Jha, Sriparna Saha, and Pushpak Bhattacharyya. 2022. A Multitask Framework for Sentiment, Emotion and Sarcasm aware Cyberbullying Detection from Multi-modal Code-Mixed Memes. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, July 11 - 15, 2022, Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpepper, and Gabriella Kazai (Eds.). ACM, Madrid, Spain, 1739--1749. https://doi.org/10.1145/3477495.3531925
[16]
Krishanu Maity, A. S. Poornash, Shaubhik Bhattacharya, Salisa Phosit, Sawarod Kongsamlit, Sriparna Saha, and Kitsuchart Pasupa. 2024. HateThaiSent: Sentiment-Aided Hate Speech Detection in Thai Language. IEEE Transactions on Computational Social Systems (2024), 1--14. https://doi.org/10.1109/TCSS.2024.3376958
[17]
Krishanu Maity, Sriparna Saha, and Pushpak Bhattacharyya. 2023. Emoji, Sentiment and Emotion Aided Cyberbullying Detection in Hinglish. IEEE Transactions on Computational Social Systems, Vol. 10, 5 (2023), 2411--2420. https://doi.org/10.1109/TCSS.2022.3183046
[18]
Krishanu Maity, Poornash Sangeetha, Sriparna Saha, and Pushpak Bhattacharyya. 2024. ToxVidLM: A Multimodal Framework for Toxicity Detection in Code-Mixed Videos. In Findings of the Association for Computational Linguistics ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand and virtual meeting, 11130--11142.
[19]
Binny Mathew, Navish Kumar, Ravina, Pawan Goyal, and Animesh Mukherjee. 2018. Analyzing the hate and counter speech accounts on Twitter. CoRR, Vol. abs/1812.02712 (2018). showeprint[arXiv]1812.02712
[20]
Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette-Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, and et al. 2024. Gemma: Open Models Based on Gemini Research and Technology. CoRR, Vol. abs/2403.08295 (2024). https://doi.org/10.48550/ARXIV.2403.08295 showeprint[arXiv]2403.08295
[21]
Carol Myers-Scotton. 1997. Duelling Languages: Grammatical Structure in Codeswitching. Oxford University Press. https://doi.org/10.1093/oso/9780198240594.001.0001
[22]
Mohammed Hussein Obaid, Shawkat Kamal Guirguis, and Saleh Mesbah Elkaffas. 2023. Cyberbullying Detection and Severity Determination Model. IEEE Access, Vol. 11 (2023), 97391--97399. https://doi.org/10.1109/ACCESS.2023.3313113
[23]
Abby Ohlheiser. 2016. Banned from Twitter? This site promises you can say whatever you want. Washington Post (2016). https://www.washingtonpost.com/news/the-intersect/wp/2016/11/29/banned-from-twitter-this-site-promises-you-can-say-whatever-you-want
[24]
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6--12, 2002. ACL, Philadelphia, PA, USA, 311--318. https://doi.org/10.3115/1073083.1073135
[25]
Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth M. Belding, and William Yang Wang. 2019. A Benchmark Dataset for Learning to Intervene in Online Hate Speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, November 3--7, 2019, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 4754--4763. https://doi.org/10.18653/V1/D19--1482
[26]
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. Robust Speech Recognition via Large-Scale Weak Supervision. In International Conference on Machine Learning, ICML 2023, 23--29 July 2023, USA (Proceedings of Machine Learning Research, Vol. 202), Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.). PMLR, Honolulu, Hawaii, 28492--28518. https://proceedings.mlr.press/v202/radford23a.html
[27]
Aneri Rana and Sonali Jha. 2022. Emotion Based Hate Speech Detection using Multimodal Learning. CoRR, Vol. abs/2202.06218 (2022). showeprint[arXiv]2202.06218
[28]
Pradeep Kumar Roy and Fenish Umeshbhai Mali. 2022. Cyberbullying detection using deep transfer learning. Complex & Intelligent Systems, Vol. 8, 6 (2022), 5449--5467. https://doi.org/10.1007/s40747-022-00772-z
[29]
Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. 2022. VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, November 28 - December 9, 2022, Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). New Orleans, LA, USA.
[30]
Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct Distillation of LM Alignment. CoRR, Vol. abs/2310.16944 (2023). https://doi.org/10.48550/ARXIV.2310.16944 showeprint[arXiv]2310.16944
[31]
Yogarshi Vyas, Spandana Gella, Jatin Sharma, Kalika Bali, and Monojit Choudhury. 2014. POS Tagging of English-Hindi Code-Mixed Social Media Content. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25--29, 2014, A meeting of SIGDAT, a Special Interest Group of the ACL, Alessandro Moschitti, Bo Pang, and Walter Daelemans (Eds.). ACL, Doha, Qatar, 974--979. https://doi.org/10.3115/V1/D14--1105
[32]
Han Wang, Ming Shan Hee, Md. Rabiul Awal, Kenny Tsu Wei Choo, and Roy Ka-Wei Lee. 2023. Evaluating GPT-3 Generated Explanations for Hateful Content Moderation. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th-25th August 2023. ijcai.org, Macao, SAR, China, 6255--6263. https://doi.org/10.24963/IJCAI.2023/694
[33]
Ching Seh Wu and Unnathi Bhandary. 2020. Detection of Hate Speech in Videos Using Machine Learning. In Proceedings of the International Conference on Computational Science and Computational Intelligence, CSCI 2020, December 16--18, 2020. IEEE, Las Vegas, NV, USA, 585--590. https://doi.org/10.1109/CSCI51800.2020.00104
[34]
Fan Yang, Xiaochang Peng, Gargi Ghosh, Reshef Shilon, Hao Ma, Eider Moore, and Goran Predovic. 2019. Exploring Deep Multimodal Fusion of Text and Photo for Hate Speech Classification. In Proceedings of the Third Workshop on Abusive Language Online, Sarah T. Roberts, Joel Tetreault, Vinodkumar Prabhakaran, and Zeerak Waseem (Eds.). Association for Computational Linguistics, Florence, Italy, 11--18. https://doi.org/10.18653/v1/W19--3502
[35]
Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating Text Generation with BERT. In Proceedings of the 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020. OpenReview.net. https://openreview.net/forum?id=SkeHuCVFDr
Index Terms
- ToxVI: a Multimodal LLM-based Framework for Generating Intervention in Toxic Code-Mixed Videos
Recommendations
Multimodal LLM Enhanced Cross-lingual Cross-modal Retrieval
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaCross-lingual cross-modal retrieval (CCR) aims to retrieve visually relevant content based on non-English queries, without relying on human-labeled cross-modal data pairs during training. One popular approach involves utilizing machine translation (MT) ...
Investigating the Intervention in Parallel Conversations
HAI '23: Proceedings of the 11th International Conference on Human-Agent InteractionIn recent years, a framework of parallel conversations has been proposed to facilitate efficient conversations through cooperation between humans and dialogue systems. This approach aims to enable simultaneous conversations with multiple users by ...
Designing mHealth intervention for women in menopausal period
PervasiveHealth '15: Proceedings of the 9th International Conference on Pervasive Computing Technologies for HealthcareExpected age of population is increasing as the quality of medical technology and services is improving. As a result, women nowadays are expected to spend 1/3 of their lives after menopause. Precise knowledge of each of their menopausal stage and how to ...
Comments
Information & Contributors
Information
Published In
October 2024
5705 pages
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 October 2024
Check for updates
Author Tags
Qualifiers
- Short-paper
Conference
CIKM '24
Sponsor:
CIKM '24: The 33rd ACM International Conference on Information and Knowledge Management
October 21 - 25, 2024
ID, Boise, USA
Acceptance Rates
Overall Acceptance Rate 1,861 of 8,427 submissions, 22%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 101Total Downloads
- Downloads (Last 12 months)101
- Downloads (Last 6 weeks)18
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in