skip to main content
10.1145/3488560.3501392acmconferencesArticle/Chapter ViewAbstractPublication PageswsdmConference Proceedingsconference-collections
tutorial

Half-Day Tutorial on Combating Online Hate Speech: The Role of Content, Networks, Psychology, User Behavior, etc.

Published: 15 February 2022 Publication History

Abstract

While the rise in popularity of social media is seen as a hugely positive development, it is also accompanied by a proliferation of hate speech, which has recently become a major concern. On the one hand, hateful content creates an unsafe environment for certain members of society. On the other hand, manual moderation causes distress to content moderators, and the volume of harmful content is far beyond what human moderators can manually flag and react to. Thus, researchers in machine learning, social computing, and other areas have worked on developing tools to help automate the process. While initially studied as a text classification problem, over time, researchers realized that hate speech is multi-faceted and requires analysis of the role of linguistic expressions, context, and network structure, while using inspiration from psychology and user behavior, among others. With this in mind, we provide a holistic view of what the research community has explored so far, and what we believe are promising future research directions.

References

[1]
Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, and Preslav Nakov. 2021. Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society. In Findings of the Association for Computational Linguistics: EMNLP 2021 . ACL, Punta Cana, Dominican Republic, 611--649.
[2]
Pinkesh Badjatiya, Manish Gupta, and Vasudeva Varma. 2019. Stereotypical bias removal for hate speech detection task using knowledge-based generalizations. In Proceedings of the International Conference on World Wide Web (WWW '19). ACM, San Francisco, CA, USA, 49--59.
[3]
Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th International Conference on World Wide Web companion (WWW '17). ACM, Perth, Australia, 759--760.
[4]
Arunkumar Bagavathi, Pedram Bashiri, Shannon Reid, Matthew Phillips, and Siddharth Krishnan. 2019. Examining untempered social media: analyzing cascades of polarized conversations. In Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM '19). IEEE Computer Society, Los Alamitos, CA, USA, 625--632.
[5]
Mohit Bhardwaj, Md Shad Akhtar, Asif Ekbal, Amitava Das, and Tanmoy Chakraborty. 2020. Hostility detection dataset in Hindi. arXiv:2011.03588 (2020).
[6]
Pete Burnap and Matthew L Williams. 2016. Us and them: identifying cyber hate on Twitter across multiple protected characteristics. EPJ Data Science, Vol. 5 (2016), 1--15.
[7]
Mohit Chandra, Ashwin Pathak, Eesha Dutta, Paryul Jain, Manish Gupta, Manish Shrivastava, and Ponnurangam Kumaraguru. 2020. AbuseAnalyzer: Abuse detection, severity and target prediction for Gab posts. In Proceedings of the 28th International Conference on Computational Linguistics (COLING '20). ICCL, Barcelona, Spain (Online), 6277--6283.
[8]
Lu Cheng, Kai Shu, Siqi Wu, Yasin N Silva, Deborah L Hall, and Huan Liu. 2020. Unsupervised cyberbullying detection via time-informed Gaussian mixture model. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM '20). ACM, Online, 185--194.
[9]
Patricia Chiril, Véronique Moriceau, Farah Benamara, Alda Mari, Gloria Origgi, and Marlène Coulomb-Gully. 2020. He said “Who's gonna take care of your children when you are at ACL?”: Reported sexist acts are not sexist. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL '20). ACL, Online, 4055--4066.
[10]
Ke-Li Chiu and Rohan Alexander. 2021. Detecting hate speech with GPT-3. arXiv:2103.12407 (2021).
[11]
Snehil Dahiya, Shalini Sharma, Dhruv Sahnan, Vasu Goel, Emilie Chouzenoux, V'ictor Elvira, Angshul Majumdar, Anil Bandhakavi, and Tanmoy Chakraborty. 2021. Would your tweet invoke hate on the fly? Forecasting hate intensity of reply threads on Twitter. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '21). ACM, Singapore (Online), 2732--2742.
[12]
Abhishek Das, Japsimar Singh Wahi, and Siyao Li. 2020. Detecting hate speech in multi-modal memes. arXiv:2012.14891 (2020).
[13]
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the Eleventh International Conference on Web and Social Media (ICWSM '17). AAAI Press, Montré al, Qué bec, Canada, 512--515.
[14]
Agneta Fischer, Eran Halperin, Daphna Canetti, and Alba Jasini. 2018. Why we hate. Emotion Review, Vol. 10, 4 (2018), 309--320.
[15]
Paula Fortuna and Sérgio Nunes. 2018. A survey on automatic detection of hate speech in text. ACM Computing Surveys (CSUR), Vol. 51, 4 (2018), 1--30.
[16]
Antigoni Maria Founta, Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Athena Vakali, and Ilias Leontiadis. 2019. A unified deep learning architecture for abuse detection. In Proceedings of the 10th ACM Conference on Web Science (WebSci '19). ACM, Boston, MA, USA, 105--114.
[17]
Lewis R. Goldberg. 1990. An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, Vol. 59, 6 (1990), 1216--1229.
[18]
Raul Gomez, Jaume Gibert, Lluis Gomez, and Dimosthenis Karatzas. 2020. Exploring hate speech detection in multimodal publications. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV '20). IEEE, Snowmass Village, CO, USA, 1470--1478.
[19]
N. F. Johnson, R. Leahy, N. Johnson Restrepo, N. Velasquez, M. Zheng, P. Manrique, P. Devkota, and S. Wuchty. 2019. Hidden resilience and adaptive dynamics of the global online hate ecology. Nature, Vol. 573, 7773 (01 Sep 2019), 261--265.
[20]
Mika Juuti, Tommi Gröndahl, Adrian Flanagan, and N Asokan. 2020. A little goes a long way: Improving toxic language classification despite data scarcity. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings (EMNLP '20). ACL, Online, 2991--3009.
[21]
Md. Rezaul Karim, Sumon Kanti Dey, and Bharathi Raja Chakravarthi. 2021. DeepHateExplainer: Explainable hate speech detection in under-resourced Bengali language. arXiv:2012.14353 (2021).
[22]
Sweta Karlekar and Mohit Bansal. 2018. SafeCity: Understanding diverse forms of sexual harassment personal stories. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP '18). ACL, Brussels, Belgium, 2805--2811.
[23]
Geoff F Kaufman and Lisa K Libby. 2012. Changing beliefs and behavior through experience-taking. Journal of personality and social psychology, Vol. 103, 1 (July 2012), 1-19.
[24]
Brendan Kennedy, Xisen Jin, Aida Mostafazadeh Davani, Morteza Dehghani, and Xiang Ren. 2020. Contextualizing hate speech classifiers with post-hoc explanation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL '20). ACL, Online, 5435--5442.
[25]
Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The Hateful Memes Challenge: Detecting hate speech in multimodal memes. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeuIPS '20, Vol. 33). Online.
[26]
Carlo Kopp, Kevin B. Korb, and Bruce I. Mills. 2018. Information-theoretic models of deception: Modelling cooperation and diffusion in populations exposed to "fake news". PLOS One, Vol. 13, 11 (11 2018), 1--35.
[27]
Son T. Luu, Kiet Van Nguyen, and Ngan Luu-Thuy Nguyen. 2021. A large-scale dataset for hate speech detection on Vietnamese social media texts. In Advances and Trends in Artificial Intelligence. Springer, 415--426.
[28]
Sean MacAvaney, Hao-Ren Yao, Eugene Yang, Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. PLOS One, Vol. 14, 8 (2019), e0221152.
[29]
Sarah Masud, Subhabrata Dutta, Sakshi Makkar, Chhavi Jain, Vikram Goyal, Amitava Das, and Tanmoy Chakraborty. 2021. Hate is the new infodemic: A topic-aware modeling of hate speech diffusion on Twitter. In Proceedings of the 37th IEEE International Conference on Data Engineering (ICDE '21). IEEE, Chania, Greece, 504--515.
[30]
Binny Mathew, Ritam Dutt, Pawan Goyal, and Animesh Mukherjee. 2019. Spread of hate speech in online social media. In Proceedings of the 10th ACM Conference on Web Science (WebSci '19). ACM, Boston, MA, USA, 173--182.
[31]
Binny Mathew, Navish Kumar, Ravina, Pawan Goyal, and Animesh Mukherjee. 2018. Analyzing the hate and counter speech accounts on Twitter. arXiv/1812.02712 (2018).
[32]
Lucia Ková?ová, and Edita Rigová. 2020. Countering hate speech on Facebook: The case of the Roma minority in Slovakia. Social Science Computer Review, Vol. 38, 2 (2020), 128--146.
[33]
Guanyi Mou, Pengyi Ye, and Kyumin Lee. 2020. SWE2: Subword enriched and significant word emphasized framework for hate speech detection. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM '20). ACM, Online, 1145--1154.
[34]
Luke Munn. 2020. Angry by design: toxic communication and technical architectures. Humanities and Social Sciences Communications, Vol. 7, 1 (Jul 2020), 53.
[35]
Preslav Nakov, Vibha Nayak, Kyle Dent, Ameya Bhatawdekar, Sheikh Muhammad Sarwar, Momchil Hardalov, Yoan Dinkov, Dimitrina Zlatkova, Guillaume Bouchard, and Isabelle Augenstein. 2021. Detecting abusive language on online platforms: A critical analysis., Vol. arXiv/2103.00153 (2021).
[36]
Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and Yi Chang. 2016. Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web (WWW '16). WWW, Montréal, Québec, Canada, 145--153.
[37]
Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2021. A joint learning approach with knowledge injection for zero-shot cross-lingual hate speech detection. Information Processing & Management, Vol. 58, 4 (2021), 102544.
[38]
Pulkit Parikh, Harika Abburi, Pinkesh Badjatiya, Radhika Krishnan, Niyati Chhaya, Manish Gupta, and Vasudeva Varma. 2019. Multi-label categorization of accounts of sexism using a neural framework. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP '19). ACL, Hong Kong, China, 1642--1652.
[39]
Pulkit Parikh, Harika Abburi, Niyati Chhaya, Manish Gupta, and Vasudeva Varma. 2021. Categorizing sexism and misogyny through neural approaches. ACM Trans. Web, Vol. 15, 4, Article 17 (June 2021).
[40]
Delroy Paulhus and Kevin Williams. 2002. The dark triad of personality: narcissism, machiavellianism, and psychopathy. J. Research in Personality, Vol. 36, 6 (2002), 556--563.
[41]
Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021 a. Detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. ACL, Online, 2783--2796.
[42]
Shraman Pramanick, Shivam Sharma, Dimitar Dimitrov, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021 b. MOMENTA: A multimodal framework for detecting harmful memes and their targets. In Findings of the Association for Computational Linguistics: EMNLP 2021 . ACL, Punta Cana, Dominican Republic, 4439--4455.
[43]
Jing Qian, Anna Bethke, Yinyin Liu, Elizabeth Belding, and William Yang Wang. 2019. A benchmark dataset for learning to intervene in online hate speech. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP '19). ACL, Hong Kong, China, 4755--4764.
[44]
Jing Qian, Mai ElSherief, Elizabeth Belding, and William Yang Wang. 2018. Hierarchical CVAE for fine-grained hate speech classification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP '18). ACL, Brussels, Belgium, 3550--3559.
[45]
Hammad Rizwan, Muhammad Haroon Shakeel, and Asim Karim. 2020. Hate-speech and offensive language detection in Roman Urdu. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP '20). ACL, Online, 2512--2522.
[46]
Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, and Preslav Nakov. 2021. SOLID: A large-scale semi-supervised dataset for offensive language identification. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 . ACL, Online, 915--928.
[47]
Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Jurafsky, Noah A. Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power implications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL '20). ACL, Online, 5477--5490.
[48]
Sheikh Muhammad Sarwar, Dimitrina Zlatkova, Momchil Hardalov, Yoan Dinkov, Isabelle Augenstein, and Preslav Nakov. 2022. A neighbourhood framework for resource-lean content flagging. Trans. Assoc. Comput. Linguistics (2022).
[49]
Shalom H Schwartz. 2012. An overview of the Schwartz theory of basic values. Online Readings in Psychology and Culture, Vol. 2, 1 (2012), 11.
[50]
John Suler. 2004. The Online Disinhibition Effect. Cyberpsychol Behav., Vol. 7, 3 (2004), 321--326.
[51]
Ashima Suvarna and Grusha Bhalla. 2020. #NotAWhore! A computational linguistic perspective of rape culture and victimization on social media. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop . ACL, Online, 328--335.
[52]
Serra Sinem Tekiroug lu, Yi-Ling Chung, and Marco Guerini. 2020. Generating counter narratives against online hate speech: Data and strategies. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL '20). ACL, Online, 1177--1190.
[53]
Bertie Vidgen and Leon Derczynski. 2021. Directions in abusive language training data, a systematic review: Garbage in, garbage out. PLOS One, Vol. 15, 12 (12 2021), 1--32. https://doi.org/10.1371/journal.pone.0243300
[54]
Zeerak Waseem. 2016. Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science. ACL, Austin, TX, USA, 138--142.
[55]
Zeerak Waseem and Dirk Hovy. 2016. Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In Proceedings of the NAACL Student Research Workshop. ACL, San Diego, CA, USA, 88--93.
[56]
Mengzhou Xia, Anjalie Field, and Yulia Tsvetkov. 2020. Demoting racial bias in hate speech detection. In Proceedings of the Eighth International Workshop on Natural Language Processing for Social Media (SocialNLP '20). ACL, Online, 7--14.
[57]
Fan Yang, Xiaochang Peng, Gargi Ghosh, Reshef Shilon, Hao Ma, Eider Moore, and Goran Predovic. 2019. Exploring deep multimodal fusion of text and photo for hate speech classification. In Proceedings of the Third Workshop on Abusive Language Online. ACL, Florence, Italy, 11--18.
[58]
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019 a. Predicting the type and target of offensive posts in social media. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT '19). ACL, Minneapolis, MN, USA, 1415--1420.
[59]
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019 b. SemEval-2019 Task 6: Identifying and categorizing offensive language in social media (OffensEval). In Proceedings of the Workshop on Semantic Evaluation (SemEval '19). ACL, Minneapolis, MN, USA, 75--86.
[60]
Marcos Zampieri, Preslav Nakov, Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Hamdy Mubarak, Leon Derczynski, Zeses Pitenis, and cC aug ri cC öltekin. 2020. SemEval-2020 task 12: Multilingual offensive language identification in social media (OffensEval 2020). In Proceedings of the Fourteenth Workshop on Semantic Evaluation (SemEval '20). ICCL, Barcelona (Online), 1425--1447.
[61]
Ziqi Zhang and Lei Luo. 2019. Hate speech detection: A solved problem? The challenging case of long tail on Twitter. Semantic Web, Vol. 10, 5 (2019), 925--945.
[62]
Haoti Zhong, Hao Li, Anna Squicciarini, Sarah Rajtmajer, Christopher Griffin, David Miller, and Cornelia Caragea. 2016. Content-driven detection of cyberbullying on the Instagram social network. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI '16). AAAI Press, New York, NY, USA, 3952--3958.
[63]
Xuhui Zhou, Maarten Sap, Swabha Swayamdipta, Yejin Choi, and Noah Smith. 2021. Challenges in automated debiasing for toxic language detection. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics (EACL '21). ACL, Online, 3143--3155.

Cited By

View all
  • (2023)Hate Speech: Detection, Mitigation and BeyondProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3572721(1232-1235)Online publication date: 27-Feb-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
February 2022
1690 pages
ISBN:9781450391320
DOI:10.1145/3488560
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. detection of hate
  2. hate speech
  3. network analysis
  4. psycho-linguistic analysis of hate speech
  5. spread of hate
  6. user behavior

Qualifiers

  • Tutorial

Conference

WSDM '22

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)23
  • Downloads (Last 6 weeks)1
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Hate Speech: Detection, Mitigation and BeyondProceedings of the Sixteenth ACM International Conference on Web Search and Data Mining10.1145/3539597.3572721(1232-1235)Online publication date: 27-Feb-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media