Skip to main content
SpringerLink
Log in

Exposing Racial Dialect Bias in Abusive Language Detection: Can Explainability Play a Role?

  • Conference paper
  • First Online:
Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1752))

  • 865 Accesses

Abstract

Biases can arise and be introduced during each phase of a supervised learning pipeline, eventually leading to harm. Within the task of automatic abusive language detection, this matter becomes particularly severe since unintended bias towards sensitive topics such as gender, sexual orientation, or ethnicity can harm underrepresented groups. The role of the datasets used to train these models is crucial to address these challenges. In this contribution, we investigate whether explainability methods can expose racial dialect bias attested within a popular dataset for abusive language detection. Through preliminary experiments, we found that pure explainability techniques cannot effectively uncover biases within the dataset under analysis: the rooted stereotypes are often more implicit and complex to retrieve.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The results of the experiments are available at https://github.com/MartaMarchiori/Exposing-Racial-Dialect-Bias.

  2. 2.

    https://huggingface.co/bert-base-uncased.

  3. 3.

    https://github.com/slanglab/twitteraae.

  4. 4.

    SHAP was not used on the entire test set (i.e., within the Global Explanations Section) due to the high computational costs of this explainability method. It was therefore preferred to apply it when analysing a narrower subset, i.e., in the sub-global setting.

  5. 5.

    https://github.com/marcotcr/anchor.

  6. 6.

    https://github.com/fdalvi/NeuroX.

References

  1. Angerschmid, A., Zhou, J., Theuermann, K., Chen, F., Holzinger, A.: Fairness and explanation in AI-informed decision making. Mach. Learn. Knowl. Extraction 4(2), 556–579 (2022)

    Article  Google Scholar 

  2. Balkir, E., Kiritchenko, S., Nejadgholi, I., Fraser, K.C.: Challenges in applying explainability methods to improve the fairness of NLP models. arXiv preprint arXiv:2206.03945 (2022)

  3. Ball-Burack, A., Lee, M.S.A., Cobbe, J., Singh, J.: Differential tweetment: mitigating racial dialect bias in harmful tweet detection. In: FAccT, pp. 116–128. ACM (2021)

    Google Scholar 

  4. Baniecki, H., Kretowicz, W., Piatyszek, P., Wisniewski, J., Biecek, P.: dalex: responsible machine learning with interactive explainability and fairness in python. arXiv preprint arXiv:2012.14406 (2020)

  5. Basile, V., Cabitza, F., Campagner, A., Fell, M.: Toward a perspectivist turn in ground truthing for predictive computing. arXiv preprint arXiv:2109.04270 (2021)

  6. Bhargava, V., Couceiro, M., Napoli, A.: LimeOut: an ensemble approach to improve process fairness. In: Koprinska, I., et al. (eds.) ECML PKDD 2020. CCIS, vol. 1323, pp. 475–491. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65965-3_32

    Chapter  Google Scholar 

  7. Binder, A., Montavon, G., Lapuschkin, S., Müller, K.-R., Samek, W.: Layer-wise relevance propagation for neural networks with local renormalization layers. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9887, pp. 63–71. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44781-0_8

    Chapter  MATH  Google Scholar 

  8. Bird, S., et al.: Fairlearn: a toolkit for assessing and improving fairness in AI. Tech. Rep. MSR-TR-2020-32, Microsoft (2020)

    Google Scholar 

  9. Blodgett, S.L., Barocas, S., Daumé III, H., Wallach, H.: Language (technology) is power: a critical survey of “bias” in NLP. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5454–5476 (2020)

    Google Scholar 

  10. Blodgett, S.L., Green, L., O’Connor, B.: Demographic dialectal variation in social media: a case study of african-american english. In: Proceedings of EMNLP (2016)

    Google Scholar 

  11. Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., Rinzivillo, S.: Benchmarking and survey of explanation methods for black box models. CoRR abs/2102.13076 (2021)

    Google Scholar 

  12. Caselli, T., Basile, V., Mitrović, J., Kartoziya, I., Granitzer, M.: I feel offended, don’t be abusive! implicit/explicit messages in offensive and abusive language. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 6193–6202. European Language Resources Association, Marseille, France (2020). https://www.aclweb.org/anthology/2020.lrec-1.760

  13. Dalvi, F., et al.: Neurox: a toolkit for analyzing individual neurons in neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (2019)

    Google Scholar 

  14. Davidson, T., Bhattacharya, D., Weber, I.: Racial bias in hate speech and abusive language detection datasets. arXiv preprint arXiv:1905.12516 (2019)

  15. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  16. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics (2019)

    Google Scholar 

  17. Dixon, L., Li, J., Sorensen, J., Thain, N., Vasserman, L.: Measuring and mitigating unintended bias in text classification. In: AIES, pp. 67–73. ACM (2018)

    Google Scholar 

  18. Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)

  19. Founta, A., et al.: Large scale crowdsourcing and characterization of twitter abusive behavior. In: ICWSM, pp. 491–500. AAAI Press (2018)

    Google Scholar 

  20. Freitas, A.A.: Comprehensible classification models: a position paper. SIGKDD Explor. 15(1), 1–10 (2013)

    Article  Google Scholar 

  21. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 1–42 (2019)

    Google Scholar 

  22. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  23. Holzinger, A., Saranti, A., Molnar, C., Biecek, P., Samek, W.: Explainable AI methods-a brief overview. In: Holzinger, A., Goebel, R., Fong, R., Moon, T., Müller, K.R., Samek, W. (eds.) xxAI - Beyond Explainable AI. xxAI 2020. LNCS, vol. 13200, pp. 13–38. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-04083-2_2

  24. Kiritchenko, S., Nejadgholi, I., Fraser, K.C.: Confronting abusive language online: a survey from the ethical and human rights perspective. J. Artif. Intell. Res. 71, 431–478 (2021)

    Article  MATH  Google Scholar 

  25. Kowsari, K., Jafari Meimandi, K., Heidarysafa, M., Mendu, S., Barnes, L., Brown, D.: Text classification algorithms: a survey. Information 10(4), 150 (2019)

    Article  Google Scholar 

  26. Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  27. Longo, L., Goebel, R., Lecue, F., Kieseberg, P., Holzinger, A.: Explainable artificial intelligence: concepts, applications, research challenges and visions. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-MAKE 2020. LNCS, vol. 12279, pp. 1–16. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-57321-8_1

    Chapter  Google Scholar 

  28. Lundberg, S.M., Lee, S.: A unified approach to interpreting model predictions. In: NIPS, pp. 4765–4774 (2017)

    Google Scholar 

  29. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, 2–4 May 2013, Workshop Track Proceedings (2013)

    Google Scholar 

  30. Ntoutsi, E., et al.: Bias in data-driven artificial intelligence systems - an introductory survey. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 10(3), e1356 (2020)

    Google Scholar 

  31. Pedreschi, D., et al.: Open the black box data-driven explanation of black box decision systems. CoRR abs/1806.09936 (2018)

    Google Scholar 

  32. Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: Empirical methods in natural language processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  33. Peters, M.E., et al.: Deep contextualized word representations. In: NAACL-HLT, pp. 2227–2237. Association for Computational Linguistics (2018)

    Google Scholar 

  34. Ribeiro, M.T., Singh, S., Guestrin, C.: why should I trust you?: explaining the predictions of any classifier. In: KDD, pp. 1135–1144. ACM (2016)

    Google Scholar 

  35. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: High-precision model-agnostic explanations. In: AAAI, pp. 1527–1535. AAAI Press (2018)

    Google Scholar 

  36. Samek, W., Montavon, G., Lapuschkin, S., Anders, C.J., Müller, K.: Toward interpretable machine learning: transparent deep neural networks and beyond. CoRR abs/2003.07631 (2020)

    Google Scholar 

  37. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of Bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)

  38. Sap, M., Card, D., Gabriel, S., Choi, Y., Smith, N.A.: The risk of racial bias in hate speech detection. In: ACL (1), pp. 1668–1678. Association for Computational Linguistics (2019)

    Google Scholar 

  39. Sokol, K., Hepburn, A., Poyiadzi, R., Clifford, M., Santos-Rodriguez, R., Flach, P.: FAT forensics: a python toolbox for implementing and deploying fairness, accountability and transparency algorithms in predictive systems. J. Open Source Softw. 5(49), 1904 (2020)

    Article  Google Scholar 

  40. Sundararajan, M., Taly, A., Yan, Q.: Axiomatic attribution for deep networks. In: International Conference on Machine Learning, pp. 3319–3328. PMLR (2017)

    Google Scholar 

  41. Suresh, H., Guttag, J.V.: A framework for understanding unintended consequences of machine learning. CoRR abs/1901.10002 (2019)

    Google Scholar 

  42. Vashishth, S., Upadhyay, S., Tomar, G.S., Faruqui, M.: Attention interpretability across NLP tasks. arXiv preprint arXiv:1909.11218 (2019)

  43. Vaswani, A., et al.: Attention is all you need. In: NIPS, pp. 5998–6008 (2017)

    Google Scholar 

  44. Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6000–6010. NIPS2017, Curran Associates Inc., Red Hook, NY, USA (2017)

    Google Scholar 

  45. Wang, T., Saar-Tsechansky, M.: Augmented fairness: an interpretable model augmenting decision-makers’ fairness. arXiv preprint arXiv:2011.08398 (2020)

  46. Wiegand, M., Ruppenhofer, J., Kleinbauer, T.: Detection of abusive language: the problem of biased datasets. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 602–608 (2019)

    Google Scholar 

  47. Zampieri, M., et al.: Semeval-2020 task 12: multilingual offensive language identification in social media (offenseval 2020). In: SemEval@COLING, pp. 1425–1447. International Committee for Computational Linguistics (2020)

    Google Scholar 

Download references

Acknowledgements

This work has been partially supported by the European Community Horizon 2020 programme under the funding schemes: H2020-INFRAIA-2019-1: Research Infrastructure G.A. 871042 SoBigData++, G.A. 952026 HumanE AI Net, ERC-2018-ADG G.A. 834756 XAI: Science and technology for the eXplanation of AI decision making), G.A. 952215 TAILOR.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Pisa, Pisa, Italy

    Marta Marchiori Manerba & Virginia Morini

  2. KDD Laboratory, ISTI, National Research Council, Pisa, Italy

    Virginia Morini

Authors
  1. Marta Marchiori Manerba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Virginia Morini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marta Marchiori Manerba .

Editor information

Editors and Affiliations

  1. University of Sydney, Sydney, Australia

    Irena Koprinska

  2. University of Bari Aldo Moro, Bari, Italy

    Paolo Mignone

  3. University of Pisa, Pisa, Italy

    Riccardo Guidotti

  4. Warsaw University of Technology, Warsaw, Poland

    Szymon Jaroszewicz

  5. Heidelberg University, Heidelberg, Germany

    Holger Fröning

  6. UniCredit, Rome, Italy

    Francesco Gullo

  7. University of Lisbon, Lisbon, Portugal

    Pedro M. Ferreira

  8. Roche, Basel, Switzerland

    Damian Roqueiro

  9. Barcelona Supercomputing Center, Barcelona, Spain

    Gaia Ceddia

  10. Halmstad University, Halmstad, Sweden

    Slawomir Nowaczyk

  11. University of Porto, Porto, Portugal

    João Gama

  12. University of Porto, Porto, Portugal

    Rita Ribeiro

  13. UPC BarcelonaTech, Barcelona, Spain

    Ricard Gavaldà

  14. University of Naples Federico II, Naples, Italy

    Elio Masciari

  15. University of North Carolina, Charlotte, USA

    Zbigniew Ras

  16. ICAR-CNR, Rende, Italy

    Ettore Ritacco

  17. University of Pisa, Pisa, Italy

    Francesca Naretto

  18. Aalen University of Applied Sciences, Aalen, Germany

    Andreas Theissler

  19. Warsaw University of Technology, Warszaw, Poland

    Przemyslaw Biecek

  20. KU Leuven, Leuven, Belgium

    Wouter Verbeke

  21. University of Duisburg-Essen, Essen, Germany

    Gregor Schiele

  22. Graz University of Technology, Graz, Austria

    Franz Pernkopf

  23. AMD, Dublin, Ireland

    Michaela Blott

  24. UniCredit, Rome, Italy

    Ilaria Bordino

  25. UniCredit, Milan, Italy

    Ivan Luciano Danesi

  26. National Agency for New Technologies, Rome, Italy

    Giovanni Ponti

  27. Unicredit, Rome, Italy

    Lorenzo Severini

  28. University of Bari Aldo Moro, Bari, Italy

    Annalisa Appice

  29. University of Bari Aldo Moro, Bari, Italy

    Giuseppina Andresini

  30. University of Lisbon, Lisbon, Portugal

    Ibéria Medeiros

  31. University of Lisbon, Lisbon, Portugal

    Guilherme Graça

  32. Northwestern University, Chicago, USA

    Lee Cooper

  33. Roche, Basel, Switzerland

    Naghmeh Ghazaleh

  34. University of Lausanne, Lausanne, Switzerland

    Jonas Richiardi

  35. Novartis, Basel, Switzerland

    Diego Saldana

  36. Novartis, Basel, Switzerland

    Konstantinos Sechidis

  37. Fondazione IRCCS Ca’ Granda Ospedale Maggiore Policlinico, Milan, Italy

    Arif Canakoglu

  38. Politecnico di Milano, Milan, Italy

    Sara Pido

  39. Politecnico di Milano, Milan, Italy

    Pietro Pinoli

  40. University of Waikato, Hamilton, New Zealand

    Albert Bifet

  41. Halmstad University, Halmstad, Sweden

    Sepideh Pashami

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Manerba, M.M., Morini, V. (2023). Exposing Racial Dialect Bias in Abusive Language Detection: Can Explainability Play a Role?. In: Koprinska, I., et al. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. ECML PKDD 2022. Communications in Computer and Information Science, vol 1752. Springer, Cham. https://doi.org/10.1007/978-3-031-23618-1_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23618-1_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23617-4

  • Online ISBN: 978-3-031-23618-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation