Abstract
The detection of abuse language remains a long-standing challenge with the extensive use of social networks. The detection task of abuse language suffers from limited accuracy. We argue that the existing detection methods utilize the fine-tuning technique of the pre-trained language models (PLMs) to handle downstream tasks. Hence, these methods fail to stimulate the general knowledge of the PLMs. To address the problem, we propose a novel Deep Prompt Multi-task Network (DPMN) for abuse language detection. Specifically, DPMN first attempts to design two forms of deep prompt tuning and light prompt tuning for the PLMs. The effects of different prompt lengths, tuning strategies, and prompt initialization methods on detecting abuse language are studied. In addition, we propose a Task Head based on Bi-LSTM and FFN, which can be used as a short text classifier. Eventually, DPMN utilizes multi-task learning to improve detection metrics further. The multi-task network has the function of transferring effective knowledge. The proposed DPMN is evaluated against eight typical methods on three public datasets: OLID, SOLID, and AbuseAnalyzer. The experimental results show that our DPMN outperforms the state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541ā551 (1989)
Mikolov, T., KarafiĆ”t, M., Burget, L., Cernockį»³, J., Khudanpur, S.: Recurrent neural network based language model. In: Interspeech, vol. 2, pp. 1045ā1048. Makuhari (2010)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Radford, A., Narasimhan, K., Sutskever, I., Salimans, T., et al.: Improving language understanding by generative pre-training (2018)
Liu, P., Li, W., Zou, L.: Nuli at semeval-2019 task 6: transfer learning for offensive language detection using bidirectional transformers. In: SemEval@ NAACL-HLT, pp. 87ā91 (2019)
Chandra, M., et al.: AbuseAnalyzer: abuse detection, severity and target prediction for gab posts. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6277ā6283. International Committee on Computational Linguistics (2020)
Dai, W., Yu, T., Liu, Z., Fung, P.: Kungfupanda at semeval-2020 task 12: bert-based multi-task learning for offensive language detection. arXiv preprint arXiv:2004.13432 (2020)
Wiedemann, G., Yimam, S.M., Biemann, C.: Uhh-lt at semeval-2020 task 12: fine-tuning of pre-trained transformer networks for offensive language detection. arXiv preprint arXiv:2004.11493 (2020)
Hakimov, S., Ewerth, R.: Combining textual features for the detection of hateful and offensive language. arXiv preprint arXiv:2112.04803 (2021)
Zhang, S., Zheng, D., Hu, X., Yang, M.: Bidirectional long short-term memory networks for relation classification. In: Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pp. 73ā78 (2015)
Bebis, G., Georgiopoulos, M.: Feed-forward neural networks. IEEE Pot. 13(4), 27ā31 (1994)
Kumar, R., Ojha, A.K., Malmasi, S., Zampieri, M.: Benchmarking aggression identification in social media. In: Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 1ā11 (2018)
Huang, Q., Singh, V.K., Atrey, P.K.: Cyber bullying detection using social and textual analysis. In: Proceedings of the 3rd International Workshop on Socially-aware Multimedia, pp. 3ā6 (2014)
Park, J.H., Fung, P.: One-step and two-step classification for abusive language detection on twitter. arXiv preprint arXiv:1706.01206 (2017)
Xu, J.M., Jun, K.S., Zhu, X., Bellmore, A.: Learning from bullying traces in social media. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 656ā666 (2012)
Kwok, I., Wang, Y.: Locate the hate: detecting tweets against blacks. In: Twenty-Seventh AAAI Conference on Artificial Intelligence (2013)
Burnap, P., Williams, M.L.: Cyber hate speech on twitter: an application of machine classification and statistical modeling for policy and decision making. Policy Internet 7(2), 223ā242 (2015)
Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., Bhamidipati, N.: Hate speech detection with comment embeddings. In: Proceedings of the 24th International Conference on World Wide Web, pp. 29ā30 (2015)
Davidson, T., Warmsley, D., Macy, M., Weber, I.: Automated hate speech detection and the problem of offensive language. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 11, pp. 512ā515 (2017)
Chen, Y., Zhou, Y., Zhu, S., Xu, H.: Detecting offensive language in social media to protect adolescent online safety. In: 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Conference on Social Computing, pp. 71ā80. IEEE (2012)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., Kumar, R.: Predicting the type and target of offensive posts in social media. arXiv preprint arXiv:1902.09666 (2019)
Rosenthal, S., Atanasova, P., Karadzhov, G., Zampieri, M., Nakov, P.: A large-scale semi-supervised dataset for offensive language identification. arXiv preprint arXiv:2004.14454 (2020)
Wang, Y.S., Chang, Y.: Toxicity detection with generative prompt-based inference. arXiv preprint arXiv:2205.12390 (2022)
Brown, T., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems, vol. 33, pp. 1877ā1901 (2020)
Ben-David, E., Oved, N., Reichart, R.: Pada: a prompt-based autoregressive approach for adaptation to unseen domains. arXiv preprint arXiv:2102.12206 (2021)
Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021)
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021)
Lu, Y., Bartolo, M., Moore, A., Riedel, S., Stenetorp, P.: Fantastically ordered prompts and where to find them: overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786 (2021)
Reynolds, L., McDonell, K.: Prompt programming for large language models: beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1ā7 (2021)
Le Scao, T., Rush, A.M.: How many data points is a prompt worth? arXiv preprint arXiv:2103.08493 (2021)
Hu, S., Ding, N., Wang, H., Liu, Z., Li, J., Sun, M.: Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification. arXiv preprint arXiv:2108.02035 (2021)
Gao, T., Fisch, A., Chen, D.: Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723 (2020)
Shin, T., Razeghi, Y., Logan IV, R.L., Wallace, E., Singh, S.: Autoprompt: eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980 (2020)
Hambardzumyan, K., Khachatrian, H., May, J.: Warp: word-level adversarial reprogramming. arXiv preprint arXiv:2101.00121 (2021)
He, J., Zhou, C., Ma, X., Berg-Kirkpatrick, T., Neubig, G.: Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366 (2021)
Li, X.L., Liang, P.: Prefix-tuning: optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190 (2021)
Liu, X., et al.: Gpt understands, too. arXiv preprint arXiv:2103.10385 (2021)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., Kumar, R.: Semeval-2019 task 6: identifying and categorizing offensive language in social media (offenseval). arXiv preprint arXiv:1903.08983 (2019)
Chandra, M., et al. Abuseanalyzer: abuse detection, severity and target prediction for gab posts. arXiv preprint arXiv:2010.00038 (2020)
Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., Klein, M.: Logistic Regression. Springer, Heidelberg (2002)
Chen, T., Guestrin, C.: Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, pp. 785ā794 (2016)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123ā140 (1996)
Hearst, M.A., Dumais, S.T., Osuna, E., Platt, J., Scholkopf, B.: Support vector machines. IEEE Intell. Syst. Appl. 13(4), 18ā28 (1998)
Acknowledgment
This work was supported by the National Key Research and Development Program of China (Grant No. 2021ZD0201501), the Youth Foundation Project of Zhejiang Province (Grant No. LQ22F020035), the National Natural Science Foundation of China (No. 32200860), and the Youth Foundation Project of Zhejiang Province (Grant No. LQ22F020035).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, J. et al. (2025). Deep Prompt Multi-task Network for Abuse Language Detection. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15301. Springer, Cham. https://doi.org/10.1007/978-3-031-78107-0_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-78107-0_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78106-3
Online ISBN: 978-3-031-78107-0
eBook Packages: Computer ScienceComputer Science (R0)