Abstract
Humor and pun are widely used in our daily life. Due to the incongruous and inconsistent expressions in humor and pun, humor recognition and pun detection are challenging tasks in sentiment analysis and natural language processing. Pun is usually used as one of the humorous rhetorical devices. Thus, determining the puns in the texts can be pertinent to the successful recognition of humor and vice versa. Existing models of humor recognition and pun detection usually dealt with the two tasks independently. Therefore, we propose an attention-based multi-task learning model (ABML) to train the two tasks jointly. Our model unifies the two highly pertinent tasks, including the humor recognition and pun detection. In the ABML model, we design a Co-Encoder module to capture the common features between the two tasks by weight sharing. Apart from the Co-Encoder module, we also design two Private Encoder modules for the two tasks, respectively. The Private Encoder module is used to capture the private semantic feature of the two tasks. We conduct sufficient experiments on two widely used datasets, including a humor dataset and a pun dataset, to verify the effectiveness of our model. The experimental results show our model can get comparable performance compared with state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The pun dataset is available in the link [http://alt.qcri.org/semeval2017/task7/]. The 16000 one-liner dataset is available from the corresponding author (Mihalcea and Strapparava 2005) on reasonable request.
References
Anderson CA, Arnoult LH (1989) An examination of perceived control, humor, irrational beliefs, and positive stress as moderators of the relation between negative stress and health. Basic Appl Soc Psychol 10(2):101–117
Ando RK, Zhang T (2005) A framework for learning predictive structures from multiple tasks and unlabeled data. J Mach Learn Res 6:1817–1853
Bertero D, Fung P (2016a) Deep learning of audio and language features for humor prediction. In: Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pp 496–501
Bertero D, Fung P (2016b) A long short-term memory framework for predicting humor in dialogues. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 130–135
Bertero D, Fung P (2016c) Predicting humor response in dialogues from tv sitcoms. 2016 IEEE International Conference on Acoustics. Speech and Signal Processing (ICASSP), IEEE, pp 5780–5784
Bohnet B, Nivre J (2012) A transition-based system for joint part-of-speech tagging and labeled non-projective dependency parsing. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Association for Computational Linguistics, pp 1455–1465
Buscaldi D, Rosso P (2007) Some experiments in humour recognition using the italian wikiquote collection. In: International Workshop on Fuzzy Logic and Applications, Springer, pp 464–468
Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75
Chen LC, Lee CM, Chen MY (2019) Exploration of social media for sentiment analysis using deep learning. Soft Computing (1)
Chen PY, Soo VW (2018) Humor recognition using deep learning. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp 113–117
Collobert R, Weston J (2008) A unified architecture for natural language processing: Deep neural networks with multitask learning. In: Proceedings of the 25th international conference on Machine learning, ACM, pp 160–167
Devlin J, Chang MW, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT
Diao Y, Yang L, Zhang D, Xu L, Fan X, Wu D, Lin H (2017) Homographic puns recognition based on latent semantic structures. In: National CCF Conference on Natural Language Processing and Chinese Computing, Springer, pp 565–576
Diao Y, Lin H, Wu D, Yang L, Xu K, Yang Z, Wang J, Zhang S, Xu B, Zhang D (2018) Weca: A wordnet-encoded collocation-attention network for homographic pun recognition. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 2507–2516
Diao Y, Lin H, Yang L, Fan X, Wu D, Zhang D, Xu K (2019) Heterographic pun recognition via pronunciation and spelling understanding gated attention network. In: The World Wide Web Conference, ACM, pp 363–371
Diao Y, Lin H, Yang L, Fan X, Xu K (2020) Homographic pun location using multi-dimensional semantic relationships. Soft Computing 24(4)
Fan X, Lin H, Yang L, Diao Y, Shen C, Chu Y, Zou Y (2020) Humor detection via an internal and external neural network. Neurocomputing
Feng J, Sevgili Ö, Remus S, Ruppert E, Biemann C (2020) Supervised pun detection and location with feature engineering and logistic regression. In: SwissText/KONVENS
Graves A, Jaitly N, Mohamed Ar (2013) Hybrid speech recognition with deep bidirectional lstm. In: 2013 IEEE workshop on automatic speech recognition and understanding, IEEE, pp 273–278
Hatori J, Matsuzaki T, Miyao Y, Tsujii J (2012) Incremental joint approach to word segmentation, pos tagging, and dependency parsing in chinese. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, Association for Computational Linguistics, pp 1045–1053
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Huang YH, Huang HH, Chen HH (2017) Identification of homographic pun location for pun understanding. In: Proceedings of the 26th International Conference on World Wide Web Companion, International World Wide Web Conferences Steering Committee, pp 797–798
Jaech A, Koncel-Kedziorski R, Ostendorf M (2016) Phonological pun-derstanding. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 654–663
Kant I (1790) Kritik der urteilskraft [critique of judgment]. Lagarde und Friederich, Berlin
Kao JT, Levy R, Goodman ND (2016) A computational model of linguistic humor in puns. Cogn Sci 40(5):1270–1285
Liu L, Zhang D, Song W (2018a) Exploiting syntactic structures for humor recognition. In: Proceedings of the 27th International Conference on Computational Linguistics, pp 1875–1883
Liu L, Zhang D, Song W (2018b) Modeling sentiment association in discourse for humor recognition. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp 586–591
Liu P, Qiu X, Huang X (2016) Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv:1605.05101
Liu X, Gao J, He X, Deng L, Duh K, Wang YY (2015) Representation learning using multi-task deep neural networks for semantic classification and information retrieval
Ma J, Gao W, Wong KF (2018) Detect rumor and stance jointly by neural multi-task learning. In: Companion proceedings of the the web conference 2018, pp 585–593
Ma J, Xie S, Jin M, Lianxin J, Yang M, Shen J (2020) Xsysigma at semeval-2020 task 7: Method for predicting headlines’ humor based on auxiliary sentences with ei-bert. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp 1077–1084
Mao J, Liu W (2019) A bert-based approach for automatic humor detection and scoring. In: IberLEF@ SEPLN, pp 197–202
Martineau WH (1972) A model of the social functions of humor. Theoretical perspectives and empirical issues, The psychology of humor, pp 101–125
Mihalcea R, Strapparava C (2005) Making computers laugh: Investigations in automatic humor recognition. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Association for Computational Linguistics, pp 531–538
Miller T, Gurevych I (2015) Automatic disambiguation of english puns. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp 719–729
Miller T, Turković M (2016) Towards the automatic detection and identification of english puns. European J Humour Res 4(1):59–75
de Oliveira L, Rodrigo AL (2017) Humor detection in yelp reviews
Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Raz Y (2012) Automatic humor classification on twitter. In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop, Association for Computational Linguistics, pp 66–70
Reyes A, Buscaldi D, Rosso P (2009) An analysis of the impact of ambiguity on automatic humour recognition. International Conference on Text. Springer, Speech and Dialogue, pp 162–169
Sane SR, Tripathi S, Sane KR, Mamidi R (2019) Deep learning techniques for humor detection in hindi-english code-mixed tweets. In: Proceedings of the Tenth Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp 57–61
Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. arXiv:- Learning
Tanaka K (1992) The pun in advertising: a pragmatic approach. Lingua 87(1–2):91–102
Wu H, Cheng S, Wang Z, Zhang S, Yuan F (2020) Multi-task learning based on question-answering style reviews for aspect category classification and aspect term extraction on gpu clusters. Clust Comput 23(3):1973–1986
Yang D, Lavie A, Dyer C, Hovy E (2015) Humor recognition and humor anchor extraction. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp 2367–2376
Zhang C, Yamana H (2020) Wuy at semeval-2020 task 7: Combining bert and naïve bayes-svm for humor assessment in edited news headlines. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation, pp 1071–1076
Zhang S, Hu Z, Zhu G, Jin M, Li KC (2020) Sentiment classification model for chinese micro-blog comments based on key sentences extraction. Soft Computing (1)
Zhu D (2019) Humor robot and humor generation method based on big data search through iot. Clust Comput 22(4):9169–9175
Zou Y, Lu W (2019) Joint detection and location of english puns. arXiv preprint arXiv:1909.00175
Acknowledgements
This study was partially supported by a grant from the Natural Science Foundation of China (No. 62076046, 61632011, 62006034, 61876031), the Ministry of Education Humanities and Social Science Project (No. 19YJCZH199), State Key Laboratory of Novel Software Technology (Nanjing University) (No. KFKT2021B07, and the Fundamental Research Funds for the Central Universities (No.DUT21RC(3)015).
Author information
Authors and Affiliations
Contributions
LR helped in conceptualization, methodology, data processing, experiments, visualization, writing.
BX contributed to data processing, writing, review, investigation.
HL helped in conceptualization, supervision, review.
LY was involved in review and editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animal rights
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ren, L., Xu, B., Lin, H. et al. ABML: attention-based multi-task learning for jointly humor recognition and pun detection. Soft Comput 25, 14109–14118 (2021). https://doi.org/10.1007/s00500-021-06136-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-021-06136-y