Abstract
Sarcasm is the acerbic use of words to mock someone or something, mostly in a satirical way. Scandal or mockery is used harshly, often crudely and contemptuously, for destructive purposes in sarcasm. To extract the actual sentiment of a sentence for code-mixed language is complex because of the unavailability of sufficient clues for sarcasm. In this work, we proposed a model consisting of Bidirectional Encoder Representations from Transformers (BERT) stacked with Long Short Term Memory (LSTM) (BERT-LSTM). A pre-trained BERT model is used to create embedding for the code-mixed dataset. These embedding vectors were used by an LSTM network consisting of a single layer to identify the nature of a sentence, i.e., sarcastic or non-sarcastic. The experiments show that the proposed BERT-LSTM model detects sarcastic sentences more effectively compared to other models on the code-mixed dataset, with an improvement of up to 6 % in terms of F1-score.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The dataset is available at https://github.com/sahilswami96/SarcasmDetection_CodeMixed
References
Agarwal, K., & Narula, R. (2021). Humor generation and detection in code-mixed Hindi-English. Proceedings of the Student Research Workshop Associated with RANLP, 2021, 1–6. https://doi.org/10.26615/issn.2603-2821.2021_001
Aggarwal, A., Wadhawan, A., Chaudhary, A. et al. (2020). did you really mean what you said?: Sarcasm detection in Hindi-English code-mixed data using bilingual word embeddings. arXiv:2010.00310. https://doi.org/10.48550/arXiv.2010.00310.
Alita, D., Priyanta, S., & Rokhman, N. (2019). Analysis of emoticon and sarcasm effect on sentiment analysis of Indonesian language on Twitter. Journal of Information Systems Engineering and Business Intelligence, 5, 100–109. https://doi.org/10.20473/jisebi.5.2.100-109
Bansal, S., Garimella, V., Suhane, A. et al. (2020). Code-switching patterns can be an effective route to improve performance of downstream NLP applications: A case study of humour, sarcasm and hate speech detection. arXiv:2005.02295. https://doi.org/10.48550/arXiv.2005.02295.
Bedi, M., Kumar, S., Akhtar, M. S., et al. (2021). Multi-modal sarcasm detection and humor classification in code-mixed conversations. IEEE Transactions on Affective Computing. https://doi.org/10.1109/TAFFC.2021.3083522
Berka, P. (2020). Sentiment analysis using rule-based and case-based reasoning. Journal of Intelligent Information Systems, 55, 51–66. https://doi.org/10.1007/s10844-019-00591-8
Bharti, S. K., Sathya Babu, K., Jena, S. K. (2017). Harnessing online news for sarcasm detection in Hindi tweets. In: International Conference on Pattern Recognition and Machine Intelligence (pp. 679–686). Springer. https://doi.org/10.1007/978-3-319-69900-4_86.
Biesialska, M., Biesialska, K., & Rybinski, H. (2021). Leveraging contextual embeddings and self-attention neural networks with bi-attention for sentiment analysis. Journal of Intelligent Information Systems, 57, 601–626. https://doi.org/10.1007/s10844-021-00664-7
Bouazizi, M., & Ohtsuki, T. O. (2016). A pattern-based approach for sarcasm detection on Twitter. IEEE Access, 4, 5477–5488. https://doi.org/10.1109/ACCESS.2016.2594194
Burfoot, C., Baldwin, T. (2009). Automatic satire detection: Are you having a laugh? In: Proceedings of the ACL-IJCNLP 2009 Conference short papers (pp. 161–164)
Buschmeier, K., Cimiano, P., Klinger, R. (2014). An impact analysis of features in a classification approach to irony detection in product reviews. In: Proceedings of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (pp. 42–49). https://doi.org/10.3115/v1/W14-2608.
Cai, Y., Cai, H., Wan, X. (2019). Multi-modal sarcasm detection in Twitter with hierarchical fusion model. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 2506–2515)
Carvalho, P., Sarmento, L., Silva, M. J. et al. (2009). Clues for detecting irony in user-generated contents: oh...!! it’s so easy. In: Proceedings of the 1st International CIKM Workshop on Topic-Sentiment Analysis for Mass Opinion (pp. 53–56). https://doi.org/10.1145/1651461.1651471.
Devlin, J., Chang, M.-W., Lee, K. et al. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. https://doi.org/10.48550/arXiv.1810.04805.
Dutta, S., Chakraborty, A. (2019). A deep learning-inspired method for social media satire detection. In: Soft Computing and Signal Processing (pp. 243–251). Springer. https://doi.org/10.1007/978-981-13-3393-4_25.
Eke, C. I., Norman, A. A., Shuib, L., & Nweke, H. F. (2020). Sarcasm identification in textual data: systematic review, research challenges and open directions. Artificial Intelligence Review, 53, 4215–4258. https://doi.org/10.1007/s10462-019-09791-8
Fang, X., & Zhan, J. (2015). Sentiment analysis using product review data. Journal of Big Data, 2, 1–14. https://doi.org/10.1186/s40537-015-0015-2
Ghosh, S., Ghosh, S., Das, D. (2017). Sentiment identification in code-mixed social media text. arXiv:1707.01184. https://doi.org/10.48550/arXiv.1707.01184.
González-Ibánez, R., Muresan, S., Wacholder, N. (2011). Identifying sarcasm in Twitter: a closer look. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (pp. 581–586). https://doi.org/10.5555/2002736.2002850.
Harish, B., & Kumar, K. (2019). Automatic irony detection using feature fusion and ensemble classifier. International Journal of Interactive Multimedia and Artificial Intelligence, 70–79,. https://doi.org/10.9781/ijimai.2019.07.002
Hiai, S., & Shimada, K. (2019). Sarcasm detection using RNN with relation vector. International Journal of Data Warehousing and Mining (IJDWM), 15, 66–78. https://doi.org/10.4018/IJDWM.2019100104
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9, 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Jain, D., Kumar, A., & Garg, G. (2020). Sarcasm detection in mash-up language using soft-attention based bi-directional LSTM and feature-rich CNN. Applied Soft Computing, 91, 1–15. https://doi.org/10.1016/j.asoc.2020.106198
Jamatia, A., Gambäck, B., Das, A. (2015). Part-of-speech tagging for code-mixed English-Hindi Twitter and Facebook chat messages. In: Proceedings of the International Conference Recent Advances in Natural Language Processing (pp. 239–248)
Joshi, A., Tripathi, V., Patel, K. et al. (2016a). Are word embedding-based features useful for sarcasm detection? arXiv:1610.00883. https://doi.org/10.48550/arXiv.1610.00883.
Joshi, A., Tripathi, V., Bhattacharyya, P. et al. (2016b). Harnessing sequence labeling for sarcasm detection in dialogue from TV series ‘Friends’. In: CoNLL (pp. 146–155). https://doi.org/10.18653/v1/K16-1015.
Khatri, A. et al. (2020). Sarcasm detection in tweets with BERT and GloVe embeddings. arXiv:2006.11512. https://doi.org/10.48550/arXiv.2006.11512.
Kumar, A., & Garg, G. (2019). Empirical study of shallow and deep learning models for sarcasm detection using context in benchmark datasets. Journal of Ambient Intelligence and Humanized Computing, 1–16,. https://doi.org/10.1007/s12652-019-01419-7
Kumar, A., Sangwan, S. R., Arora, A., et al. (2019). Sarcasm detection using soft attention-based bidirectional long short-term memory model with convolution network. IEEE ACCESS, 7, 23319–23328. https://doi.org/10.1109/ACCESS.2019.2899260
Liebrecht, C., Kunneman, F., van Den Bosch, A. (2013). The perfect solution for detecting sarcasm in tweets# not. In: 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. New Brunswick, NJ: ACL
Majumder, N., Poria, S., Peng, H., et al. (2019). Sentiment and sarcasm classification with multitask learning. IEEE Intelligent Systems, 34, 38–43. https://doi.org/10.1109/MIS.2019.2904691
Mehndiratta, P., & Soni, D. (2019). Identification of sarcasm in textual data: A comparative study. Journal of Data and Information Science, 4, 56–83. https://doi.org/10.2478/jdis-2019-0021
Mehndiratta, P., & Soni, D. (2019). Identification of sarcasm using word embeddings and hyperparameters tuning. Journal of Discrete Mathematical Sciences and Cryptography, 22, 465–489. https://doi.org/10.1080/09720529.2019
Naz, F., Kamran, M., Mehmood, W., et al. (2019). Automatic identification of sarcasm in tweets and customer reviews. Journal of Intelligent & Fuzzy Systems, 37, 6815–6828. https://doi.org/10.3233/JIFS-190596
Nelatoori, K. B., & Kommanti, H. B. (2022). Multi-task learning for toxic comment classification and rationale extraction. Journal of Intelligent Information Systems, 1–25,. https://doi.org/10.1007/s10844-022-00726-4
Neto, M. V. d. S., Amaral, A. D. d. S., da Silva, N. F. F. et al. (2020). Deep learning brasil–nlp at semeval-2020 task 9: Overview of sentiment analysis of code-mixed tweets. arXiv preprint arXiv:2008.01544. https://doi.org/10.48550/arXiv.2008.01544.
Pandey, A. C., Seth, S. R., Varshney, M. (2019). Sarcasm detection of amazon Alexa sample set. In: Advances in Signal Processing and Communication, 559–564. Springer. https://doi.org/10.1007/978-981-13-2553.
Pandey, R., Kumar, A., Singh, J. P., et al. (2021). Hybrid attention-based long short-term memory network for sarcasm identification. Applied Soft Computing, 106, 1–15. https://doi.org/10.1016/j.asoc.2021.107348
Parameswaran, P., Trotman, A., Liesaputra, V., et al. (2021). Detecting the target of sarcasm is hard: Really?? Information Processing & Management, 58, 1–22. https://doi.org/10.1016/j.ipm.2021.102599
Parameswaran, P., Trotman, A., Liesaputra, V. et al. (2021b). BERT’s the word : Sarcasm target detection using BERT. In: 19th Annual Workshop of the Australasian Language Technology Association (pp. 185–191) https://aclanthology.org/2021.alta-1.21.
Parshad, R. D., Bhowmick, S., Chand, V., et al. (2016). What is India speaking? exploring the Hinglish invasion. Physica A: Statistical Mechanics and its Applications, 449, 375–389. https://doi.org/10.1016/j.physa.2016.01.015
Rajadesingan, A., Zafarani, R., Liu, H. (2015). Sarcasm detection on Twitter: A behavioral modeling approach. In: Proceedings of the Eighth ACM International Conference on Web Search and Data Mining (pp. 97–106). https://doi.org/10.1145/2684822.2685316.
Rani, P., Suryawanshi, S., Goswami, K. et al. (2020). A comparative study of different state-of-the-art hate speech detection methods in Hindi-English code-mixed data. In: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying (pp. 42–48)
Ren, L., Xu, B., Lin, H., et al. (2020). Sarcasm detection with sentiment semantics enhanced multi-level memory network. Neurocomputing, 401, 320–326. https://doi.org/10.1016/j.neucom.2020.03.081
Riloff, E., Qadir, A., Surve, P. et al. (2013). Sarcasm as contrast between a positive sentiment and negative situation. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (pp. 704–714). https://doi.org/10.1.1.593.1531
Sabty, C., Elmahdy, M., Abdennadher, S. (2019). Named entity recognition on Arabic-English code-mixed data. In: 2019 IEEE 13th International Conference on Semantic Computing (ICSC) (pp. 93–97). IEEE. https://doi.org/10.1109/ICOSC.2019.8665500.
Savini, E., & Caragea, C. (2022). Intermediate-task transfer learning with BERT for sarcasm detection. Mathematics, 10, 844. https://doi.org/10.3390/math10050844
Shrawankar, U., & Chandankhede, C. (2019). Sarcasm detection for workplace stress management. International Journal of Synthetic Emotions (IJSE), 10, 1–17. https://doi.org/10.4018/IJSE.2019070101
Shukla, V., Sinha, M., Dasgupta, T. (2019). Automatic humor detection from code-mixed tweets. In: Proceedings of the 11th Forum for Information Retrieval Evaluation (pp. 56–59). https://doi.org/10.1145/3368567.3368576.
Singh, J. P., Irani, S., N. P. R., et al. (2017). Predicting the helpfulness of online consumer reviews. Journal of Business Research, 70, 346–355. https://doi.org/10.1016/j.jbusres.2016.08.008
Singh, L. G., & Singh, S. R. (2021). Empirical study of sentiment analysis tools and techniques on societal topics. Journal of Intelligent Information Systems, 56, 379–407. https://doi.org/10.1007/s10844-020-00616-7
Swami, S., Khandelwal, A., Singh, V. et al. (2018). A corpus of English-Hindi code-mixed tweets for sarcasm detection. arXiv:1805.11869. https://doi.org/10.48550/arXiv.1805.11869.
Vijay, D., Bohra, A., Singh, V. et al. (2018). A dataset for detecting irony in Hindi-English code-mixed social media text. EMSASW@ ESWC, 2111, 38–46.
Wang, Z., Wu, Z., Wang, R. et al. (2015). Twitter sarcasm detection exploiting a context-based model. In: International Conference on Web Information Systems Engineering (pp. 77–91). Springer. https://doi.org/10.1007/978-3-319-26190-4_6.
Zhang, Y., Ma, D., Tiwari, P., et al. (2022). Stance level sarcasm detection with BERT and stance-centred graph attention networks. ACM Transactions on Internet Technology (TOIT). https://doi.org/10.1145/3533430
Acknowledgements
The authors acknowledge the dataset creators for their support.
Author information
Authors and Affiliations
Contributions
Rajnish Pandey and Jyoti Prakash Singh conceived the idea of the BERT embedding and LSTM network to classify the text. The experiments and initial draft were developed by Rajnish Pandey. Jyoti Prakash Singh corrected the initial draft. All authors reviewed the manuscript.
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pandey, R., Singh, J.P. BERT-LSTM model for sarcasm detection in code-mixed social media post. J Intell Inf Syst 60, 235–254 (2023). https://doi.org/10.1007/s10844-022-00755-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10844-022-00755-z