Skip to main content
SpringerLink
Log in

Class-biased sarcasm detection using BiLSTM variational autoencoder-based synthetic oversampling

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

A Correction to this article was published on 27 March 2023

This article has been updated

Abstract

Recent research works have established the importance of sarcasm detection in the domain of sentiment analysis. Automatic sarcasm detection using social media data is a challenging task in the presence of imbalanced classes. Real-life social media data often suffer from this problem of class imbalance resulting in dramatical degradation of the performance of classification models attempting to detect sarcasm. Motivated by this, in the current article, a Bi-LSTM variational autoencoder model has been proposed to alleviate the problem of imbalanced classes in social media datasets targeted to train sarcasm detection models. The proposed BVA model is trained with a large corpus of sarcastic and non-sarcastic tweets to obtain the most suitable latent space representation of the same. These inherently class-biased latent vectors are then oversampled using synthetic minority oversampling techniques. The quality of the proposed method is established by training and testing a set of well-known classifiers in terms of precision, recall, F1-score, AUC, and G-mean. Extensive experiments reveal that the proposed BVA model combined with oversampling techniques can improve classifier performance for sarcasm detection to a greater extent.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availibility Statement

Enquiries about data availability should be directed to the authors.

Change history

References

  • Abercrombie G, Hovy D (2016) Putting sarcasm detection into context: the effects of class imbalance and manual labelling on supervised machine classification of twitter conversations. In: Proceedings of the ACL 2016 student research workshop, pp 107–113

  • Bamman D, Smith N (2015) Contextualized sarcasm detection on twitter. Proc Int AAAI Conf Web Soc Media 9:574–577

    Article  Google Scholar 

  • Banerjee A, Bhattacharjee M, Ghosh K, Chatterjee S (2020) Synthetic minority oversampling in addressing imbalanced sarcasm detection in social media. Multimedia Tools Appl 79(47):35995–36031

    Article  Google Scholar 

  • Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13(2)

  • Bharti SK, Babu KS, Jena SK (2015) Parsing-based sarcasm sentiment recognition in twitter data. In: 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), pp 1373–1380. IEEE

  • Bowman SR, Vilnis L, Vinyals O, Dai AM, Jozefowicz R, Bengio S (2015) Generating sentences from a continuous space. arXiv preprint arXiv:1511.06349

  • Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2012) Dbsmote: density-based synthetic minority over-sampling technique. Appl Intell 36(3):664–684

    Article  Google Scholar 

  • Chaudhari P, Chandankhede C (2017) Literature survey of sarcasm detection. In 2017 International conference on wireless communications, signal processing and networking (WiSPNET), pp 2041–2046. IEEE

  • Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

  • Cheng M, Nazarian S, Bogdan P (2020) Vroc: variational autoencoder-aided multi-task rumor classifier based on text. Proc Web Conf 2020:2892–2898

    Google Scholar 

  • Dablain D, Krawczyk B, Chawla NV (2022) Deepsmote: fusing deep learning and smote for imbalanced data. IEEE Trans Neural Netw Learn Syst

  • Douzas G, Bacao F, Last F (2018) Improving imbalanced learning through a heuristic oversampling method based on k-means and smote. Inf Sci 465:1–20

    Article  Google Scholar 

  • Dozat T (2016) Incorporating nesterov momentum into adam

  • Eke CI, Norman AA, Shuib L (2021) Context-based feature technique for sarcasm identification in benchmark datasets using deep learning and bert model. IEEE Access 9:48501–48518

    Article  Google Scholar 

  • Ertekin S (2013) Adaptive oversampling for imbalanced data classification. Inf Sci Syst 2013, pp 261–269. Springer

  • Fei H, Ren Y, Shengqiong W, Li B, Ji D (2021) Latent target-opinion as prior for document-level sentiment classification: a variational approach from fine-grained perspective. Proc Web Conf 2021:553–564

    Google Scholar 

  • Gentina E, Chen R, Yang Z (2021) Development of theory of mind on online social networks: evidence from facebook, twitter, instagram, and snapchat. J Bus Res 124:652–666

    Article  Google Scholar 

  • Ghosh K, Bellinger C, Corizzo R, Krawczyk B, Japkowicz N (2021) On the combined effect of class imbalance and concept complexity in deep learning. In: 2021 IEEE international conference on big data (big data), pp 4859–4868. IEEE

  • Gibbs Raymond W (2000) Irony in talk among friends. Metaphor Symbol 15(1–2):5–27

    Article  Google Scholar 

  • Guo S, Chen R, Li H, Zhang T, Liu Y (2019) Identify severity bug report with distribution imbalance by cr-smote and elm. Int J Softw Eng Knowl Eng 29(02):139–175

    Article  Google Scholar 

  • Guo X, Li B, Yu H, Miao C (2021) Latent-optimized adversarial neural transfer for sarcasm detection. arXiv preprint arXiv:2104.09261

  • Han H, Wang W-Y, Mao B-H (2005) Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International conference on intelligent computing, pp 878–887. Springer

  • Hazarika D, Poria S, Gorantla S, Cambria E, Zimmermann R, Mihalcea R (2018) Cascade: contextual sarcasm detection in online discussion forums. arXiv preprint arXiv:1805.06413

  • He H, Bai Y, Garcia EA, Li S (2008) Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE international joint conference on neural networks (IEEE world congress on computational intelligence), pp 1322–1328. IEEE

  • Hsu Bi-Min (2020) Comparison of supervised classification models on textual data. Mathematics 8(5):851

    Article  Google Scholar 

  • Huang C, Trabelsi A, Qin X, Farruque N, Mou L, Zaiane OR (2021) Seq2emo: a sequence to multi-label emotion classification model. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 4717–4724

  • Hwang K, Sung W (2017) Character-level language modeling with hierarchical recurrent neural networks. In: 2017 IEEE international conference on acoustics, speech and signal processing (icassp), pp 5720–5724. IEEE

  • Jang B, Kim M, Harerimana G, Kang S, Kim JW (2020) Bi-lstm model to increase accuracy in text classification: combining word2vec cnn and attention mechanism. Appl Sci 10(17):5841

    Article  Google Scholar 

  • Japkowicz N, Stephen S (2002) The class imbalance problem: a systematic study. Intell Data Anal 6(5):429–449

    Article  MATH  Google Scholar 

  • Khotijah S, Tirtawangsa J, Suryani AA (2020) Using lstm for context based approach of sarcasm detection in twitter. In: Proceedings of the 11th international conference on advances in information technology, pp 1–7

  • Kim Y, Geng J, Ney H (2019) Improving unsupervised word-by-word translation with language model and denoising autoencoder. arXiv preprint arXiv:1901.01590

  • Kocher G, Kumar G (2021) Machine learning and deep learning methods for intrusion detection systems: recent developments and challenges. Soft Comput 25(15):9731–9763

    Article  Google Scholar 

  • Kubat M, Holte R, Matwin S (1997) Learning when negative examples abound. In: European conference on machine learning, pp 146–153. Springer

  • Kubat M, Matwin S et al. (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: Icml, vol 97, p 179. Citeseer

  • Kumar A, Narapareddy VT, Srikanth VA, Malapati A, Neti LBM (2020) Sarcasm detection using multi-head attention based bidirectional lstm. IEEE Access 8:6388–6397

    Article  Google Scholar 

  • Kumar A, Garg G (2019) Empirical study of shallow and deep learning models for sarcasm detection using context in benchmark datasets. J Ambient Intell Human Comput, pp 1–16

  • Li L, Zhang Q, Wang X, Zhang J, Wang T, Gao Tian-Lu, Duan Wei, Tsoi Kelvin Kam-fai, Wang Fei-Yue (2020) Characterizing the propagation of situational information in social media during covid-19 epidemic: a case study on weibo. IEEE Trans Comput Soc Syst 7(2):556–562

    Article  Google Scholar 

  • Liang XW, Jiang AP, Li T, Xue YY, Wang GT (2020) Lr-smote-an improved unbalanced data set oversampling based on k-means and svm. Knowl Based Syst 196:105845

    Article  Google Scholar 

  • Li Z, Han J, Li Q et al. (2020) On the curse of memory in recurrent neural networks: approximation and optimization analysis. arXiv preprint arXiv:2009.07799

  • Li R, Li X, Lin C, Collinson M, Mao R (2019) A stable variational autoencoder for text modelling. arXiv preprint arXiv:1911.05343

  • Liu G, Guo J (2019) Bidirectional lstm with attention mechanism and convolutional layer for text classification. Neurocomputing 337:325–338

    Article  Google Scholar 

  • Liu F, Zheng L, Zheng J (2020) Hienn-dwe: a hierarchical neural network with dynamic word embeddings for document level sentiment classification. Neurocomputing 403:21–32

    Article  Google Scholar 

  • Liu P, Chen W, Ou G, Wang T, Yang D, Lei K (2014) Sarcasm detection in social media based on imbalanced classification. In: International conference on web-age information management, pp 459–471. Springer

  • Liu L, Priestley JL, Zhou Y, Ray HE, Han M (2019) A2text-net: A novel deep neural network for sarcasm detection. In: 2019 IEEE first international conference on cognitive machine intelligence (CogMI), pp 118–126. IEEE

  • Ma L, Fan S (2017) Cure-smote algorithm and hybrid algorithm for feature selection and parameter optimization based on random forests. BMC Bioinf 18(1):1–18

    Article  MathSciNet  Google Scholar 

  • Maity S, Mandal RP, Bhattacharjee S, Chatterjee S (2022) Variational autoencoder-based imbalanced alzheimer detection using brain mri images. In: Proceedings of international conference on computational intelligence, data science and cloud computing: IEM-ICDC 2021, pp 165–178. Springer

  • Majumder Navonil, Poria Soujanya, Peng Haiyun, Chhaya Niyati, Cambria Erik, Gelbukh Alexander (2019) Sentiment and sarcasm classification with multitask learning. IEEE Intell Syst 34(3):38–43

    Article  Google Scholar 

  • Marchi E, Vesperini F, Eyben F, Squartini S, Schuller B (2015) A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional lstm neural networks. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 1996–2000. IEEE

  • Mukherjee S, Bala PK (2017) Sarcasm detection in microblogs using naïve bayes and fuzzy clustering. Technol Soc 48:19–27

  • Ning Q, Zhao X, Ma Z (2021) A novel method for identification of glutarylation sites combining borderline-smote with tomek links technique in imbalanced data. IEEE/ACM Trans Comput Biol Bioinf

  • Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  • Prokhorov V, Shareghi E, Li Y, Pilehvar MT, Collier N (2019) On the importance of the kullback-leibler divergence term in variational autoencoders for text generation. arXiv preprint arXiv:1909.13668

  • Rajadesingan A, Zafarani R, Liu H (2015) Sarcasm detection on twitter: a behavioral modeling approach. In: Proceedings of the eighth ACM international conference on web search and data mining, pp 97–106

  • Ren L, Bo X, Lin H, Liu X, Yang L (2020) Sarcasm detection with sentiment semantics enhanced multi-level memory network. Neurocomputing 401:320–326

    Article  Google Scholar 

  • Ruz GA, Henríquez PA, Mascareno A (2020) Sentiment analysis of twitter data during critical events through bayesian networks classifiers. Future Generation Comput Syst 106:92–104

  • Sáez J, Luengo J, Stefanowski J, Herrera F (2015) Smote-ipf: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering. Inf Sci 291:184–203

    Article  Google Scholar 

  • Salehinejad H, Sankar S, Barfett J, Colak E, Valaee S (2017) Recent advances in recurrent neural networks. arXiv preprint arXiv:1801.01078

  • Sarsam SM, Al-Samarraie H, Alzahrani AI, Wright B (2020) Sarcasm detection using machine learning algorithms in twitter: a systematic review. Int J Market Res 62(5):578–598

    Article  Google Scholar 

  • Shahi GK, Dirkson A, Majchrzak TA (2021) An exploratory study of covid-19 misinformation on twitter. Online Social Netw Media 22:100104

    Article  Google Scholar 

  • Shah B, Shah M (2021) A survey on machine learning and deep learning based approaches for sarcasm identification in social media. Data Sci Intell Appl, pp 247–259. Springer

  • Shen T, Mueller J, Barzilay R, Jaakkola T (2020) Educating text autoencoders: Latent representation guidance via denoising. In: International conference on machine learning, pp 8719–8729. PMLR

  • Singh Raghuwanshi Bhagat, Sanyam Shukla (2020) Smote based class-specific extreme learning machine for imbalanced learning. Knowledge-Based Systems 187:104814

    Article  Google Scholar 

  • Smiti S, Soui M (2020) Bankruptcy prediction using deep learning approach based on borderline smote. Inf Syst Front 22(5):1067–1083

    Article  Google Scholar 

  • Suhaimin MSM, Hijazi MHA, Alfred R, Coenen F (2017) Natural language processing based features for sarcasm detection: an investigation using bilingual social media texts. In: 2017 8th International conference on information technology (ICIT), pp 703–709. IEEE

  • Sun J, Li H, Fujita H, Binbin F, Ai W (2020) Class-imbalanced dynamic financial distress prediction based on adaboost-svm ensemble combined with smote and time weighting. Inf Fusion 54:128–144

    Article  Google Scholar 

  • Wang H-Y (2008) Combination approach of smote and biased-svm for imbalanced datasets. In: 2008 IEEE international joint conference on neural networks (IEEE World Congress on Computational Intelligence), pp 228–231. IEEE,

  • Wankhade M, Rao ACS, Kulkarni C (2022) A survey on sentiment analysis methods, applications, and challenges. Artif Intell Rev 55(7):5731-5780

    Article  Google Scholar 

  • Xu Z, Shen D, Nie T, Kou Y (2020) A hybrid sampling algorithm combining m-smote and enn based on random forest for medical imbalanced data. J Biomed Inf 107:103465

    Article  Google Scholar 

  • Xu Z, Shen D, Nie T, Kou Y, Yin N, Han X (2021) A cluster-based oversampling algorithm combining smote and k-means for imbalanced medical data. Inf Sci 572:574–589

    Article  MathSciNet  Google Scholar 

  • Xu W, Sun H, Deng C, Tan Y(2017) Variational autoencoder for semi-supervised text classification. In: Thirty-first AAAI conference on artificial intelligence

  • Yang Z, Hu Z, Salakhutdinov R, Berg-Kirkpatrick T (2017) Improved variational autoencoders for text modeling using dilated convolutions. In: International conference on machine learning, pp 3881–3890. PMLR

  • Zhang Y, Liu Y, Li Q, Tiwari P, Wang B, Li Y, Pandey HM, Zhang P, Song D (2021) Cfn: a complex-valued fuzzy network for sarcasm detection in conversations. IEEE Trans Fuzzy Syst 29(12):3696–3710

    Article  Google Scholar 

  • Zhang J, Li X (2017) Phishing detection method based on borderline-smote deep belief network. In: International conference on security, privacy and anonymity in computation, communication and storage, pp 45–53. Springer

  • Zhao J, Jin J, Chen S, Ruifeng Z, Yu B, Liu Q (2020) A weighted hybrid ensemble method for classifying imbalanced data. Knowl Based Syst 203:106087

    Article  Google Scholar 

  • Zheng X, Han J, Sun A (2018) A survey of location prediction on twitter. IEEE Trans Knowl Data Eng 30(9):1652–1671

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this study.

Author information

Authors and Affiliations

  1. Department of Computer Science and Technology, Indian Institute of Engineering Science and Technology, Shibpur, Howrah, India

    Sankhadeep Chatterjee & Asit Kumar Das

  2. Department of Computer Science and Engineering, University of Engineering and Management, Kolkata, India

    Saranya Bhattacharjee

  3. Department of Computing Science, University of Alberta, Edmonton, Canada

    Kushankur Ghosh

  4. Department of Electronics and Communication Engineering, Budge Budge Institute of Technology, Budge Budge, Kolkata, 700137, West Bengal, India

    Soumen Banerjee

Authors
  1. Sankhadeep Chatterjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Saranya Bhattacharjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kushankur Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Asit Kumar Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Soumen Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Sankhadeep Chatterjee was involved in the conceptualization; Sankhadeep Chatterjee contributed to the methodology; Sankhadeep Chatterjee and Kushankur Ghosh assisted in the formal analysis and investigation; Saranya Bhattacharjee and Sankhadeep Chatterjee contributed to the simulation; Sankhadeep Chatterjee, Saranya Bhattacharjee, and Kushankur Ghosh assisted in writing—original draft preparation; Soumen Banerjee and Sankhadeep Chatterjee were involved in writing—review and editing; Asit Kumar Das contributed to the resources; Asit Kumar Das and Soumen Banerjee were involved in the supervision.

Corresponding author

Correspondence to Sankhadeep Chatterjee.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original article has been updated: Due to affiliation update.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chatterjee, S., Bhattacharjee, S., Ghosh, K. et al. Class-biased sarcasm detection using BiLSTM variational autoencoder-based synthetic oversampling. Soft Comput 27, 5603–5620 (2023). https://doi.org/10.1007/s00500-023-07956-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-023-07956-w

Keywords

Search

Navigation