Abstract
Adversarial training has shown expressive performance in image classification task. However, there are few applications in natural language processing domain. In this paper, we propose to apply adversarial training strategy to sarcasm detection with small labeled samples. Several different neural network architectures are adopted including Convolutional Neural Networks (CNN) and Hierarchical Recurrent Neural Networks (HRNN). The experimental results on three datasets show that adversarial training is effective to improve the performance on sarcasm detection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Amir, S., Wallace, B.C., Lyu, H., Carvalho, P., Silva, M.J.: Modelling context with user embeddings for sarcasm detection in social media. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, 11–12 August 2016, pp. 167–177 (2016). http://aclweb.org/anthology/K/K16/K16-1017.pdf
Bamman, D.: Contextualized sarcasm detection on Twitter (2015)
Carvalho, P., Sarmento, L., Silva, M.J., De Oliveira, E.: Clues for detecting irony in user-generated contents: oh...!! it’s “so easy”;-) pp. 53–56 (2009)
Felbo, B., Mislove, A., Søgaard, A., Rahwan, I., Lehmann, S.: Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, 9–11 September 2017, pp. 1615–1625 (2017). https://aclanthology.info/papers/D17-1169/d17-1169
Ghosh, D., Guo, W., Muresan, S.: Sarcastic or not: word embeddings to predict the literal or sarcastic meaning of words. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp. 1003–1012 (2015)
González-Ibáñez, R.I., Muresan, S., Wacholder, N.: Identifying sarcasm in Twitter: a closer look. In: Proceedings of the Conference on the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Short Papers, Portland, Oregon, USA, 19–24 June 2011, pp. 581–586 (2011). http://www.aclweb.org/anthology/P11-2102
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. CoRR abs/1412.6572 (2014). http://arxiv.org/abs/1412.6572
Guo, J., Gould, S.: Deep CNN ensemble with data augmentation for object detection. CoRR abs/1506.07224 (2015). http://arxiv.org/abs/1506.07224
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6–11 July 2015, pp. 448–456 (2015). http://jmlr.org/proceedings/papers/v37/ioffe15.html
Jia, R., Liang, P.: Adversarial examples for evaluating reading comprehension systems. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, 9–11 September 2017, pp. 2021–2031 (2017). https://aclanthology.info/papers/D17-1215/d17-1215
Jin, J., Dundar, A., Culurciello, E.: Robust convolutional neural networks under adversarial noise. arXiv:Learning (2015)
Joshi, A., Tripathi, V., Bhattacharyya, P., Carman, M.J.: Harnessing sequence labeling for sarcasm detection in dialogue from TV series ‘friends’. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, 11–12 August 2016. pp. 146–155 (2016). http://aclweb.org/anthology/K/K16/K16-1015.pdf
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014), http://arxiv.org/abs/1412.6980
Kreuz, R.J., Caucci, G.M.: Lexical influences on the perception of sarcasm. In: The Workshop on Computational Approaches to Figurative Language (2009)
Lin, R., Liu, S., Yang, M., Li, M., Zhou, M., Li, S.: Hierarchical recurrent neural network for document modeling. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, 17–21 September 2015, pp. 899–907 (2015). http://aclweb.org/anthology/D/D15/D15-1106.pdf
Lin, Z., Feng, M., dos Santos, C.N., Yu, M., Xiang, B., Zhou, B., Bengio, Y.: A structured self-attentive sentence embedding. CoRR abs/1703.03130 (2017), http://arxiv.org/abs/1703.03130
Miyato, T., Dai, A.M., Goodfellow, I.: Adversarial training methods for semi-supervised text classification (2016)
Miyato, T., Maeda, S., Koyama, M., Nakae, K., Ishii, S.: Distributional smoothing with virtual adversarial training. In: International Conference on Learning Representations (2016)
Mnih, V., Heess, N., Graves, A., Kavukcuoglu, K.: Recurrent models of visual attention. In: Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014, pp. 2204–2212 (2014), http://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention
Oraby, S., Harrison, V., Reed, L., Hernandez, E., Riloff, E., Walker, M.A.: Creating and characterizing a diverse corpus of sarcasm in dialogue. CoRR abs/1709.05404 (2017), http://arxiv.org/abs/1709.05404
Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retr. 2(12), 1–135 (2008)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, A meeting of SIGDAT, a Special Interest Group of the ACL, Doha, Qatar, 25–29 October 2014, pp. 1532–1543 (2014). http://aclweb.org/anthology/D/D14/D14-1162.pdf
Reyes, A., Rosso, P., Veale, T.: A multidimensional approach for detecting irony in Twitter. Lang. Resour. Eval. 47(1), 239–268 (2013). https://doi.org/10.1007/s10579-012-9196-x
Samanta, S., Mehta, S.: Generating adversarial text samples. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 744–749. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7_71
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014). http://dl.acm.org/citation.cfm?id=2670313
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I.J., Fergus, R.: Intriguing properties of neural networks. CoRR abs/1312.6199 (2013). http://arxiv.org/abs/1312.6199
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. CoRR abs/1706.03762 (2017). http://arxiv.org/abs/1706.03762
Zhang, M., Zhang, Y., Fu, G.: Tweet sarcasm detection using deep neural network. In: COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, Osaka, Japan, 11–16 December 2016, pp. 2449–2460 (2016). http://aclweb.org/anthology/C/C16/C16-1231.pdf
Zhang, X., Zhao, J.J., Lecun, Y.: Character-level convolutional networks for text classification. In: Neural Information Processing Systems, pp. 649–657 (2015)
Acknowledgements
This work was supported by the National Natural Science Foundation of China U1636103, 61632011, Key Technologies Research and Development Program of Shenzhen JSGG20170817140856618, Shenzhen Foundational Research Funding 20170307150024907.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhang, Q., Chen, G., Chen, D. (2018). Adversarial Training for Sarcasm Detection. In: Xiao, J., Mao, ZH., Suzumura, T., Zhang, LJ. (eds) Cognitive Computing – ICCC 2018. ICCC 2018. Lecture Notes in Computer Science(), vol 10971. Springer, Cham. https://doi.org/10.1007/978-3-319-94307-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-94307-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94306-0
Online ISBN: 978-3-319-94307-7
eBook Packages: Computer ScienceComputer Science (R0)