Abstract
Recurrent neural networks (RNN) has been widely used in sentiment classification. RNN can memorize the previous information and is applied to calculate the current output. For sentiment binary classification, RNN calculates the probabilities and then performs binary classification according to the probability values, and some emotions near the median are forcibly divided. But, it does not consider the existence of some samples that are not very clearly polarized in sentiment binary classification. Three-way decisions theory divides the dataset into three regions, positive region, negative region, boundary region. In the process of training classification, the probabilities of some samples belonging to different categories are very close, and three-way decisions can divide them into the boundary region by setting thresholds. Reasonable processing of the boundary region can get better results for binary classification by adjusting the probability of samples in the boundary region. Therefore, in this paper, we propose three-way decisions based RNN models for sentiment classification. Firstly, we use basic RNN models to classify the data. Secondly, we apply three-way decisions theory to set the thresholds, divide the boundary region based on probability. Finally, the probabilities of samples in the boundary region are adjusted and applied in the next round of training. Experiments on four real datasets show that our proposed models are better than corresponding basic RNN models in terms of classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhang, Y., Xiang, X., Yin, C., Shang, L.: Parallel sentiment polarity classification method with substring feature reduction. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 121–132 (2013)
Ju, S., Li, S.: Active learning on sentiment classification by selecting both words and documents. In: Ji, D., Xiao, G. (eds.) CLSW 2012. LNCS (LNAI), vol. 7717, pp. 49–57. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36337-5_6
Hailong, Z., Wenyan, G., Bo, J.: Machine learning and lexicon based methods for sentiment classification: a survey. In: 2014 11th Web Information System and Application Conference, pp. 262–265 (2015)
Tang, D., Qin, B., Liu, T.: Deep learning for sentiment analysis: successful approaches and future challenges. Wiley Interdisc. Rev.: Data Min. Knowl. Discov. 5(6), 292–303 (2015)
Jia, X., Deng, Z., Min, F., Liu, D.: Three-way decisions based feature fusion for Chinese irony detection. Int. J. Approx. Reason. 113, 324–335 (2019)
Raffel, C., Ellis, D.P.: Feed-forward networks with attention can solve some long-term memory problems. arXiv preprint arXiv:1512.08756 (2015)
Schmidt, R.M.: Recurrent neural networks (RNNs): a gentle introduction and overview. arXiv preprint arXiv:1912.05911 (2019)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Dong, Y., Fu, Y., Wang, L., Chen, Y., Dong, Y., Li, J.: A sentiment analysis method of capsule network based on biLSTM. IEEE Access 8, 37014–37020 (2020)
Tao, H., Tong, S., Zhao, H., Xu, T., Liu, Q.: A radical-aware attention-based model for Chinese text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5125–5132 (2019)
Myagmar, B., Li, J., Kimura, S.: Cross-domain sentiment classification with bidirectional contextualized transformer language models. IEEE Access 7, 163219–163230 (2019)
Sharfuddin, A.A., Tihami, M.N., Islam, M.S.: A deep recurrent neural network with biLSTM model for sentiment classification. In: 2018 International Conference on Bangla Speech and Language Processing (ICBSLP), pp. 1–4 (2018)
Pal, S., Ghosh, S., Nag, A.: Sentiment analysis in the light of LSTM recurrent neural networks. Int. J. Synthetic Emot. 9(1), 33–39 (2018)
Yu, H., Ji, Y., Li, Q.: Student sentiment classification model based on GRU neural network and TF-IDF algorithm. J. Intell. Fuzzy Syst. 40(2), 2301–2311 (2021)
Tran, N.M.: Aspect based sentiment analysis using neuroner and bidirectional recurrent neural network. In: Proceedings of the Ninth International Symposium on Information and Communication Technology, pp. 1–7 (2018)
Liu, G., Guo, J.: Bidirectional LSTM with attention mechanism and convolutional layer for text classification. Neurocomputing 337(APR.14), 325–338 (2019)
Han, Y., Liu, M., Jing, W.: Aspect-level drug reviews sentiment analysis based on double BiGRU and knowledge transfer. IEEE Access 8, 21314–21325 (2020)
Pan, Y., Liang, M.: Chinese text sentiment analysis based on bi-GRU and self-attention. In: 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 1983–1988 (2020)
Arevian, G., Panchev, C.: Optimising the hystereses of a two context layer RNN for text classification. In: International Joint Conference on Neural Networks, pp. 2936–2941. IEEE (2007)
Chen, J., Li, Y., Zhao, S., Wang, X., Zhang, Y.: Three-way decisions community detection model based on weighted graph representation. In: Bello, R., Miao, D., Falcon, R., Nakata, M., Rosete, A., Ciucci, D. (eds.) IJCRS 2020. LNCS (LNAI), vol. 12179, pp. 153–165. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52705-1_11
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Ranzato, M., Chopra, S., Auli, M., Zaremba, W.: Sequence level training with recurrent neural networks. In: ICLR (2016)
Xu, X., Ye, F.: Sentences similarity analysis based on word embedding and syntax analysis. In: 2017 IEEE 17th International Conference on Communication Technology (ICCT), pp. 1896–1900. IEEE (2017)
Qian, F., Sha, L., Chang, B., Liu, L.C., Zhang, M.: Syntax aware LSTM model for semantic role labeling. In: Proceedings of the 2nd Workshop on Structured Prediction for Natural Language Processing, pp. 27–32 (2017)
Zhu, R., Yang, D., Li, Y.: Learning improved semantic representations with tree-structured LSTM for hashtag recommendation: an experimental study. Information 10, 127 (2019)
Skantze, G.: Towards a general, continuous model of turn-taking in spoken dialogue using LSTM recurrent neural networks. In: Sigdial Meeting on Discourse Dialogue, pp. 220–230 (2017)
Su, C., Huang, H., Shi, S., Jian, P., Shi, X.: Neural machine translation with Gumbel tree-LSTM based encoder. J. Vis. Commun. Image Represent. 71, 102811 (2020)
Cheng, J., Dong, L., Lapata, M.: Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733 (2016)
Yangsen, Z., Jia, Z., Yuru, J., Gaijuan, H., Ruoyu, C.: A text sentiment classification modeling method based on coordinated CNN-LSTM-attention model. Chin. J. Electron. 28(001), 120–126 (2019)
Feng, S., Wang, Y., Liu, L., Wang, D., Yu, G.: Attention based hierarchical LSTM network for context-aware microblog sentiment classification. World Wide Web 22(1), 59–81 (2019)
Feng, X., Liu, X.: Sentiment classification of reviews based on BiGRU neural network and fine-grained attention. In: Journal of Physics: Conference Series. vol. 1229, p. 012064. IOP Publishing (2019)
Yao, Y.: Three-way decisions with probabilistic rough sets. Inform. Sci. 180(3), 341–353 (2010)
Zhang, Z., Wang, R.: Applying three-way decisions to sentiment classification with sentiment uncertainty. In: Miao, D., Pedrycz, W., Ślȩzak, D., Peters, G., Hu, Q., Wang, R. (eds.) RSKT 2014. LNCS (LNAI), vol. 8818, pp. 720–731. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11740-9_66
Zhou, Z., Zhao, W., Shang, L.: Sentiment analysis with automatically constructed lexicon and three-way decision. In: Miao, D., Pedrycz, W., Ślȩzak, D., Peters, G., Hu, Q., Wang, R. (eds.) RSKT 2014. LNCS (LNAI), vol. 8818, pp. 777–788. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11740-9_71
Zhang, Y., Zhang, Z., Miao, D., Wang, J.: Three-way enhanced convolutional neural networks for sentence-level sentiment classification. Inform. Sci. 477, 55–64 (2018)
Zhang, Y., Miao, D., Wang, J., Zhang, Z.: A cost-sensitive three-way combination technique for ensemble learning in sentiment classification. Int. J. Approx. Reason. 105, 85–97 (2018)
Zhu, Y., Tian, H., Ma, J., Liu, J., Liang, T.: An integrated method for micro-blog subjective sentence identification based on three-way decisions and naive bayes. In: Miao, D., Pedrycz, W., Ślȩzak, D., Peters, G., Hu, Q., Wang, R. (eds.) RSKT 2014. LNCS (LNAI), vol. 8818, pp. 844–855. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11740-9_77
Acknowledgments
This work was supported by the Universities Natural Science Key Project of Anhui Province (No. KJ2020A0637) and the Universities Natural Science Foundation of Anhui Province (No. KJ2011Z400). The authors of the paper express great acknowledgment of these supports.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Ma, Y., Yu, J., Ji, B., Chen, J., Zhao, S., Chen, J. (2021). Three-Way Decisions Based RNN Models for Sentiment Classification. In: Ramanna, S., Cornelis, C., Ciucci, D. (eds) Rough Sets. IJCRS 2021. Lecture Notes in Computer Science(), vol 12872. Springer, Cham. https://doi.org/10.1007/978-3-030-87334-9_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-87334-9_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87333-2
Online ISBN: 978-3-030-87334-9
eBook Packages: Computer ScienceComputer Science (R0)