research-article

Neural Co-training for Sentiment Classification with Product Attributes

Authors:

Zhongqing Wang,

Guodong ZhouAuthors Info & Claims

ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), Volume 19, Issue 5

Article No.: 74, Pages 1 - 17

https://doi.org/10.1145/3394113

Published: 04 August 2020 Publication History

Abstract

Sentiment classification aims to detect polarity from a piece of text. The polarity is usually positive or negative, and the text genre is usually product review. The challenges of sentiment classification are that it is hard to capture semantic of reviews, and the labeled data is hard to annotate. Therefore, we propose neural co-training to learn the semantic representation of each review using the neural network model, and learn the information from unlabeled data using a co-training framework. In particular, we use the attention-based bi-directional Gated Recurrent Unit (Att-BiGRU) to model the semantic content of each review and regard different categories of the target product as different views. We then use a co-training framework to learn and predict the unlabeled reviews with different views. Experiment results with the Yelp dataset demonstrate the effectiveness of our approach.

References

[1]

Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, June 18--22, 2018, Salt Lake City, UT. 6077--6086.

[2]

Alina Andreevskaia and Sabine Bergler. 2008. When specialists and generalists work together: Overcoming domain dependence in sentiment tagging. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, June 15--20, 2008, Columbus, Ohio. 290--298.

[3]

John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.

[4]

Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009. Semi-supervised learning. IEEE Trans. Neural Networks 20, 3 (2009), 542--542.

Digital Library

[5]

Huimin Chen, Maosong Sun, Cunchao Tu, Yankai Lin, and Zhiyuan Liu. 2016. Neural sentiment classification with user and product attention. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, November 1--4, 2016, Austin, Texas. 1650--1659.

[6]

Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 25--29, 2014, Doha, Qatar. 1724--1734.

[7]

Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. 2015. Attention-based models for speech recognition. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 577--585.

[8]

Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised sequence learning. In Proceedings of the Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 3079--3087.

[9]

Francois Denis, Anne Laurent, Rémi Gilleron, and Marc Tommasi. 2003. Text classification and co-training from positive and unlabeled examples. In Proceedings of the ICML 2003 Workshop: The Continuum from Labeled to Unlabeled Data. 80--87.

[10]

Ann Devitt and Khurshid Ahmad. 2007. Sentiment polarity identification in financial news: A cohesion-based approach. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.

[11]

Xin Dong and Gerard de Melo. 2018. A helping hand: Transfer learning for deep sentiment analysis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, July 15--20, 2018, Melbourne, Australia. 2524--2534.

[12]

Zi-Yi Dou. 2017. Capturing user and product information for document level sentiment analysis with deep memory network. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 521--526.

[13]

Gayatree Ganu, Noemie Elhadad, and Amélie Marian. 2009. Beyond the stars: Improving rating predictions using review text content. In Proceedings of the 12th International Workshop on the Web and Databases, Providence, Rhode Island, June 28, 2009.

[14]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (1997), 1735--1780.

Digital Library

[15]

Rie Johnson and Tong Zhang. 2015. Semi-supervised convolutional neural networks for text categorization via region embedding. In Proceedings of the Annual Conference on Neural Information Processing Systems, December 7--12, 2015, Montreal, Quebec, Canada. 919--927.

[16]

Rie Johnson and Tong Zhang. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. In Proceedings of the 33nd International Conference on Machine Learning, June 19--24, 2016, New York City, NY. 526--534.

[17]

Hiroshi Kanayama, Tetsuya Nasukawa, and Hideo Watanabe. 2004. Deeper sentiment analysis using machine translation technology. In Proceedings of the 20th International Conference on Computational Linguistics, August 23--27, 2004, Geneva, Switzerland.

[18]

Alistair Kennedy and Diana Inkpen. 2006. Sentiment classification of movie reviews using contextual valence shifters. Comput. Intell. 22, 2 (2006), 110--125.

[19]

Soo-Min Kim and Eduard H. Hovy. 2004. Determining the sentiment of opinions. In Proceedings of the 20th International Conference on Computational Linguistics, August 23--27, 2004, Geneva, Switzerland.

[20]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 25--29, 2014, Doha, Qatar. 1746--1751.

[21]

Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proceedings of the 31th International Conference on Machine Learning, June 21--26, 2014, Beijing, China. 1188--1196.

[22]

Huayu Li, Martin Renqiang Min, Yong Ge, and Asim Kadav. 2017. A context-aware attention network for interactive question answering. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 13--17, 2017, Halifax, NS, Canada. 927--935.

Digital Library

[23]

Linghui Li, Sheng Tang, Lixi Deng, Yongdong Zhang, and Qi Tian. 2017. Image caption with global-local attention. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, February 4--9, 2017, San Francisco, California. 4133--4139.

[24]

Shoushan Li, Chu-Ren Huang, Guodong Zhou, and Sophia Yat Mei Lee. 2010. Employing personal/impersonal views in supervised and semi-supervised sentiment classification. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, July 11--16, 2010, Uppsala, Sweden. 414--423.

Digital Library

[25]

Shoushan Li, Zhongqing Wang, Guodong Zhou, and Sophia Yat Mei Lee. 2011. Semi-supervised learning for imbalanced sentiment classification. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, July 16--22, 2011, Barcelona, Catalonia, Spain. 1826--1831.

[26]

Tao Li, Yi Zhang, and Vikas Sindhwani. 2009. A non-negative matrix tri-factorization approach to sentiment classification with lexical prior knowledge. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1. 244--252.

Digital Library

[27]

Yi Luan, Mari Ostendorf, and Hannaneh Hajishirzi. 2017. Scientific information extraction with semi-supervised neural tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 2641--2651.

[28]

Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, September 17--21, 2015, Lisbon, Portugal. 1412--1421.

[29]

Ryan T. McDonald, Kerry Hannan, Tyler Neylon, Mike Wells, and Jeffrey C. Reynar. 2007. Structured models for fine-to-coarse sentiment analysis. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23--30, 2007, Prague, Czech Republic.

[30]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of the 27th Annual Conference on Neural Information Processing Systems December 5--8, 2013, Lake Tahoe, Nevada. 3111--3119.

[31]

Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725 (2016).

[32]

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, Ken Nakae, and Shin Ishii. 2015. Distributional smoothing with virtual adversarial training. arXiv preprint arXiv:1507.00677 (2015).

[33]

Rodrigo Moraes, João Francisco Valiati, and Wilson P. Gavião Neto. 2013. Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Syst. Appl. 40, 2 (2013), 621--633.

Digital Library

[34]

Tony Mullen and Nigel Collier. 2004. Sentiment analysis using support vector machines with diverse information sources. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, SIGDAT, July 25--26, 2004, Barcelona, Spain. 412--418.

[35]

Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, July 21--26, 2004, Barcelona, Spain. 271--278.

Digital Library

[36]

Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. ACL, 79--86.

Digital Library

[37]

Lizhen Qu, Georgiana Ifrim, and Gerhard Weikum. 2010. The bag-of-opinions method for review rating prediction from sparse text patterns. In Proceedings of the 23rd International Conference on Computational Linguistics, August 23--27, 2010, Beijing, China. 913--921.

Digital Library

[38]

Jonathon Read. 2005. Using emoticons to reduce dependency in machine learning techniques for sentiment classification. In, Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, June 25--30, 2005, University of Michigan. 43--48.

[39]

Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, September 9--11, 2017, Copenhagen, Denmark. 338--348.

[40]

Yafeng Ren, Ruimin Wang, and Donghong Ji. 2016. A topic-enhanced word embedding for Twitter sentiment classification. Inf. Sci. 369 (2016), 188--198.

Digital Library

[41]

Aliaksei Severyn and Alessandro Moschitti. 2015. Twitter sentiment analysis with deep convolutional neural networks. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, August 9--13, 2015, Santiago, Chile. 959--962.

Digital Library

[42]

Vikas Sindhwani and Prem Melville. 2008. Document-word co-regularization for semi-supervised sentiment analysis. In Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15--19, 2008, Pisa, Italy. 1025--1030.

Digital Library

[43]

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, SIGDAT, October 18--21, 2013, Seattle, Washington. 1631--1642.

[44]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Document modeling with gated recurrent neural network for sentiment classification. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, September 17-21, 2015, Lisbon, Portugal. 1422--1432.

[45]

Duyu Tang, Bing Qin, and Ting Liu. 2015. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Volume 1: Long Papers, July 26--31, 2015, Beijing, China. 1014--1023.

[46]

Peter D. Turney. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6--12, 2002, Philadelphia, PA. 417--424.

[47]

Xiaojun Wan. 2009. Co-training for cross-lingual sentiment classification. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th International Joint Conference on Natural Language Processing of the AFNLP, August 2--7, 2009, Singapore. 235--243.

[48]

Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, October 6--8, 2005, Vancouver, British Columbia, Canada. 347--354.

Digital Library

[49]

Weidi Xu, Haoze Sun, Chao Deng, and Ying Tan. 2017. Variational autoencoder for semi-supervised text classification. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, February 4--9, 2017, San Francisco, California. 3358--3364.

[50]

Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, June 12--17, 2016, San Diego, California. 1480--1489.

[51]

Huaguang Zhang, Zhanshan Wang, and Derong Liu. 2014. A comprehensive review of stability analysis of continuous-time recurrent neural networks. IEEE Trans. Neural Netw. Learning Syst. 25, 7 (2014), 1229--1262.

[52]

Xiaojin Zhu. 2006. Semi-supervised learning literature survey. Computer Science, University of Wisconsin-Madison 2, 3 (2006), 4.

Cited By

Van Dinh CLuu S(2024)Metadata Integration for Spam Reviews Detection on Vietnamese E-commerce WebsitesInternational Journal of Asian Language Processing10.1142/S2717554524500024Online publication date: 29-Jul-2024
https://doi.org/10.1142/S2717554524500024
Liu LHuang PYu HMin F(2023)Safe co-training for semi-supervised regressionIntelligent Data Analysis10.3233/IDA-22671827:4(959-975)Online publication date: 20-Jul-2023
https://doi.org/10.3233/IDA-226718
Liu XRen QYin YLi LZhang Q(2023)Mongolian Text Sentiment Analysis Based on Multi-scale CNN and mLSTM Mode2023 International Conference on Advances in Electrical Engineering and Computer Applications (AEECA)10.1109/AEECA59734.2023.00117(625-631)Online publication date: 18-Aug-2023
https://doi.org/10.1109/AEECA59734.2023.00117

Index Terms

Neural Co-training for Sentiment Classification with Product Attributes
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Document analysis

Recommendations

DCPE co-training for classification

Co-training is a well-known semi-supervised learning technique that applies two basic learners to train the data source, which uses the most confident unlabeled data to augment labeled data in the learning process. In the paper, we use the diversity of ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

In multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Tri-Training: Exploiting Unlabeled Data Using Three Classifiers

In many practical data mining applications, such as Web page classification, unlabeled training examples are readily available, but labeled ones are fairly expensive to obtain. Therefore, semi-supervised learning algorithms such as co-training have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Asian and Low-Resource Language Information Processing

ACM Transactions on Asian and Low-Resource Language Information Processing Volume 19, Issue 5

September 2020

278 pages

ISSN:2375-4699

EISSN:2375-4702

DOI:10.1145/3403646

Editor:
Imed Zitouni
Microsoft, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2020

Online AM: 07 May 2020

Accepted: 01 April 2020

Revised: 01 February 2020

Received: 01 May 2019

Published in TALLIP Volume 19, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Natural Science Foundation of China
Jiangsu High School Research

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
150
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Van Dinh CLuu S(2024)Metadata Integration for Spam Reviews Detection on Vietnamese E-commerce WebsitesInternational Journal of Asian Language Processing10.1142/S2717554524500024Online publication date: 29-Jul-2024
https://doi.org/10.1142/S2717554524500024
Liu LHuang PYu HMin F(2023)Safe co-training for semi-supervised regressionIntelligent Data Analysis10.3233/IDA-22671827:4(959-975)Online publication date: 20-Jul-2023
https://doi.org/10.3233/IDA-226718
Liu XRen QYin YLi LZhang Q(2023)Mongolian Text Sentiment Analysis Based on Multi-scale CNN and mLSTM Mode2023 International Conference on Advances in Electrical Engineering and Computer Applications (AEECA)10.1109/AEECA59734.2023.00117(625-631)Online publication date: 18-Aug-2023
https://doi.org/10.1109/AEECA59734.2023.00117
Liu LHuang PMin F(2022)Safe Multi-view Co-training for Semi-supervised Regression2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA54385.2022.10032437(1-10)Online publication date: 13-Oct-2022
https://doi.org/10.1109/DSAA54385.2022.10032437

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents