Prompt and Contrastive Learning for Few-shot Sentiment Classification
Authors: 
Fei Wang, Long Chen, Xiaohua Huang, Cai Xu, Wei Zhao, Ziyu Guan, Guangyue LuAuthors Info & Claims
Fei Wang, Long Chen, Xiaohua Huang, Cai Xu, Wei Zhao, Ziyu Guan, Guangyue LuAuthors Info & ClaimsPages 173 - 179
Abstract
Sentiment classification is a hot topic in the field of natural language processing. Currently, state-of-the-art classification models follow two steps: pre-training a large language model on upstream tasks, and then using human-labeled data to fine-tune a task-related model. However, there is a large gap between the upstream tasks of the pre-trained model and the downstream tasks being performed, resulting in the need for more labeled data to achieve excellent performance. Manually annotating data is expensive. In this paper, we propose a few-shot sentiment classification method based on Prompt and Contrastive Learning (PCL), which can significantly improve the performance of large-scale pre-trained language models in low-data and high-data regimes. Prompt learning aims to alleviate the gap between upstream and downstream tasks, and the contrastive learning is designed to capture the inter-class and intra-class distribution patterns of labeled data. Thanks to the integration of the two strategies, PCL markedly exceeds baselines with low resources. Extensive experiments on three datasets show that our method has outstanding performance in the few-shot settings.
References
[1]
Turney P D. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews[J]. arXiv preprint cs/0212032, 2002.
[2]
Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment classification using machine learning techniques[J]. arXiv preprint cs/0205070, 2002.
[3]
Liu H, Dai Z, So D R, Pay Attention to MLPs[J]. arXiv preprint arXiv:2105.08050, 2021.
[4]
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. Mlp-mixer: An all-mlp architecture for vision. arXiv preprint arXiv:2105.01601, 2021.
[5]
Zhao W, Guan Z, Chen L, Weakly-supervised deep embedding for product review sentiment analysis[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 30(1): 185-197.
[6]
Chen Long, Guan Ziyu, He Jinhong, Peng Jinye. A Survey on Sentiment Classification[J]. Journal of Computer Research and Development, 2017, 54(6): 1150-1170.
[7]
Zhang S, Xu X, Pang Y, Multi-layer attention based CNN for target-dependent sentiment classification[J]. Neural processing letters, 2020, 51(3): 2089-2103.
[8]
Al-Smadi M, Talafha B, Al-Ayyoub M, Using long short-term memory deep neural networks for aspect-based sentiment analysis of Arabic reviews[J]. International Journal of Machine Learning and Cybernetics, 2019, 10(8): 2163-2175.
[9]
Ling M, Chen Q, Sun Q, Hybrid neural network for Sina Weibo sentiment analysis[J]. IEEE Transactions on Computational Social Systems, 2020, 7(4): 983-990.
[10]
Devlin J, Chang M W, Lee K, Bert: Pre-training of deep bidirectional transformers for language understanding[J]. arXiv preprint arXiv:1810.04805, 2018.
[11]
Schick T, Schütze H. Exploiting cloze questions for few shot text classification and natural language inference[J]. arXiv preprint arXiv:2001.07676, 2020.
[12]
Chen T, Kornblith S, Norouzi M, A simple framework for contrastive learning of visual representations[C]//International conference on machine learning. PMLR, 2020: 1597-1607.
[13]
Gunel B, Du J, Conneau A, Supervised contrastive learning for pre-trained language model fine-tuning[J]. arXiv preprint arXiv:2011.01403, 2020.
[14]
Guan Z, Chen L, Zhao W, Weakly-Supervised Deep Learning for Customer Review Sentiment Classification[C]//IJCAI. 2016: 3719-3725.
[15]
Socher R, Perelygin A, Wu J, Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 conference on empirical methods in natural language processing. 2013: 1631-1642.
[16]
Go A, Bhayani R, Huang L. Twitter sentiment classification using distant supervision[J]. CS224N project report, Stanford, 2009, 1(12): 2009.
[17]
Wang S, Manning C D. Baselines and bigrams: Simple, Good Sentiment and Topic Classification [C] //Proc of the 50th Annual Meeting of the Association for Computational Linguistics. Cambridge, MA: MIT Press, 2012: 90-94.
[18]
Wang B, Shan D, Fan A, A Sentiment Classification Method of Web Social Media Based on Multi-dimension and Multi-level Modeling[J]. IEEE Transactions on Industrial Informatics, 2021.
[19]
Van der Maaten L, Hinton G. Visualizing data using t-SNE[J]. Journal of machine learning research, 2008, 9(11).
[20]
Teng Jinbao, Kong Weiwei, Chang Yidan, Tian Qiaoxin, Shi Chenyuan, and Li Long. 2021. Text Classification Method Based on BiGRU-Attention and CNN Hybrid Model. In 2021 4th International Conference on Artificial Intelligence and Pattern Recognition. 614–622.
Index Terms
- Prompt and Contrastive Learning for Few-shot Sentiment Classification
Recommendations
Sentiment labeling for extending initial labeled data to improve semi-supervised sentiment classification
Semi-supervised framework which exploits unsupervised approach (JST) is proposed.Self-training suffers from incorrectly labeling problem with insufficient data.Confidently predicted instances are labeled and used as training data by JST.Self-training ...
Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification
WWW '22: Proceedings of the ACM Web Conference 2022Large-scale multi-label text classification (LMTC) aims to associate a document with its relevant labels from a large candidate set. Most existing LMTC approaches rely on massive human-annotated training data, which are often costly to obtain and suffer ...
Prompt-Based Joint Contrastive Learning for Zero-Shot Relation Extraction
Natural Language Processing and Chinese ComputingAbstractZero-shot relation extraction is proposed to address the issue that the model performance drops dramatically when identifying novel relations that cannot be observed at the training stage. Existing works focus more on the pattern of matching ...
Comments
Information & Contributors
Information
Published In
September 2022
1221 pages
ISBN:9781450396899
DOI:10.1145/3573942
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AIPR 2022
AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
September 23 - 25, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 85Total Downloads
- Downloads (Last 12 months)24
- Downloads (Last 6 weeks)4
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format