A Stance Detection Approach Based on Generalized Autoregressive pretrained Language Model in Chinese Microblogs
Pages 232 - 238
Abstract
Timely identification of Chinese Microblogs users' stance and tendency is of great significance for social managers to understand the trends of online public opinion. Traditional stance detection methods underutilize target information, which affects the detection effect. This paper proposes to integrate the target subject information into a Chinese Microblogs stance detection method based on a generalized autoregressive pretraining language model, and use the advantages of the generalized autoregressive model to extract deep semantics to weaken the high randomness of Microblogs self-media text language and lack of grammar. The impact of norms on text modeling. First carry out microblog data preprocessing to reduce the influence of noise data on the detection effect; then connect the target subject information and the text sequence to be tested into the XLNet network for fine-tuning training; Finally, the fine-tuned XLNet network is combined with the Softmax regression model for stance classification. The experimental results show that the value of the proposed method in the NLPCC2016 Chinese Microblogs detection and evaluation task reaches 0.75, which is better than the existing public model, and the effect is improved significantly.
References
[1]
Biber D, Finegan E. Adverbial Stance Types in English.[J]. Discourse Processes, 1988, 11(1): 1-34
[2]
YANG Kaimo, WU Mingfen, CHEN Tao. Generalized text sentiment analysis review [J]. Journal of Computer Applications, 2019, 039(0z2):6-14.
[3]
Yao Yan. Social media text by incorporating topic target information[D]
[4]
Xu R, Zhou Y, Wu D, et al .Overview of NLPCC shared task 4:Stance detection in Chinese Microblogs[M]//Natural language understanding and intelligent applications. [S.l.]:Springer International Publishing, 2016
[5]
Yang Z, Dai Z, Yang Y, XLNet: Generalized Autoregressive Pretraining for Language Understanding[J]. 2019.
[6]
Sun Q, Wang Z, Zhu Q, Exploring Various Linguistic Features for Stance Detection[C]// International Conference on Computer Processing of Oriental Languages. Springer International Publishing, 2016.
[7]
ZHENG Hai-Yang, GAO Jun-Bo,QIU Jie, Stance detection in Chinese microblog topic based on word embedding technology and thematic words feature[J].Computer System &Application.,2018,27(09):118-123.
[8]
Mikolov T,Chen K,Corrado G, Efficient Estimation of Word Representations in Vector Space[J]. Computer ence, 2013.
[9]
Lecun Y, Kavukcuoglu K, Cle´ment Farabet. Convolutional Networks and Applications in Vision[C]// Proceedings of 2010 IEEE International Symposium on Circuits and Systems. IEEE, 2010.
[10]
Toma´sˇMikolov, Kombrink S, Luka´sˇBurget, EXTENSIONS OF RECURRENT NEURAL NETWORK LANGUAGE MODEL[C]// IEEE International Conference on Acoustics, Speech & Signal Processing. IEEE, 2011.
[11]
Sundermeyer M,Ralf Schlüter, Ney H . LSTM Neural Networks for Language Modeling[C]// Interspeech. 2012.
[12]
Bai Jing, Li Fei, Ji Donghong. Attention based bilstm-cnn Chinese Microblogging position detection model[J].Computer Application and Software, 2018(3):266-274.
[13]
YUE Tianchi,ZHANG Shaowu, YANG Liang, Stance Detection Method Based on Two-Stage Attention Mechanism[J].Journal of Guangxi Normal University(Natural Science Edition),2019,37(01):42-49.
[14]
Du J, Xu R, He Y, Stance classification with target-specific neural attention networks[C]. international joint conference on artificial intelligence, 2017: 3988-3994
[15]
ZHOU Yan-fang, ZHOU Gang, Approach of Stance Detection in Micro-blog Based on Transfer Learning and Multi-representation[J]. COMPUTER SCIENCE, 2018, 45(009):243-247.
[16]
WANG An-Jun, HUANG Kai-Kai, Stance Detection in Chinese Microblogs via Bert-Condition-CNN Model[J]. Computer System &Application, 2019.
[17]
Devlin J, Chang M W, Lee K, BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[J]. 2018.
[18]
LI ZHOU-Jun, FAN Yun, Survey of Natural Language Processing Pre-training Techniques[J]. COMPUTER SCIENCE, 2020, v.47(03):170-181.
[19]
Vaswani A, Shazeer N, Parmar N, Attention is all you need[C]//Advances in neural information processing systems. 2017: 5998-6008.
[20]
Dai Z, Yang Z, Yang Y, Transformer-xl: Attentive language models beyond a fixed-length context[J]. arXiv preprint arXiv:1901.02860, 2019.
[21]
Cui Y, Che W, Liu T, Revisiting Pre-Trained Models for Chinese Natural Language Processing[J]. 2020.
Recommendations
Pretrained Embeddings for Stance Detection with Hierarchical Capsule Network on Social Media
Stance detection on social media aims to identify the stance of social media users toward a topic or claim, which can provide powerful information for various downstream tasks. Many existing stance detection approaches neglect to model the deep semantic ...
Early Detection of Rumours on Twitter via Stance Transfer Learning
Advances in Information RetrievalAbstractRumour detection on Twitter is an important problem. Existing studies mainly focus on high detection accuracy, which often requires large volumes of data on contents, source credibility or propagation. In this paper we focus on early detection of ...
A language model based on semantically clustered words in a Chinese character recognition system
ICDAR '95: Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1) - Volume 1This paper presents a new method for clustering the words in a dictionary into word groups, which are applied in a Chinese character recognition system with a language model to describe the contextual information. The Chinese synonym dictionary ...
Comments
Information & Contributors
Information
Published In
February 2021
601 pages
ISBN:9781450389310
DOI:10.1145/3457682
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- National Social Science Foundation of China
Conference
ICMLC 2021
ICMLC 2021: 2021 13th International Conference on Machine Learning and Computing
February 26 - March 1, 2021
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 97Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)1
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format