skip to main content
10.1145/3655532.3655553acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicrsaConference Proceedingsconference-collections
research-article

Chinese News Classification Based on ERNIE and Attention Fusion Features

Published: 28 June 2024 Publication History

Abstract

In order to improve the classification performance of Chinese news and correlate the features extracted by multiple network models at the same time, we propose a deep network model based on the pre-trained model ERNIE and attention fusion features. Use ERNIE as the word embedding layer to obtain dynamic word vectors, use the self-attention mechanism SAT and DPCNN network to obtain the long-distance dependence of the text, and use the bidirectional gating unit BIGRU and the soft attention mechanism AT to obtain contextual timing features. Combine the label CLS processed by ERNIE, the hidden layer state of BIGRU at the last moment, and the above two features into a feature sequence, and use the attention mechanism to assign the weight ratio of each feature. Finally, the weighted and summed feature vectors are classified and output using the fully connected layer. Trained on the public Chinese dataset THUCNews, the experimental results show that this model can effectively improve the accuracy of text classification compared with other comparison models.

References

[1]
Yoon Kim. 2014. Convolutional neural networks for sentence classification. CoRR abs/1408.5882. arXiv preprint arXiv:1408.5882.
[2]
Chunting Zhou, Chonglin Sun, Zhiyuan Liu, and Francis C.M. Lau. 2015. A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630.
[3]
Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
[4]
Wiwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. "Recurrent convolutional neural networks for text classification." Proceedings of the AAAI conference on artificial intelligence. Vol. 29. No. 1. https://doi.org/10aaai.v29i1.9513.
[5]
Rie Johnson, and Tong Zhang. 2017. "Deep pyramid convolutional neural networks for text categorization." Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
[6]
Yao, Liang, Chengsheng Mao, and Yuan Luo. 2019. "Graph convolutional networks for text classification." Proceedings of the AAAI conference on artificial intelligence. Vol. 33. No. 01.
[7]
Ke Zhao, Lan Huang, Rui Song, Qiang Shen, and Hao Xu. 2021. A sequential graph neural network for short text classification. Algorithms,14(12):352. https://doi.org/10.3390/a14120352.
[8]
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv 2018. arXiv preprint arXiv:1802.05365, 12.
[9]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
[10]
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, OpenAI.
[11]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[12]
Yu Sun, Shouhuang Wang, Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Dangxiang Zhu, Hao Tian, and Hua Wu. 2019. Ernie: Enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223.
[13]
Meng Zhang, and Xiang Shang. 2022. Chinese Short Text Classification by ERNIE Based on LTC_Block. Wireless Communications and Mobile Computing, 2022.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICRSA '23: Proceedings of the 2023 6th International Conference on Robot Systems and Applications
September 2023
335 pages
ISBN:9798400708039
DOI:10.1145/3655532
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 June 2024

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. BIGRU
  2. DPCNN
  3. ERNIE
  4. attention mechanism
  5. text classification

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

ICRSA 2023

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 15
    Total Downloads
  • Downloads (Last 12 months)15
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media