Abstract
In view of the problems existing in the ESG classification task of Chinese financial texts, such as feature loss caused by excessively long texts, this paper proposes an interactive multi-task model AmultiESG for ESG classification of Chinese financial texts. The model divides Chinese financial text ESG classification and financial sentiment dictionary expansion into primary and secondary tasks. First, BiLSTM model is used to learn the original representation of the text. Then, in the secondary task, the attention mechanism and full connection layers are combined with the domain dictionary to realize the extraction of emotional words. In the main task, in order to prevent feature loss due to the excessively long texts, we process the text again and divide it into blocks according to the period. Meanwhile, we learned new feature representation of the text by combining text label representation, text block representation, BiLSTM output features and domain dictionary features. And we introduce an interactive information transfer mechanism to iteratively improve the predicted results of the two tasks and strengthen the association between them. It has been experimentally demonstrated that the proposed method shows superior performance compared to other baselines for the ESG classification task of Chinese financial text, especially for long-text classification tasks.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability Statements
Due to commercial reasons, the data set analyzed in the current study is only partially disclosed, but can be obtained from the corresponding author under reasonable requirements.
References
Zhao C, Guo Y, Yuan J, Wu M, Li D, Zhou Y, Kang J (2018) Esg and corporate financial performance: empirical evidence from china’s listed power generation companies. Sustainability 10(8):2607
Mehra S, Louka R, Zhang Y (2022) Esgbert: language model to help with classification tasks related to companies environmental, social, and governance practices. arXiv preprint arXiv:2203.16788
Manoharan JS (2021) Capsule network algorithm for performance optimization of text classification. J Soft Comput Paradigm (JSCP) 3(01):1–9
Ragesh R, Sellamanickam S, Iyer A, Bairi R, Lingam V (2021) Hetegcn: heterogeneous graph convolutional networks for text classification. In: Proceedings of the 14th ACM international conference on web search and data mining, pp 860–868
Liang Y, Li H, Guo B, Yu Z, Zheng X, Samtani S, Zeng DD (2021) Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification. Inf Sci 548:295–312
Deng J, Cheng L, Wang Z (2021) Attention-based bilstm fused cnn with gating mechanism model for chinese long text classification. Comput Speech Lang 68:101182
Jiang T, Wang D, Sun L, Yang H, Zhao Z, Zhuang F (2021) Lightxml: transformer with dynamic negative sampling for high-performance extreme multi-label text classification
Moon SJ, Mo S, Lee K, Lee J, Shin J (2021) Masker: masked keyword regularization for reliable text classification. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 13578–13586
Piao G (2021) Scholarly text classification with sentence bert and entity embeddings. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 79–87
González-Carvajal S, Garrido-Merchán EC (2020) Comparing bert against traditional machine learning text classification. arXiv preprint arXiv:2005.13012
Bai J, Bai S, Chu Y, Cui Z, Dang K, Deng X, Fan Y, Ge W, Han Y, Huang F et al (2023) Qwen technical report. arXiv preprint arXiv:2309.16609
Boitel E, Mohasseb A, Haig E (2023) A comparative analysis of gpt-3 and bert models for text-based emotion recognition: performance, efficiency, and robustness. In: UK workshop on computational intelligence. Springer, pp 567–579
Yang B, Luo X, Sun K, Luo MY (2023) Recent progress on text summarisation based on bert and gpt. In: International conference on knowledge science, engineering and management. Springer, pp 225–241
Nazir A, Wang Z (2023) A comprehensive survey of chatgpt: advancements, applications, prospects, and challenges. Meta-radiology 100022
Ding M, Zhou C, Yang H, Tang J (2020) Cogltx: applying bert to long texts. Adv Neural Inf Process Syst 33:12792–12804
Li X, Chan S, Zhu X, Pei Y, Ma Z, Liu X, Shah S (2023) Are chatgpt and gpt-4 general-purpose solvers for financial text analytics? a study on several typical tasks. In: Proceedings of the 2023 conference on empirical methods in natural language processing: industry track, pp 408–422
Qiu Y, Jin Y (2024) Chatgpt and finetuned bert: a comparative study for developing intelligent design support systems. Intell Syst Appl 21:200308
Yang L, Kenny EM, Ng TLJ, Yang Y, Smyth B, Dong R (2020) Generating plausible counterfactual explanations for deep transformers in financial text classification. arXiv preprint arXiv:2010.12512
Liang X, Cheng D, Yang F, Luo Y, Qian W, Zhou A (2020) F-hmtc: detecting financial events for investment decisions based on neural hierarchical multi-label text classification. In: IJCAI, pp 4490–4496
Arslan Y, Allix K, Veiber L, Lothritz C, Bissyandé TF, Klein J, Goujon A (2021) A comparison of pre-trained language models for multi-class text classification in the financial domain. In: Companion proceedings of the web conference 2021, pp 260–268
Rizun N, Waloszek W (2018) Methodology for text classification using manually created corpora-based sentiment dictionary. In: Proceedings of the 10th international joint conference on knowledge discovery, knowledge engineering and knowledge management (IC3K 2018)–Volume, vol 1, pp 212–220
Abel J, Lantow B (2019) A methodological framework for dictionary and rule-based text classification. In: KDIR, pp 330–337
Tang H-j, Yan D-f, Yuan T (2013) Semantic dictionary based method for short text classification. J China Univ Posts Telecommunications 20:15–19
Lu K, Wu J (2019) Sentiment analysis of film review texts based on sentiment dictionary and svm. In: Proceedings of the 2019 3rd international conference on innovation in artificial intelligence, pp 73–77
Zhang S, Wei Z, Wang Y, Liao T (2018) Sentiment analysis of chinese micro-blog text based on extended sentiment dictionary. Futur Gener Comput Syst 81:395–403
Xu G, Yu Z, Yao H, Li F, Meng Y, Wu X (2019) Chinese text sentiment analysis based on extended sentiment dictionary. IEEE Access 7:43749–43762
Novikova A, Stupnikov S (2017) Sentiment analysis of short texts from social networks using sentiment lexicons and blending of machine learning algorithms. In: Proc CEUR Workshop, pp 190–201
Cho H, Kim S, Lee J, Lee J-S (2014) Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews. Knowl-Based Syst 71:61–71
He R, Lee WS, Ng HT, Dahlmeier D (2019) An interactive multi-task learning network for end-to-end aspect-based sentiment analysis. arXiv preprint arXiv:1906.06906
Deng D (2019) Research on algorithms and applications for sentient lexicon construction. Beingjing Jiaotong University
Xiong Y, Feng Y, Wu H, Kamigaito H, Okumura M (2021) Fusing label embedding into bert: an efficient improvement for text classification. In: Findings of the association for computational linguistics: ACL-IJCNLP 2021, pp 1743–1750
Pappagari R, Zelasko P, Villalba J, Carmiel Y, Dehak N (2019) Hierarchical transformers for long document classification. In: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, pp 838–844
Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao J (2021) Deep learning-based text classification: a comprehensive review. ACM Comput Surv (CSUR) 54(3):1–40
Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K et al (2018) Deep contextualized word representations. arXiv:1802.05365. https://doi.org/10.18653/v1. N18-1202
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805
Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692
Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942
Sanh V, Debut L, Chaumond J, Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108
Sun Y, Wang S, Li Y, Feng S, Chen X, Zhang H, Tian X, Zhu D, Tian H, Wu H (2019) Ernie: enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223
Araci DF, Genc Z (2019) Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063
Zhong Q, Ding L, Liu J, Du B, Tao D (2023) Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198
Zhang H, Sun S, Hu Y, Liu J, Guo Y (2020) Sentiment classification for chinese text based on interactive multitask learning. IEEE Access 8:129626–129635
Yang L, Kenny EM, Ng TLJ, Yang Y, Smyth B, Dong R (2020) Generating plausible counterfactual explanations for deep transformers in financial text classification. arXiv preprint arXiv:2010.12512
Clark K, Luong MT, Le QV, Manning CD (2020) Electra: pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555
Diao S, Bai J, Song Y, Zhang T, Wang Y (2019) Zen: pre-training chinese text encoder enhanced by n-gram representations. arXiv preprint arXiv:1911.00720
Arslan Y, Allix K, Veiber L, Lothritz C, Bissyandé TF, Klein J, Goujon A (2021) A comparison of pre-trained language models for multi-class text classification in the financial domain. In: Companion proceedings of the web conference 2021, pp 260–268
Dai ZYY, Yang Z (2019) Transformer-xl: attentive language models beyond a fixed-length context
Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9
Baidu: ERNIE Bot (2021). https://cloud.baidu.com/product/wenxinworkshop/
Funding
The authors did not receive support from any organization for the submitted work.
Author information
Authors and Affiliations
Contributions
Han Zhang mainly contributed to the study conception and wrote the first draft of the manuscript. Material preparation, data collection and analysis were performed by Yazhou Zhang and Xinyu Wang. The data set is annotated under the guidance of Lei Wang. Lixia Ji contributed to schedule control for the research. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors have no relevant financial or non-financial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, H., Zhang, Y., Wang, X. et al. An interactive multi-task ESG classification method for Chinese financial texts. Appl Intell 55, 191 (2025). https://doi.org/10.1007/s10489-024-06068-8
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06068-8