Skip to main content

Advertisement

An interactive multi-task ESG classification method for Chinese financial texts

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

In view of the problems existing in the ESG classification task of Chinese financial texts, such as feature loss caused by excessively long texts, this paper proposes an interactive multi-task model AmultiESG for ESG classification of Chinese financial texts. The model divides Chinese financial text ESG classification and financial sentiment dictionary expansion into primary and secondary tasks. First, BiLSTM model is used to learn the original representation of the text. Then, in the secondary task, the attention mechanism and full connection layers are combined with the domain dictionary to realize the extraction of emotional words. In the main task, in order to prevent feature loss due to the excessively long texts, we process the text again and divide it into blocks according to the period. Meanwhile, we learned new feature representation of the text by combining text label representation, text block representation, BiLSTM output features and domain dictionary features. And we introduce an interactive information transfer mechanism to iteratively improve the predicted results of the two tasks and strengthen the association between them. It has been experimentally demonstrated that the proposed method shows superior performance compared to other baselines for the ESG classification task of Chinese financial text, especially for long-text classification tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability Statements

Due to commercial reasons, the data set analyzed in the current study is only partially disclosed, but can be obtained from the corresponding author under reasonable requirements.

References

  1. Zhao C, Guo Y, Yuan J, Wu M, Li D, Zhou Y, Kang J (2018) Esg and corporate financial performance: empirical evidence from china’s listed power generation companies. Sustainability 10(8):2607

    Article  Google Scholar 

  2. Mehra S, Louka R, Zhang Y (2022) Esgbert: language model to help with classification tasks related to companies environmental, social, and governance practices. arXiv preprint arXiv:2203.16788

  3. Manoharan JS (2021) Capsule network algorithm for performance optimization of text classification. J Soft Comput Paradigm (JSCP) 3(01):1–9

    Article  MATH  Google Scholar 

  4. Ragesh R, Sellamanickam S, Iyer A, Bairi R, Lingam V (2021) Hetegcn: heterogeneous graph convolutional networks for text classification. In: Proceedings of the 14th ACM international conference on web search and data mining, pp 860–868

  5. Liang Y, Li H, Guo B, Yu Z, Zheng X, Samtani S, Zeng DD (2021) Fusion of heterogeneous attention mechanisms in multi-view convolutional neural network for text classification. Inf Sci 548:295–312

    Article  MATH  Google Scholar 

  6. Deng J, Cheng L, Wang Z (2021) Attention-based bilstm fused cnn with gating mechanism model for chinese long text classification. Comput Speech Lang 68:101182

    Article  MATH  Google Scholar 

  7. Jiang T, Wang D, Sun L, Yang H, Zhao Z, Zhuang F (2021) Lightxml: transformer with dynamic negative sampling for high-performance extreme multi-label text classification

  8. Moon SJ, Mo S, Lee K, Lee J, Shin J (2021) Masker: masked keyword regularization for reliable text classification. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 13578–13586

  9. Piao G (2021) Scholarly text classification with sentence bert and entity embeddings. In: Pacific-Asia conference on knowledge discovery and data mining. Springer, pp 79–87

  10. González-Carvajal S, Garrido-Merchán EC (2020) Comparing bert against traditional machine learning text classification. arXiv preprint arXiv:2005.13012

  11. Bai J, Bai S, Chu Y, Cui Z, Dang K, Deng X, Fan Y, Ge W, Han Y, Huang F et al (2023) Qwen technical report. arXiv preprint arXiv:2309.16609

  12. Boitel E, Mohasseb A, Haig E (2023) A comparative analysis of gpt-3 and bert models for text-based emotion recognition: performance, efficiency, and robustness. In: UK workshop on computational intelligence. Springer, pp 567–579

  13. Yang B, Luo X, Sun K, Luo MY (2023) Recent progress on text summarisation based on bert and gpt. In: International conference on knowledge science, engineering and management. Springer, pp 225–241

  14. Nazir A, Wang Z (2023) A comprehensive survey of chatgpt: advancements, applications, prospects, and challenges. Meta-radiology 100022

  15. Ding M, Zhou C, Yang H, Tang J (2020) Cogltx: applying bert to long texts. Adv Neural Inf Process Syst 33:12792–12804

    Google Scholar 

  16. Li X, Chan S, Zhu X, Pei Y, Ma Z, Liu X, Shah S (2023) Are chatgpt and gpt-4 general-purpose solvers for financial text analytics? a study on several typical tasks. In: Proceedings of the 2023 conference on empirical methods in natural language processing: industry track, pp 408–422

  17. Qiu Y, Jin Y (2024) Chatgpt and finetuned bert: a comparative study for developing intelligent design support systems. Intell Syst Appl 21:200308

    MATH  Google Scholar 

  18. Yang L, Kenny EM, Ng TLJ, Yang Y, Smyth B, Dong R (2020) Generating plausible counterfactual explanations for deep transformers in financial text classification. arXiv preprint arXiv:2010.12512

  19. Liang X, Cheng D, Yang F, Luo Y, Qian W, Zhou A (2020) F-hmtc: detecting financial events for investment decisions based on neural hierarchical multi-label text classification. In: IJCAI, pp 4490–4496

  20. Arslan Y, Allix K, Veiber L, Lothritz C, Bissyandé TF, Klein J, Goujon A (2021) A comparison of pre-trained language models for multi-class text classification in the financial domain. In: Companion proceedings of the web conference 2021, pp 260–268

  21. Rizun N, Waloszek W (2018) Methodology for text classification using manually created corpora-based sentiment dictionary. In: Proceedings of the 10th international joint conference on knowledge discovery, knowledge engineering and knowledge management (IC3K 2018)–Volume, vol 1, pp 212–220

  22. Abel J, Lantow B (2019) A methodological framework for dictionary and rule-based text classification. In: KDIR, pp 330–337

  23. Tang H-j, Yan D-f, Yuan T (2013) Semantic dictionary based method for short text classification. J China Univ Posts Telecommunications 20:15–19

    Article  MATH  Google Scholar 

  24. Lu K, Wu J (2019) Sentiment analysis of film review texts based on sentiment dictionary and svm. In: Proceedings of the 2019 3rd international conference on innovation in artificial intelligence, pp 73–77

  25. Zhang S, Wei Z, Wang Y, Liao T (2018) Sentiment analysis of chinese micro-blog text based on extended sentiment dictionary. Futur Gener Comput Syst 81:395–403

    Article  MATH  Google Scholar 

  26. Xu G, Yu Z, Yao H, Li F, Meng Y, Wu X (2019) Chinese text sentiment analysis based on extended sentiment dictionary. IEEE Access 7:43749–43762

    Article  Google Scholar 

  27. Novikova A, Stupnikov S (2017) Sentiment analysis of short texts from social networks using sentiment lexicons and blending of machine learning algorithms. In: Proc CEUR Workshop, pp 190–201

  28. Cho H, Kim S, Lee J, Lee J-S (2014) Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews. Knowl-Based Syst 71:61–71

    Article  MATH  Google Scholar 

  29. He R, Lee WS, Ng HT, Dahlmeier D (2019) An interactive multi-task learning network for end-to-end aspect-based sentiment analysis. arXiv preprint arXiv:1906.06906

  30. Deng D (2019) Research on algorithms and applications for sentient lexicon construction. Beingjing Jiaotong University

  31. Xiong Y, Feng Y, Wu H, Kamigaito H, Okumura M (2021) Fusing label embedding into bert: an efficient improvement for text classification. In: Findings of the association for computational linguistics: ACL-IJCNLP 2021, pp 1743–1750

  32. Pappagari R, Zelasko P, Villalba J, Carmiel Y, Dehak N (2019) Hierarchical transformers for long document classification. In: 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, pp 838–844

  33. Minaee S, Kalchbrenner N, Cambria E, Nikzad N, Chenaghlu M, Gao J (2021) Deep learning-based text classification: a comprehensive review. ACM Comput Surv (CSUR) 54(3):1–40

    Article  MATH  Google Scholar 

  34. Peters M, Neumann M, Iyyer M, Gardner M, Clark C, Lee K et al (2018) Deep contextualized word representations. arXiv:1802.05365. https://doi.org/10.18653/v1. N18-1202

  35. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30

  36. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  37. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

  38. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942

  39. Sanh V, Debut L, Chaumond J, Wolf T (2019) Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108

  40. Sun Y, Wang S, Li Y, Feng S, Chen X, Zhang H, Tian X, Zhu D, Tian H, Wu H (2019) Ernie: enhanced representation through knowledge integration. arXiv preprint arXiv:1904.09223

  41. Araci DF, Genc Z (2019) Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063

  42. Zhong Q, Ding L, Liu J, Du B, Tao D (2023) Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198

  43. Zhang H, Sun S, Hu Y, Liu J, Guo Y (2020) Sentiment classification for chinese text based on interactive multitask learning. IEEE Access 8:129626–129635

    Article  MATH  Google Scholar 

  44. Yang L, Kenny EM, Ng TLJ, Yang Y, Smyth B, Dong R (2020) Generating plausible counterfactual explanations for deep transformers in financial text classification. arXiv preprint arXiv:2010.12512

  45. Clark K, Luong MT, Le QV, Manning CD (2020) Electra: pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555

  46. Diao S, Bai J, Song Y, Zhang T, Wang Y (2019) Zen: pre-training chinese text encoder enhanced by n-gram representations. arXiv preprint arXiv:1911.00720

  47. Arslan Y, Allix K, Veiber L, Lothritz C, Bissyandé TF, Klein J, Goujon A (2021) A comparison of pre-trained language models for multi-class text classification in the financial domain. In: Companion proceedings of the web conference 2021, pp 260–268

  48. Dai ZYY, Yang Z (2019) Transformer-xl: attentive language models beyond a fixed-length context

  49. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9

    Google Scholar 

  50. Baidu: ERNIE Bot (2021). https://cloud.baidu.com/product/wenxinworkshop/

Download references

Funding

The authors did not receive support from any organization for the submitted work.

Author information

Authors and Affiliations

Authors

Contributions

Han Zhang mainly contributed to the study conception and wrote the first draft of the manuscript. Material preparation, data collection and analysis were performed by Yazhou Zhang and Xinyu Wang. The data set is annotated under the guidance of Lei Wang. Lixia Ji contributed to schedule control for the research. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Lixia Ji.

Ethics declarations

Competing interests

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, H., Zhang, Y., Wang, X. et al. An interactive multi-task ESG classification method for Chinese financial texts. Appl Intell 55, 191 (2025). https://doi.org/10.1007/s10489-024-06068-8

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06068-8

Keywords