Skip to main content
SpringerLink
Log in

An event-based opinion summarization model for long chinese text with sentiment awareness and parameter fusion mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

During the outbreak of a specific social event, end-to-end automatic opinion summarization is needed to analyze the surge of text related to the event. However, in the Chinese domain, the major existing works either emphasize salient aspects or sentence extraction in a discrete fashion with no consideration of human readability, or focus on short Chinese text. To remedy the drawbacks of these methods, in this paper, an event-based opinion summarization model for long Chinese text with a parameter fusion mechanism is proposed to address the human readability and imbalance issue of the event-based datasets. In particular, to capture the sentiment information in the source article in an end-to-end manner, a sentiment attention layer and a sentiment cross-entropy loss function are presented. In addition, when facing the issue of imbalance in event-based datasets, a parameter fusion mechanism inspired by the federated learning is proposed, which can further improve the human readability of the output. Finally, the efficacy of the proposed model is substantiated via comprehensive experiments performed on the collected event-based datasets, the Chinese long text summarization dataset (CLTS), and the cable news network/daily mail (CNN/DM) dataset using Recall-Oriented Understudy for Gisting Evaluation (ROUGE) and sentiment classification accuracy metrics. In addition, the source code is made available at https://github.com/ShawnYoung97/opinion-sum.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://github.com/Embedding/Chinese-Word-Vectors

  2. http://data.people.com.cn/

  3. https://github.com/thunlp/OpenHowNet

  4. https://download.pytorch.org/whl/torch_stable.html

  5. https://github.com/thunlp/OpenHowNet

  6. https://news.baidu.com/

  7. https://github.com/lxj5957/CLTS-Dataset

  8. https://www.thepaper.cn/

  9. https://github.com/deepmind/rc-data/

  10. https://github.com/SophonPlus/ChineseNlpCorpus/tree/master/datasets/online_shopping_10_cats

  11. https://github.com/fxsjy/jieba

  12. https://download.pytorch.org/whl/torch_stable.html

References

  1. Vanetik N, Litvak M, Churkin E, Last M (2020) An unsupervised constrained optimization approach to compressive summarization. Inf Sci 509:22–35. https://doi.org/10.1016/j.ins.2019.08.079

    Article  Google Scholar 

  2. Atri YK, Pramanick S, Goyal V, Chakraborty T (2021) See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization. Knowl-Based Syst 227:107152. https://doi.org/10.1016/j.knosys.2021.107152

    Article  Google Scholar 

  3. See A, Liu P, Manning CD (2017) Get to the point: Summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp 1073–1083. https://doi.org/10.18653/v1/p17-1099

  4. Lierde HV, Chow TWS (2019) Learning with fuzzy hypergraphs: a topical approach to query-oriented text summarization. Inf Sci 496:212–224. https://doi.org/10.1016/j.ins.2019.05.020

    Article  MATH  Google Scholar 

  5. Keneshloo Y, Shi T, Ramakrishnan N, Reddy CK (2020) Deep reinforcement learning for sequence-to-sequence models. IEEE Trans Neural Netw Learn Syst 31(7):2469–2489. https://doi.org/10.1109/tnnls.2019.2929141

    Google Scholar 

  6. Nallapati R, Zhou B, Santos Cd, Gu̇lçehre Ç, Xiang B (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, pp 280–290 https://doi.org/10.18653/v1/k16-1028

  7. Guarasci R, Silvestri S, De Pietro G, Fujita H, Esposito M (2022) BERT Syntactic transfer: A computational experiment on Italian, French and English languages. Comput Speech Lang 71:101261. https://doi.org/10.1016/j.csl.2021.101261

    Article  Google Scholar 

  8. Pota M, Ventura M, Fujita H, Esposito M (2021) Multilingual evaluation of pre-processing for BERT-based sentiment analysis of tweets. Expert Syst Appl 181:115119. https://doi.org/10.1016/j.eswa.2021.115119

    Article  Google Scholar 

  9. Guarasci R, Silvestri S, De Pietro G, Fujita H, Esposito M (2021) Assessing BERT’s ability to learn Italian syntax: A study on null-subject and agreement phenomena. J. Ambient Intell. Humaniz. Comput. https://doi.org/10.1007/s12652-021-03297-4

  10. Catelli R, Casola V, De Pietro G, Fujita H, Esposito M (2021) Combining contextualized word representation and sub-document level analysis through bi-LSTM+CRF architecture for clinical de-identification. Knowl-Based Syst 213:106649. https://doi.org/10.1016/j.knosys.2020.106649

    Article  Google Scholar 

  11. Catelli R, Gargiulo F, Casola V, De Pietro G, Fujita H, Esposito M (2020) Crosslingual named entity recognition for clinical de-identification applied to a COVID-19 Italian data set. Appl Soft Comput 97:106779. https://doi.org/10.1016/j.asoc.2020.106779

    Article  Google Scholar 

  12. Bengio S, Vinyals O, Jaitly N, Shazeer N (2015) Scheduled sampling for sequence prediction with recurrent neural networks. In: Proceedings of the Neural Information Processing Systems, pp 1171–1179. https://papers.nips.cc/paper/2015/file/e995f98d56967d946471af29d7bf99f1-Paper.pdf

  13. Chen Z, Ren J (2021) Multi-label text classification with latent word-wise label information. Appl Intell 51:966–979. https://doi.org/10.1007/s10489-020-01838-6

    Article  Google Scholar 

  14. Paulus R, Xiong C, Socher R (2017) A deep reinforced model for abstractive summarization. arXiv:https://export.arxiv.org/abs/1705.04304

  15. Wang L, Yao J, Tao Y, Zhong L, Liu W, Du Q (2018) A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pp 4453–4460. https://doi.org/10.24963/ijcai.2018/619

  16. Mohsen F, Wang J, Al-Sabahi K (2020) A hierarchical self-attentive neural extractive summarizer via reinforcement learning (HSASRL). Appl Intell 50 (9):2633–2646. https://doi.org/10.1007/s10489-020-01669-5

    Article  Google Scholar 

  17. Yang M, Wang X, Lu Y, Lv J, Shen Y, Li C (2020) Plausibility-promoting generative adversarial network for abstractive text summarization with multi-task constraint. Inf Sci 521:46–61. https://doi.org/10.1016/j.ins.2020.02.040

    Article  Google Scholar 

  18. Vo A-D, Nguyen Q-P, Ock C-Y (2020) Semantic and syntactic analysis in learning representation based on a sentiment analysis model. Appl Intell 50:663–680. https://doi.org/10.1007/s10489-019-01540-2

    Article  Google Scholar 

  19. Abdi A, Hasan S, Shamsuddin SM, Idris N, Piran J (2021) A hybrid deep learning architecture for opinion-oriented multi-document summarization based on multi-feature fusion. Knowl-Based Syst. 213:106658. https://doi.org/10.1016/j.knosys.2020.106658

    Article  Google Scholar 

  20. Yang M, Tu W, Wang J, Xu F, Chen X (2017) Attention-based LSTM for target-dependent sentiment classification. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp 5013–5014. https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14151/14248

  21. Wang C, Wang B, Xiang W, Xu M (2019) Encoding syntactic dependency and topical information for social emotion classification. In: Proceedings of the 42nd International ACM SIGIR Conference on research and development in information retrieval, pp 881–884. https://doi.org/10.1145/3331184.3331287

  22. Ji Y, Wu W, Chen S, Chen Q, Hu W, He L (2020) Two-stage sentiment classification based on user-product interactive information. Knowl-Based Syst 203:106091. https://doi.org/10.1016/j.knosys.2020.106091

    Article  Google Scholar 

  23. Winster SG, Kumar MN (2020) Automatic classification of emotions in news articles through ensemble decision tree classification techniques. J Ambient Intell Humaniz Comput 12(5):5709–5720. https://doi.org/10.1007/s12652-020-02373-5

    Article  Google Scholar 

  24. Sun S, Luo C, Chen J (2017) A review of natural language processing techniques for opinion mining systems. Inf Fusion 36:10–25. https://doi.org/10.1016/j.inffus.2016.10.004

    Article  Google Scholar 

  25. Liao W, Zeng B, Yin X, Wei P (2021) An improved aspect-category sentiment analysis model for text sentiment analysis based on roBERTa. Appl Intell 51:3522–3533. https://doi.org/10.1007/s10489-020-01964-1

    Article  Google Scholar 

  26. Abdi A, Shamsuddin SM, Hasan S, Piran J (2018) Machine learning-based multi-documents sentiment-oriented summarization using linguistic treatment. Expert Syst Appl 109:66–85. https://doi.org/10.1016/j.eswa.2018.05.010

    Article  Google Scholar 

  27. Mishra SK, Saini N, Saha S, Bhattacharyya P (2021) Scientific document summarization inmulti-objective clustering framework. Appl. Intell. https://doi.org/10.1007/s10489-021-02376-5

  28. Saini N, Saha S, Bhattacharyya P (2021) Microblog summarization using self-adaptive multi-objective binary differential evolution. Appl. Intell. https://doi.org/10.1007/s10489-020-02178-1

  29. Day M-Y, Lin Y-D (2017) Deep learning for sentiment analysis on google play consumer review. In: Proceedings of 2017 IEEE International Conference on Information Reuse and Integration, pp 382–388. https://doi.org/10.1109/iri.2017.79

  30. Rana T, Cheah Y-N, Rana T (2020) Multi-level knowledge-based approach for implicit aspect identification. Appl Intell 50:4616–4630. https://doi.org/10.1007/s10489-020-01817-x

    Article  Google Scholar 

  31. Li S, Zhao Z, Hu R, Li W, Liu T, Du X (2018) Analogical reasoning on Chinese morphological and semantic relations. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp 138–143. https://doi.org/10.18653/v1/p18-2023

  32. Dang X, Liao S, Cheng P, Liu J (2021) MF-COTE: A Chinese Opinion target extraction model based on multiple features. J Intell Fuzzy Syst 41(1):1611–1626. https://doi.org/10.3233/JIFS-210440

    Article  Google Scholar 

  33. Condori REL, Pardo TAS (2017) Opinion summarization methods: Comparing and extending extractive and abstractive approaches. Expert Syst Appl 78:124–134. https://doi.org/10.1016/j.eswa.2017.02.006

    Article  Google Scholar 

  34. Marcheggiani D, Frolov A, Titov I (2017) A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. In: Proceedings of the 21st Conference on Computational Natural Language Learning, pp 411–420. https://doi.org/10.18653/v1/k17-1041

  35. Li L, Fan Y, Tse M, Lin K (2020) A review of applications in federated learning. Comput Ind Eng 149:106854. https://doi.org/10.1016/j.cie.2020.106854

    Article  Google Scholar 

  36. Yao K, Zhang L, Du D, Luo T, Tao L, Wu Y (2020) Dual encoding for abstractive text summarization. IEEE Trans Cybern 50(3):985–996. https://doi.org/10.1109/tcyb.2018.2876317

    Article  Google Scholar 

  37. Chu E, Liu P (2019) Meansum: A neural model for unsupervised multi-document abstractive summarization. In: Proceedings of the 36th international conference on machine learning, vol 97, pp 1223–1232. http://proceedings.mlr.press/v97/chu19b/chu19b.pdf

  38. Chen M, Li L, Liu W (2018) A multi-view abstractive summarization model jointly considering semantics and sentiment. In: Proceedings of the 5th IEEE International Conference on Cloud Computing and Intelligence Systems, pp 741–746. https://doi.org/10.1109/ccis.2018.8691322

  39. Shuang K, Zhang Z, Guo H, Loo J (2018) A sentiment information collector–extractor architecture based neural network for sentiment analysis. Inf Sci 467:549–558. https://doi.org/10.1016/j.ins.2018.08.026

    Article  Google Scholar 

  40. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 31st Conference on Neural Information Processing Systems. https://papers.nips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

  41. Xu C, Feng J, Zhao P, Zhuang F, Wang D, Liu Y, Sheng VS (2021) Long- and short-term self-attention network for sequential recommendation. Neurocomputing 423:580–589. https://doi.org/10.1016/j.neucom.2020.10.066

    Article  Google Scholar 

  42. McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pp 1273–1282. http://proceedings.mlr.press/v54/mcmahan17a/mcmahan17a.pdf

  43. Wang S, Tuor T, Salonidis T, Leung KK, Makaya C, He T, Chan K (2019) Adaptive federated learning in resource constrained edge computing systems. IEEE J Sel Areas Commun 37(6):1205–1221. https://doi.org/10.1109/jsac.2019.2904348

    Article  Google Scholar 

  44. Liu W, Chen L, Chen Y, Zhang W (2020) Accelerating federated learning via momentum gradient descent. IEEE Trans Parallel Distrib Syst 31(8):1754–1766. https://doi.org/10.1109/tpds.2020.2975189

    Article  Google Scholar 

  45. Han S, Huang H, Tang Y (2020) Knowledge of words: an interpretable approach for personality recognition from social media. Knowl-Based Syst 194:105550. https://doi.org/10.1016/j.knosys.2020.105550

    Article  Google Scholar 

  46. Eyal M, Baumel T, Elhadad M (2019) Question answering as an automatic evaluation metric for news article summarization. In: Proceedings of the 2019 Conference of the North, pp 3938–3948. https://doi.org/10.18653/v1/n19-1395

  47. Rennie SJ, Marcheret E, Mroueh Y, Ross J, Goel V (2017) Self-critical sequence training for image captioning. In: Proc. of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2017.131

  48. Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3–4):229–2256. https://doi.org/10.1007/978-1-4615-3618-5_2

  49. Liu X, Zhang C, Chen X, Cao Y, Li J (2020) CLTS: A new Chinese long text summarization dataset. In: Proceedings of the nineth CCF International Conference on Natural Language Processing and Chinese Computing, pp 531–542. https://doi.org/10.1007/978-3-030-60450-9_42

  50. Nallapati R, Zhai F, Zhou B (2017) SummaruNNer: A recurrent neural network based sequence model for extractive summarization of documents. Proc Thirty-first AAAI Conf Artif Intell 31(1):3075–3081. https://ojs.aaai.org/index.php/AAAI/article/view/10958

    Google Scholar 

  51. Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2019) BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv:https://arxiv.org/abs/1910.13461

  52. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou Y, Li W, Liu P (2020) Exploring the limits of transfer learning with a unified text-to-text transformer. J Mach Learn Res 21:1–67

    MathSciNet  MATH  Google Scholar 

  53. Mihalcea R, Tarau P (2004) TextRank: bringing order into text. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp 404–411. https://aclanthology.org/W04-3252.pdf

  54. Zhong M, Liu P, Chen Y, Wang D, Qiuy X, Huang X (2020) Extractive summarization as text matching. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 6197–6208. https://doi.org/10.18653/v1/2020.acl-main.552

  55. Wang D, Liu P, Zheng Y, Qiuy X, Huang X (2020) Heterogeneous graph neural networks for extractive document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.553

  56. Cai X, Shi K, Jiang Y, Yang L, Liu S (2021) HITS-Based attentional neural model for abstractive summarization. Knowl-Based Syst 222:106996. https://doi.org/10.1016/j.knosys.2021.106996

    Article  Google Scholar 

  57. Liang Y, He F, Zeng X (2020) 3D mesh simplification with feature preservation based on whale optimization algorithm and differential evolution. Integr Comput-Aided Eng 27(4):417–435. https://doi.org/10.3233/ICA-200641

  58. Chen Y, He F, Li H, Zhang D, Wu Y (2020) A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration. Appl Soft Comput 93:106335. https://doi.org/10.1016/j.asoc.2020.106335

    Article  Google Scholar 

  59. Zhang S, He F (2019) DRCDN: Learning Deep residual convolutional dehazing networks. Vis Comput 36(9):1797–1808. https://doi.org/10.1007/s00371-019-01774-8

    Article  Google Scholar 

  60. Yang Y, He F, Han S, Liang Y, Cheng Y (2021) A novel attribute-based encryption approach with integrity verification for CAD assembly models. Engineering 7(6):787–797. https://doi.org/10.1016/j.eng.2021.03.011

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Key Research and Development Program of Sichuan province under Grant 2020YFG0076, in part by the Sichuan Science and Technology Program under Grant 2021YFG0159, in part by the by the Key Research and Development Program of Sichuan Province under Grant 2021YFG0156, in part by the Fundamental Research Funds for the Central Universities. Besides, kindly note that Shan Liao and Xiaoyang Li are jointly of the first authorship.

Author information

Authors and Affiliations

  1. School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China

    Shan Liao, Xiaoyang Li, Jiayong Liu, Anmin Zhou, Kai Li & Siqi Peng

Authors
  1. Shan Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiayong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anmin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Siqi Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiayong Liu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liao, S., Li, X., Liu, J. et al. An event-based opinion summarization model for long chinese text with sentiment awareness and parameter fusion mechanism. Appl Intell 53, 6682–6709 (2023). https://doi.org/10.1007/s10489-022-03231-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03231-x

Keywords

Search

Navigation