Skip to main content
SpringerLink
Log in

Legal public opinion news abstractive summarization by incorporating topic information

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Automatically generate accurate summaries from legal public opinion news can help readers to grasp the main ideas of news quickly. Although many improved sequence-to-sequence models have been proposed for the abstractive text summarization task, these approaches confront two challenges when addressing domain-specific summarization task: (1) the appropriate selection of domain knowledge; (2) the effective manner of integrating domain knowledge into summarization model. In order to tackle the above challenges, this paper selects the pre-training topic information as the legal domain knowledge, which is then integrated into the sequence-to-sequence model to improve the performance of public opinion news summarization. Concretely, two kinds of topic information are utilized: first, the topic words which denote the main aspects of the source document are encoded to guide the decoding process. Furthermore, the predicted output is forced to have a similar topic probability distribution with the source document. We evaluate our model on a large dataset of legal public opinion news collected from micro-blog, and the experimental results show that the proposed model outperforms existing baseline systems under the rouge metrics. To the best of our knowledge, this work represents the first attempt in the legal public opinion domain for text summarization task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://github.com/fxsjy/jieba.

  2. https://radimrehurek.com/gensim/models/ldamodel.html.

  3. https://github.com/bheinzerling/pyrouge.

References

  1. Klein G, Kim Y, Deng Y, Senellart J, Rush A (2017) OpenNMT: open-source toolkit for neural machine translation. In: Proceedings of the 55th ACL annual meeting of the association for computational linguistics. ACL, pp 67–72

  2. Rush AM, Chopra S, Weston J (2015) Neural attention model for abstractive sentence summarization. In: Proceedings of the 2015 conference on empirical methods in natural language processing, ACL, pp 379–389

  3. Bahdanau D, Chorowski J, Serdyuk D, Brakel P, Bengio Y (2016) End-to-end attention-based large vocabulary speech recognition. In: Proceedings of the IEEE international conference on acoustics speech and signal processing. IEEE, pp 4945–4949

  4. Zhou Q, Yang N, Wei F, Huang S, Zhou M, Zhao T (2018) Neural document summarization by jointly learning to score and select sentences. In: Proceedings of the 56th annual meeting of the association for computational linguistics. ACL, pp 654–663

  5. Nallapati R, Zhou B, dos Santos C, Gulcehre C, Xiang B (2016) Abstractive text summarization using sequence-to-sequence RNNs and beyond. In: Proceedings of the 20th conference on computational natural language learning. ACL, pp 280–290

  6. See A, Liu PJ, Manning CD (2017) Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th annual meeting of the association for computational linguistics. ACL, pp 1073–1083

  7. Gu J, Lu Z, Li H, Li VO (2016) Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th annual meeting of the association for computational linguistics. ACL, pp 1631–1640

  8. Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Proceedings of the 28th international conference on neural information processing systems. MIT Press, pp 2692–2700

  9. Gulcehre C, Ahn S, Nallapati R, Zhou B, Bengio Y (2016) Pointing the unknown words. In: Proceedings of the 54th annual meeting of the association for computational linguistics. ACL, pp 140–149

  10. Song K, Zhao L, Liu F (2018) Structure-infused copy mechanisms for abstractive summarization. In: Proceedings of the 27th international conference on computational linguistics, ACL, Santa Fe, August 20–26 2018, pp 1717–1729

  11. Paulus R, Xiong C, Socher R (2017) A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304

  12. Zhang X, Lapata M (2017) Sentence simplification with deep reinforcement learning. In: Proceedings of the 2017 conference on empirical methods in natural language processing. ACL, pp 584–594

  13. Pasunuru R, Bansal M (2018) Multi-Reward reinforced summarization with saliency and entailment. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics. ACL, pp 646–653

  14. Zhou Q, Yang N, Wei F, Zhou M (2017) Selective encoding for abstractive sentence summarization. In: Proceedings of the 55th annual meeting of the association for computational linguistics. ACL, pp 1095–1104

  15. Xia Y, Tian F, Wu L, Lin J, Qin T, Yu N, Liu TY (2017) Deliberation networks: sequence generation beyond one-pass decoding. In: Proceedings of the 31st international conference on neural information processing systems. MIT Press, pp 1784–1794

  16. Zeng W, Luo W, Fidler S, Urtasun R (2016) Efficient summarization with read-again and copy mechanism. arXiv preprint arXiv:1611.03382

  17. Chen YC, Bansal M (2018) Fast abstractive summarization with reinforce-selected sentence rewriting. In: Proceedings of the 56th annual meeting of the association for computational linguistics. ACL, pp 675–686

  18. Hsu W T, Lin C K, Lee M Y, Min K, Tang J, Sun M (2018) A unified model for extractive and abstractive summarization using inconsistency loss. In: Proceedings of the 56th annual meeting of the association for computational linguistics. ACL, pp 132–141

  19. Wang L, Yao J, Tao Y, Zhong L, Liu W, Du Q (2018) A reinforced topic-aware convolutional sequence-to-sequence model for abstractive text summarization. In: Proceedings of the international joint conference on artificial intelligence. Morgan Kaufmann, pp 4453–4460

  20. Narayan S, Cohen SB, Lapata M (2018) Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In: Proceedings of the 2018 conference on empirical methods in natural language processing. ACL, pp 1797–1807

  21. Hou L, Hu P, Cao W (2019) Automatic chinese abstractive summarization with topical keywords fusion. Acta Autom Sin 45(3):530–539

    MATH  Google Scholar 

  22. Wang Y, Li J, Chan HP (2019) Topic-aware neural keyphrase generation for social media language. arXiv preprint arXiv:1906.03889

  23. Miao Y, Grefenstette E, Blunsom P (2017) Discovering discrete latent topics with neural variational inference. In: Proceedings of the 34th international conference on machine learning. ACM, pp 2410–2419

  24. Kumar R, Raghuveer K (2012) Legal document summarization using latent dirichlet allocation. Int J Comput Sci Telecommun 3:114–117

    Google Scholar 

  25. Galgani F, Compton P, Hoffmann A (2012) Combining different summarization techniques for legal text. In: Proceedings of the workshop on innovative hybrid approaches to the processing of textual data. Association for Computational Linguistics. ACL, pp 115–123

  26. Elnaggar A, Gebendorfer C, Glaser I (2018) Multi-task deep learning for legal document translation, summarization and multi-label classification. In: Proceedings of the 2018 artificial intelligence and cloud computing conference. ACM, pp 9–15

  27. Manor L, Li JJ (2019) Plain english summarization of contracts. In: Proceedings of the natural legal language processing workshop. ACL, pp 1–11

  28. Ma S, Sun X, Lin J, Reb X(2018) A hierarchical end-to-end model for jointly improving text summarization and sentiment classification. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence. Morgan Kaufmann, pp 4251–4257

  29. Hochreiter S, Jürgen S (1997) LSTM can solve hard long time lag problems. In: Proceedings of the advances in neural information processing systems. MIT Press, pp 473–479

  30. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  31. Bahdanau D, Chorowski J, Serdyuk D, Brakel P, Bengio Y (2016) End-to-end attention-based large vocabulary speech recognition. In: Proceeding of the 2016 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 4945–4949

  32. Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  33. Lin CY (2004) Rouge: A package for automatic evaluation of summaries. In: Proceedings of the workshop on text summarization branches out, post conference workshop of ACL. ACL, pp 74–81

  34. Jonas G, Michael A, David G, Denis Y, Yann ND (2017) Convolutional sequence to sequence learning. In: Proceedings of the 34th international conference on machine learning. ACM, pp 1243–1252

  35. Paszke A, Gross S, Chintala S (2017) Automatic differentiation in PyTorch. In: Proceedings of the NIPS auto diff workshop. MIT Press

  36. Hu Z, Li X, Tu C, Liu Z, Sun M (2018) Few-shot charge prediction with discriminative legal attributes. In: Proceedings of the 27th international conference on computational linguistics. ACL, pp 487–498

  37. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  38. Sutskever I, Martens J, Dahl G, and Hinton G (2013) On the importance of initialization and momentum in deep learning. In: Proceedings of International conference on machine learning. ACM, pages 1139–1147

  39. Hermann KM, Kocisky T, Grefenstette E, Espeholt L, Kay W, Suleyman M, Blunsom P (2015) Teaching machines to read and comprehend. In: Proceedings of neural information processing systems. MIT Press, pp 1693–1701

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their constructive comments. This work was supported by National Key Research and Development Program of China (Grant Nos. 2018YFC0830105, 2018YFC0830101, 2018YFC0830100); National Natural Science Foundation of China (Grant Nos. 61972186, 61762056, 61472168).

Author information

Authors and Affiliations

  1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, China

    Yuxin Huang, Zhengtao Yu, Junjun Guo, Zhiqiang Yu & Yantuan Xian

  2. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, 650500, China

    Yuxin Huang, Zhengtao Yu, Junjun Guo, Zhiqiang Yu & Yantuan Xian

Authors
  1. Yuxin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengtao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junjun Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhiqiang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yantuan Xian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengtao Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, Y., Yu, Z., Guo, J. et al. Legal public opinion news abstractive summarization by incorporating topic information. Int. J. Mach. Learn. & Cyber. 11, 2039–2050 (2020). https://doi.org/10.1007/s13042-020-01093-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01093-8

Keywords

Search

Navigation