Skip to main content
SpringerLink
Log in

Graph-enhanced multi-answer summarization under question-driven guidance

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Multi-answer summarization for question-based queries in community Q&A requires comprehensive and in-depth analysis of lengthy and extensive information to generate concise and comprehensive answer summarization. Guided by the questions, capturing the relationships among candidate answers significantly benefits detecting salient information from multiple answers and generating an overall coherent summarization. In this paper, we propose a new Graph-enhanced Multi-answer Summarization under Question-driven Guidance model that enables explicit handling of the salience and redundancy of answer information. Specifically, the model first fully incorporates a pre-trained model to learn linguistic features through encoding and focuses on the role of questions in guiding answer generation during the encoding phase. The questions are utilized to explicitly constrain individual answers to ensure that the model more accurately identifies the information closely related to the questions in the answers and allocates more attention. Moreover, we utilize the question-driven answer graph information for encoding to capture the modeling relationships between answers and remove information redundancy. Finally, the graph-encoded information is exploited in the decoding stage to guide the generation of summaries to guarantee the informativeness, fluency, and conciseness of the summarization. Experimental results show that our proposed model brings substantial improvements compared to the state-of-the-art baseline, achieving the best outcomes on both of the community datasets ELI5 and MEDIQA, demonstrating the effectiveness of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Some data, models, and code generated or used during the study will be available under reasonable request from the corresponding author

References

  1. Hong K, Conroy J, Favre B, Kulesza A, Lin H, Nenkova A (2014) A repository of state of the art and competitive baseline summaries for generic news summarization. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pp 1608–1616. http://www.lrec-conf.org/proceedings/lrec2014/summaries/1093.html

  2. Zhang J, Tan J, Wan X (2018) Adapting neural single-document summarization model for abstractive multi-document summarization: a pilot study. In: Proceedings of the 11th International Conference on Natural Language Generation, pp 381–390. https://doi.org/10.18653/v1/w18-6545

  3. Baumel T, Eyal M, Elhadad M (2018) Query focused abstractive summarization: incorporating query relevance, multi-document coverage, and summary length constraints into seq2seq models. CoRR arXiv:abs/1801.07704

  4. Lebanoff L, Song K, Liu F (2018) Adapting the neural encoder-decoder framework from single to multi-document summarization. In: EMNLP, pp 4131–4141. https://doi.org/10.18653/v1/d18-1446

  5. Cohan A, Dernoncourt F, Kim DS, Bui T, Kim S, Chang W, Goharian N (2018) A discourse-aware attention model for abstractive summarization of long documents. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, pp 615–621. https://doi.org/10.18653/v1/n18-2097

  6. Liu PJ, Saleh M, Pot E, Goodrich B, Sepassi R, Kaiser L, Shazeer N (2018) Generating wikipedia by summarizing long sequences. In: 6th International Conference on Learning Representations. https://openreview.net/forum?id=Hyg0vbWC-

  7. Fabbri AR, Li I, She T, Li S, Radev DR (2019) Multi-news: a large-scale multi-document summarization dataset and abstractive hierarchical model. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pp 1074–1084. https://doi.org/10.18653/v1/p19-1102

  8. Liu Y, Lapata M (2019) Hierarchical transformers for multi-document summarization. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pp 5070–5081. https://doi.org/10.18653/v1/p19-1500

  9. Zhang J, Tan J, Wan X (2018) Towards a neural network approach to abstractive multi-document summarization. CoRR arXiv:abs/1804.09010

  10. Chen Y, Bansal M (2018) Fast abstractive summarization with reinforce-selected sentence rewriting. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, pp 675–686. https://aclanthology.org/P18-1063/

  11. Subramanian S, Li R, Pilault J, Pal CJ (2019) On extractive and abstractive neural document summarization with transformer language models. CoRR arXiv:abs/1909.03186

  12. Nguyen T, Rosenberg M, Song X, Gao J, Tiwary S, Majumder R, Deng L (2016) MS MARCO: a human generated machine reading comprehension dataset. In: Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 Co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016). CEUR Workshop Proceedings, vol 1773. http://ceur-ws.org/Vol-1773/CoCoNIPS_2016_paper9.pdf

  13. Rajpurkar P, Zhang J, Lopyrev K, Liang P (2016) Squad: 100, 000+ questions for machine comprehension of text. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, pp 2383–2392. https://doi.org/10.18653/v1/d16-1264

  14. Chopra S, Auli M, Rush AM (2016) Abstractive sentence summarization with attentive recurrent neural networks. In: NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp 93–98. https://doi.org/10.18653/v1/n16-1012

  15. Nallapati R, Zhou B, dos Santos CN, Gülçehre Ç, Xiang B (2016) Abstractive text summarization using sequence-to-sequence rnns and beyond. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, pp 280–290. https://doi.org/10.18653/v1/k16-1028

  16. See A, Liu PJ, Manning CD (2017) Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, pp 1073–1083. https://doi.org/10.18653/v1/P17-1099

  17. Paulus R, Xiong C, Socher R (2018) A deep reinforced model for abstractive summarization. In: 6th International Conference on Learning Representations, ICLR 2018. https://openreview.net/forum?id=HkAClQgA-

  18. Savery ME, Abacha AB, Gayen S, Demner-Fushman D (2020) Question-driven summarization of answers to consumer health questions. CoRR arXiv:abs/2005.09067

  19. Fan A, Jernite Y, Perez E, Grangier D, Weston J, Auli M (2019) ELI5: long form question answering. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pp 3558–3567. https://doi.org/10.18653/v1/p19-1346

  20. Radev DR, Jing H, Budzikowska M (2000) Centroid-based summarization of multiple documents: sentence extraction utility-based evaluation, and user studies. CoRR arXiv:cs.CL/0005020. https://doi.org/10.1016/j.ipm.2003.10.006

  21. Erkan G, Radev DR (2004) Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22:457–479. https://doi.org/10.1613/jair.1523

    Article  Google Scholar 

  22. Haghighi A, Vanderwende L (2009) Exploring content models for multi-document summarization. In: Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, pp 362–370. https://aclanthology.org/N09-1041/

  23. Christensen J, Mausam Soderland S, Etzioni O (2013) Towards coherent multi-document summarization. In: Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, pp 1163–1173. https://aclanthology.org/N13-1136/

  24. Nallapati R, Zhou B, Ma M (2016) Classify or select: neural architectures for extractive document summarization. CoRR arXiv:abs/1611.04244

  25. Lin C-Y (2004) Rouge: a package for automatic evaluation of summaries. In: Text summarization branches out, pp 74–81

  26. Li B, Yang P, Zhao H, Zhang P, Liu Z (2023) Hierarchical sliding inference generator for question-driven abstractive answer summarization. ACM Trans Inf Syst. https://doi.org/10.1145/3511891

    Article  Google Scholar 

  27. Gehrmann S, Deng Y, Rush AM (2018) Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 4098–4109. https://doi.org/10.18653/v1/d18-1443

  28. Celikyilmaz A, Bosselut A, He X, Choi Y (2018) Deep communicating agents for abstractive summarization. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, pp 1662–1675. https://doi.org/10.18653/v1/n18-1150

  29. Wan X (2008) An exploration of document impact on graph-based multi-document summarization. In: 2008 Conference on Empirical Methods in Natural Language Processing, EMNLP 2008, pp 755–762. https://aclanthology.org/D08-1079/

  30. Yin Y, Song L, Su J, Zeng J, Zhou C, Luo J (2019) Graph-based neural sentence ordering. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, pp 5387–5393. https://doi.org/10.24963/ijcai.2019/748

  31. Ganesan K, Zhai C, Han J (2010) Opinosis: a graph based approach to abstractive summarization of highly redundant opinions. In: COLING 2010, 23rd International Conference on Computational Linguistics, Proceedings of the Conference, pp 340–348. https://aclanthology.org/C10-1039/

  32. Liao K, Lebanoff L, Liu F (2018) Abstract meaning representation for multi-document summarization. In: Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, pp 1178–1190. https://aclanthology.org/C18-1101/

  33. Fan A, Gardent C, Braud C, Bordes A (2019) Using local knowledge graph construction to scale seq2seq models to multi-document inputs. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp 4184–4194. https://doi.org/10.18653/v1/D19-1428

  34. Huang L, Wu L, Wang L (2020) Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp 5094–5107. https://doi.org/10.18653/v1/2020.acl-main.457

  35. Li W, Xiao X, Liu J, Wu H, Wang H, Du J (2020) Leveraging graph to improve abstractive multi-document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp 6232–6243. https://doi.org/10.18653/v1/2020.acl-main.555

  36. Li W, Xu J, He Y, Yan S, Wu Y, Sun X (2019) Coherent comments generation for chinese articles with a graph-to-sequence model. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, pp 4843–4852. https://doi.org/10.18653/v1/p19-1479

  37. Jin H, Wang T, Wan X (2020) Semsum: semantic dependency guided neural abstractive summarization. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, pp 8026–8033. https://ojs.aaai.org/index.php/AAAI/article/view/6312

  38. Fan, A., Grave, E., Joulin, A.: Reducing transformer depth on demand with structured dropout. In: 8th International Conference on Learning Representations, ICLR 2020 (2020). https://openreview.net/forum?id=SylO2yStDr

  39. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, pp 4171–4186. https://doi.org/10.18653/v1/n19-1423

  40. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I et al (2019) Language models are unsupervised multitask learners. OpenAI blog 1(8):9

    Google Scholar 

  41. Lewis M, Liu Y, Goyal N, Ghazvininejad M, Mohamed A, Levy O, Stoyanov V, Zettlemoyer L (2020) BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, pp 7871–7880. https://doi.org/10.18653/v1/2020.acl-main.703

  42. Goodwin TR, Savery ME, Demner-Fushman D (2020) Towards zero shot conditional summarization with adaptive multi-task fine-tuning. In: Findings of the Association for Computational Linguistics: EMNLP 2020, vol EMNLP 2020, pp 3215–3226. https://doi.org/10.18653/v1/2020.findings-emnlp.289

  43. Kieuvongngam V, Tan B, Niu Y (2020) Automatic text summarization of COVID-19 medical research articles using BERT and GPT-2. CoRR arXiv:abs/2006.01997

  44. Liu Y (2019) Fine-tune BERT for extractive summarization. CoRR arXiv:abs/1903.10318

  45. Liu Y, Lapata M (2019) Text summarization with pretrained encoders. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pp 3728–3738. https://doi.org/10.18653/v1/D19-1387

  46. Zhang J, Zhao Y, Saleh M, Liu PJ (2020) PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, vol 119, pp 11328–11339. http://proceedings.mlr.press/v119/zhang20ae.html

  47. Johner T, Jana A, Biemann C (2021) Error analysis of using BART for multi-document summarization: a study for english and german language. In: Proceedings of the 23rd Nordic Conference on Computational Linguistics, NoDaLiDa 2021, pp 391–397. https://aclanthology.org/2021.nodalida-main.43/

  48. Zou Y, Zhang X, Lu W, Wei F, Zhou M (2020) Pre-training for abstractive document summarization by reinstating source text. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, pp 3646–3660. https://doi.org/10.18653/v1/2020.emnlp-main.297

  49. Zhou H, Ren W, Liu G, Su B, Lu W (2021) Entity-aware abstractive multi-document summarization. In: Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, vol ACL/IJCNLP 2021, pp 351–362. https://doi.org/10.18653/v1/2021.findings-acl.30

  50. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized BERT pretraining approach. CoRR arXiv:abs/1907.11692

  51. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21:140–114067 (2020). https://doi.org/10.48550/ARXIV.1910.10683

  52. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

  53. Krishna K, Roy A, Iyyer M (2021) Hurdles to progress in long-form question answering. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, pp 4940–4957. https://doi.org/10.18653/v1/2021.naacl-main.393

  54. Seo MJ, Kembhavi A, Farhadi A, Hajishirzi H (2017) Bidirectional attention flow for machine comprehension. In: 5th International Conference on Learning Representations, ICLR 2017. https://openreview.net/forum?id=HJ0UKP9ge

  55. Kenton JDM-WC, Toutanova LK (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, vol 1, p 2

  56. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62272100, the Consulting Project of Chinese Academy of Engineering under Grant 2023-XY-09, the Major Project of the National Social Science Fund of China under Grant 21ZD11, and the Fundamental Research Funds for the Central Universities.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Southeast University, Nanjing, China

    Bing Li, Peng Yang, Zhongjian Hu, Yuankang Sun & Meng Yi

  2. School of Cyber Science and Engineering, Southeast University, Nanjing, China

    Peng Yang

  3. Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China

    Bing Li, Peng Yang, Zhongjian Hu, Yuankang Sun & Meng Yi

Authors
  1. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhongjian Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuankang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meng Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethics approval

All authors read and approved the final version of the manuscript.

Consent to participate

All authors contributed to this work.

Consent for publication

All authors have checked the manuscript and have agreed to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, B., Yang, P., Hu, Z. et al. Graph-enhanced multi-answer summarization under question-driven guidance. J Supercomput 79, 20417–20444 (2023). https://doi.org/10.1007/s11227-023-05457-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05457-z

Keywords

Search

Navigation