Abstract
Extractive text summarization aims to select salient sentences from documents. However, most existing extractive methods struggle to capture inter-sentence relations in long documents. In addition, the hierarchical structure information of the document is ignored. For example, some scientific documents have fixed chapters, and sentences in the same chapter have the same theme. To solve these problems, this paper proposes a Fusion Hierarchical Structure Information Graph Neural Network for Extractive Long Documents Summarization. The model constructs a section node containing sentence nodes and global information according to the document structure. It integrates the hierarchical structure information of the text and uses position information to identify sentences. The section node acts as an intermediary node for information interaction between sentences, which better enriches the relationships between sentences and has higher computational efficiency. Our model has achieved excellent results on two datasets, PubMed and arXiv. Further analysis shows that the hierarchical structure information of documents helps the model select salient content better.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Beltagy, I., Peters, M.E., Cohan, A.: Longformer: the long-document transformer. arXiv preprint arXiv:2004.05150 (2020)
Cho, S., Song, K., Wang, X., Liu, F., Yu, D.: Toward unifying text segmentation and long document summarization. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 106–118 (Dec 2022)
Cohan, A., Dernoncourt, F., Kim, D.S., Bui, T., Kim, S., Chang, W., Goharian, N.: A discourse-aware attention model for abstractive summarization of long documents. arXiv preprint arXiv:1804.05685 (2018)
Cui, P., Hu, L.: Sliding selector network with dynamic memory for extractive summarization of long documents. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5881–5891 (2021)
Cui, P., Hu, L., Liu, Y.: Enhancing extractive text summarization with topic-aware graph neural networks. arXiv preprint arXiv:2010.06253 (2020)
Doan, X.D., Le Nguyen, M., Bui, K.H.N.: Multi graph neural network for extractive long document summarization. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 5870–5875 (2022)
Dong, Y., Mircea, A., Cheung, J.C.: Discourse-aware unsupervised summarization of long scientific documents. arXiv preprint arXiv:2005.00513 (2020)
Dong, Z., Tang, T., Li, L., Zhao, W.X.: A survey on long text modeling with transformers. arXiv preprint arXiv:2302.14502 (2023)
Erkan, G., Radev, D.R.: Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2004)
Grail, Q., Perez, J., Gaussier, E.: Globalizing bert-based transformer architectures for long document summarization. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 1792–1810 (2021)
Huang, Y.J., Kurohashi, S.: Extractive summarization considering discourse and coreference relations based on heterogeneous graph. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 3046–3052 (2021)
Iandola, F.N., Shaw, A.E., Krishna, R., Keutzer, K.W.: Squeezebert: what can computer vision teach NLP about efficient neural networks? arXiv preprint arXiv:2006.11316 (2020)
Jing, B., You, Z., Yang, T., Fan, W., Tong, H.: Multiplex graph neural network for extractive text summarization. arXiv preprint arXiv:2108.12870 (2021)
Liu, Y., Lapata, M.: Text summarization with pretrained encoders. arXiv preprint arXiv:1908.08345 (2019)
Miculicich, L., Han, B.: Document summarization with text segmentation. arXiv preprint arXiv:2301.08817 (2023)
Phan, T.A., Nguyen, N.D.N., Bui, K.H.N.: Hetergraphlongsum: heterogeneous graph neural network with passage aggregation for extractive long document summarization. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 6248–6258 (2022)
Rohde, T., Wu, X., Liu, Y.: Hierarchical learning for generation with long source sequences. arXiv preprint arXiv:2104.07545 (2021)
Ruan, Q., Ostendorff, M., Rehm, G.: HiStruct+: improving extractive text summarization with hierarchical structure information. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 1292–1308 (May 2022)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017)
Sefid, A., Giles, C.L.: Scibertsum: extractive summarization for scientific documents. In: Document Analysis Systems: 15th IAPR International Workshop, DAS 2022, La Rochelle, 22–25 May 2022, Proceedings, pp. 688–701 (2022)
Steinberger, J., et al.: Using latent semantic analysis in text summarization and summary evaluation. Proc. ISIM 4(93–100), 8 (2004)
Vanderwende, L., Suzuki, H., Brockett, C., Nenkova, A.: Beyond sumbasic: task-focused summarization with sentence simplification and lexical expansion. Inf. Process. Manag. 43(6), 1606–1618 (2007)
Wang, D., Liu, P., Zheng, Y., Qiu, X., Huang, X.: Heterogeneous graph neural networks for extractive document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6209–6219 (2020)
Xiao, W., Carenini, G.: Extractive summarization of long documents by combining global and local context. arXiv preprint arXiv:1909.08089 (2019)
Xiao, W., Carenini, G.: Systematically exploring redundancy reduction in summarizing long documents. arXiv preprint arXiv:2012.00052 (2020)
Xu, J., Gan, Z., Cheng, Y., Liu, J.: Discourse-aware neural extractive text summarization. arXiv preprint arXiv:1910.14142 (2019)
Yadav, A.K., Singh, A., Dhiman, M., Kaundal, R., Verma, A., Yadav, D.: Extractive text summarization using deep learning approach. Int. J. Inf. Technol. 14(5), 2407–2415 (2022)
Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., Huang, X.: Extractive summarization as text matching. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6197–6208 (2020)
Acknowledgements
This research was supported by “Pioneer” and “Leading Goose” R &D Program of Zhejiang (Grant No. 2023C03203, 2023C03180, 2022C03174).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhang, Z. et al. (2024). FHSI-GNN: Fusion Hierarchical Structure Information Graph Neural Network for Extractive Long Documents Summarization. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1963. Springer, Singapore. https://doi.org/10.1007/978-981-99-8138-0_12
Download citation
DOI: https://doi.org/10.1007/978-981-99-8138-0_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8137-3
Online ISBN: 978-981-99-8138-0
eBook Packages: Computer ScienceComputer Science (R0)