Abstract
Text summarisation has been recognised as an important Natural Language Processing task, attracting great attention from both researchers and practitioners. It has been widely adopted in various domains. For example, text summarisation of news, articles and book chapters can produce a short text, assisting the readers with grasping the main idea rapidly. In the medical domain, it is also applied to summarise the patients’ questions. However, it is very challenging to control the summariser output by producing domain-specific summaries since the focus of domain-specific information may be ignored. In this paper, we propose a novel summarisation model aiming at producing summaries by focusing on the domain-specific knowledge, where hybrid embeddings, i.e., focus, domain and context embeddings, are utilised. We conduct extensive experiments to evaluate our novel model by using the MeQSum dataset. The experimental results demonstrate that our model outperforms state-of-the-art algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abacha, A.B., M’rabet, Y., Zhang, Y., Shivade, C., Langlotz, C., Demner-Fushman, D.: Overview of the MEDIQA 2021 shared task on summarization in the medical domain. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 74–85 (2021)
Balumuri, S., Bachina, S., Kamath, S.: Sb_nitk at MEDIQA 2021: leveraging transfer learning for question summarization in medical domain. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 273–279 (2021)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 4171–4186 (2019)
Fiszman, M., Rindflesch, T.C., Kilicoglu, H.: Abstraction summarization for managing the biomedical research literature. In: Proceedings of the Computational Lexical Semantics Workshop at HLT-NAACL 2004, pp. 76–83 (2004)
Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4098–4109 (2018)
Gu, J., Lu, Z., Li, H., Li, V.O.: Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1631–1640 (2016)
Guan, W., Smetannikov, I., Tianxing, M.: Survey on automatic text summarization and transformer models applicability. In: 2020 International Conference on Control, Robotics and Intelligent System, pp. 176–184 (2020)
He, Y., Chen, M., Huang, S.: damo_nlp at MEDIQA 2021: knowledge-based preprocessing and coverage-oriented reranking for medical question summarization. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 112–118 (2021)
Hermann, K.M., et al.: Teaching machines to read and comprehend. Adv. Neural. Inf. Process. Syst. 28, 1693–1701 (2015)
Kim, D., et al.: A neural named entity recognition and multi-type normalization tool for biomedical text mining. IEEE Access 7, 73729–73740 (2019)
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)
Lee, J., Dang, H., Uzuner, O., Henry, S.: MNLP at MEDIQA 2021: fine-tuning Pegasus for consumer health question summarization. In: Proceedings of the 20th Workshop on Biomedical Language Processing, pp. 320–327 (2021)
Lewis, M., et al.: BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81 (2004)
Luhn, H.P.: The automatic creation of literature abstracts. IBM J. Res. Dev. 2(2), 159–165 (1958)
Luong, T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. In: EMNLP (2015)
Nallapati, R., Zhai, F., Zhou, B.: SummaRuNNER: a recurrent neural network based sequence model for extractive summarization of documents. In: Thirty-First AAAI Conference on Artificial Intelligence (2017)
Narayan, S., Cohen, S.B., Lapata, M.: Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797–1807 (2018)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020). http://jmlr.org/papers/v21/20-074.html
Reeve, L.H., Han, H., Brooks, A.D.: The use of domain-specific concepts in biomedical text summarization. Inf. Process. Manage. 43(6), 1765–1776 (2007)
Roberts, K., Kilicoglu, H., Fiszman, M., Demner-Fushman, D.: Decomposing consumer health questions. In: Proceedings of BioNLP 2014, pp. 29–37. Association for Computational Linguistics, Baltimore, Maryland, June 2014. https://doi.org/10.3115/v1/W14-3405, https://aclanthology.org/W14-3405
Sarkar, K.: Using domain knowledge for text summarization in medical domain. Int. J. Recent Trends Eng. 1(1), 200 (2009)
See, A., Liu, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1073–1083 (2017)
Wang, D., Liu, P., Zhong, M., Fu, J., Qiu, X., Huang, X.: Exploring domain shift in extractive text summarization. arXiv preprint arXiv:1908.11664 (2019)
Xu, H., Liu, B., Shu, L., Philip, S.Y.: Double embeddings and CNN-based sequence labeling for aspect extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 592–598 (2018)
Zhang, J., Zhao, Y., Saleh, M., Liu, P.: Pegasus: pre-training with extracted gap-sentences for abstractive summarization. In: International Conference on Machine Learning, pp. 11328–11339. PMLR (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Shi, J., Hellesoe, L., Wang, G., Li, W., Bai, Q. (2022). Focus-Based Text Summarisation with Hybrid Embeddings. In: Long, G., Yu, X., Wang, S. (eds) AI 2021: Advances in Artificial Intelligence. AI 2022. Lecture Notes in Computer Science(), vol 13151. Springer, Cham. https://doi.org/10.1007/978-3-030-97546-3_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-97546-3_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-97545-6
Online ISBN: 978-3-030-97546-3
eBook Packages: Computer ScienceComputer Science (R0)