Abstract
Seeking a relevant answer to a biomedical question became a daily activity not only for experts but also for patients. In this perspective, biomedical extractive Question Answering systems have witnessed a rapid progress especially with the emergence of pre-trained language models such as BERT and its biomedical variant BioBERT. Those systems aim to extract an answer to a given question from a biomedical context and rely on two principal components question processing and exact answer identification. Several pre-trained language models-based systems have been proposed and focused only on the second component. In this paper, we proposed a BioBERT-based question answering system which rests on a question expansion phase. The Latter intends to extract question terms synonyms, as expansion terms, from multiple knowledge resource MeSH and WordNet. Indeed, we used firstly BioBERT pre-training model as a representation model in the selection of relevant expansion MeSH and WordNet terms. Secondly, in the fine-tuning phase to perform the question answering task and identify the exact answer. The experimental results on BioASQ dataset highlight the interest of the BioBert-based question expansion phase.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Xu, G., Rong, W., Wang, Y., Ouyang, Y., Xiong, Z.: External features enriched model for biomedical question answering. BMC Bioinform. 22(1), 272 (2021)
Mutabazi, E., Ni, J., Tang, G., Cao, W.: A review on medical textual question answering systems based on deep learning approaches. Appl. Sci. 11(12), 5456 (2021)
Badugu, S., Manivannan, R.: A study on different closed domain question answering approaches. Int. J. Speech Technol. 23(2), 315–325 (2020)
Esposito, M., Damiano, E., Minutolo, A., De Pietro, G., Fujita, H.: Hybrid query expansion using lexical resources and word embeddings for sentence retrieval in question answering. Inf. Sci. 514, 88–105 (2020)
Cao, Y., et al.: AskHERMES: an online question answering system for complex clinical questions. J. Biomed. Inform. 44(2), 277–288 (2011)
Sarrouti, M., El Alaoui, S.O.: SemBioNLQA: a semantic biomedical question answering system for retrieving exact and ideal answers to natural language questions. Artif. Intell. Med. 102, 101767 (2020)
Sharma, S., Patanwala, H., Shah, M., Deulkar, K.: A survey of medical question answering systems. Int. J. Eng. Tech. Res. (IJETR) ISSN 3, 131–133 (2015)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)
Dimitriadis, D., Tsoumakas, G.: Word embeddings and external resources for answer processing in biomedical factoid question answering. J. Biomed. Inform. 92, 103118 (2019)
Mikolov, T., et al.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Pennington, J., Socher, R., Manning, C.D.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2014)
Fan, Y.: Pre-training methods in information retrieval. Found. Trends® Inf. Retrieval 16(3), 178–317 (2022). https://doi.org/10.1561/1500000100
Nentidis, A., Bougiatiotis, K., Krithara, A., Paliouras, G.: Results of the seventh edition of the bioasq challenge. In: Cellier, P., Driessens, K. (eds.) ECML PKDD 2019, vol. 1168, pp. 553–568. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43887-6_51
Fu, E.R., Djoko, R., Mansor, M., Slater, R.: BERT for question answering on bioasq. SMU Data Sci. Rev. 3(3), 3 (2020)
Das, B., Nirmala, S.J.: Improving healthcare question answering system by identifying suitable answers. In: 2022 IEEE 2nd Mysore Sub Section International Conference (MysuruCon), pp. 1–6. IEEE (2022)
Du, Y., Yan, J., Lu, Y., Zhao, Y., Jin, X.: Improving biomedical question answering by data augmentation and model weighting. IEEE/ACM Trans. Comput. Biol. Bioinf. 20(2), 1114–1124 (2022)
Alzubi, J.A., Jain, R., Singh, A., Parwekar, P., Gupta, M.: COBERT: COVID-19 question answering system using BERT. Arab. J. Sci. Eng. 48(8), 11003–11013 (2023)
Lewandowski, B., Morris, R., Paul, P.M., Slater, R.: Question answering with distilled BERT models: a case study for biomedical data. SMU Data Sci. Rev. 7(1), 9 (2023)
Kammoun, H., Gabsi, I., Amous, I.: Mesh-based semantic indexing approach to enhance biomedical information retrieval. Comput. J. 65(3), 516–536 (2022)
Selmi, W., Kammoun, H., Amous, I.: Semantic-based hybrid query reformulation for biomedical information retrieval. Comput. J. 66(9), 2296–2316 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gabsi, I., Kammoun, H., Wederni, A., Amous, I. (2024). BioBERT for Multiple Knowledge-Based Question Expansion and Biomedical Extractive Question Answering. In: Nguyen, N.T., et al. Computational Collective Intelligence. ICCCI 2024. Lecture Notes in Computer Science(), vol 14810. Springer, Cham. https://doi.org/10.1007/978-3-031-70816-9_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-70816-9_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-70815-2
Online ISBN: 978-3-031-70816-9
eBook Packages: Computer ScienceComputer Science (R0)