Abstract
This paper presents a study using the Bidirectional Encoder Representations from Transformers (BERT) base model to classifying relations based on Vietnamese Covid-19 information. The study applies two BERT-base models: R-BERT and BERT with entity start. In this work, instead of using entity markers for input, typed entity markers are used. The typed entities include the patient with name, the patient with age, the patient with the job, patient with gender, patient with symptom and disease, patient with transportation. A Vietnamese dataset is labeled manually and the final Bert base model to classify Covid-19 relation is slightly better than the model applied entity marked.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
World Health Organization coronavirus website (2021). https://covid19.who.int/
Ministry of Health - website about the evidence of the respiratory disease Covid-19 (2021). https://ncov.moh.gov.vn/
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2019). arXiv:1810.04805
Devlin, J.: BERT: pre-training of deep bidirectional transformers for language understanding(2019). https://nlp.stanford.edu/seminar/details/jdevlin.pdf
Wu, S., He, Y.: Enriching pretrained language model with entity information for relation classification. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 2361–2364. ACM (2019)
Soares, L.B., FitzGerald, N., Ling, J., Kwiatkowski, T.: Matching the blanks: distributional similarity for relation learning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2895–2905 (2019)
Hendrickx, I., Kim, S.K., Kozareva, Z., et al.: SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In Proceedings of the 5th International Workshop on Semantic Evaluation, Uppsala, Sweden, pp. 33–38. Association for Computational Linguistics (2010)
Zhou, W., Chen, M.: An improved baseline for sentence-level relation extraction (2021). arXiv:2102.01373
Hebbar, S., Xie, Y.: CovidBERT-biomedical relation extraction for Covid-19. In: Proceedings of the International FLAIRS Conference, vol. 34 (2021)
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020)
Tran, M.V., Le, H.Q., Can, D.C., Nguyen, T.M.H., Nguyen, T.N.L., Doan, T.T.: Overview of VLSP RelEx shared task: a data challenge for semantic relation extraction from Vietnamese news. In: Proceedings of the 7th International Workshop on Vietnamese Language and Speech Processing (VLSP 2020), pp. 92–98 (2020)
Nguyen, T.M.H., Ngo, T.Q., Vu, X.L., Tran, M.V., Nguyen, T.T.H.: VLSP 2018 - named entity recognition for Vietnamese (VNER 2018) (2018)
Truong, H.T., Dao, H.M., Nguyen, Q.D.: Covid-19 named entity recognition for Vietnamese. In: Annual Conference of the North American Chapter of the Association for Computational Linguistics (2021)
Nguyen, Q.D., Nguyen, T.A.: PhoBERT: Pre-trained language models for Vietnamese. In: Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1037–1042 (2020)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach (2019). arXiv:1907.11692
Dataset (2021). https://github.com/GTMtremolo/Covid-19-relation-dataset
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Giang, T.M., Hung, P.D. (2021). Relation Classification Based on Vietnamese Covid-19 Information Using BERT Model with Typed Entity Markers. In: Dang, T.K., Küng, J., Chung, T.M., Takizawa, M. (eds) Future Data and Security Engineering. Big Data, Security and Privacy, Smart City and Industry 4.0 Applications. FDSE 2021. Communications in Computer and Information Science, vol 1500. Springer, Singapore. https://doi.org/10.1007/978-981-16-8062-5_33
Download citation
DOI: https://doi.org/10.1007/978-981-16-8062-5_33
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-8061-8
Online ISBN: 978-981-16-8062-5
eBook Packages: Computer ScienceComputer Science (R0)