Abstract
As diabetes has become one of the major public health challenges worldwide, the importance of automatic question-answering services for diabetes in daily healthcare is gradually being highlighted. To address this challenge, this paper aims to improve the accuracy of classifying Chinese diabetes questions. In order to achieve this goal, we utilized BERT pre-training models to extract text feature vectors, combined them with convolutional neural networks to extract local features, and employed pooling and fully connected layers for classification. Additionally, we also tested different BERT models and obtained the best results using voting fusion technology. The final accuracy improved by 2% compared to a single model. Experimental results demonstrate that the model can effectively classify Chinese diabetes questions, providing robust support for the implementation of automated diabetes question-answering services.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wang, Y., Zhou, Z., Jin, S., Liu, D., Lu, M.: Comparisons and selections of features and classifiers for short text classification. In: Iop Conference Series: Materials Science and Engineering, vol. 261, p. 012018. IOP Publishing (2017)
Wang, Y., Zhang, X., Mi, L., Wang, H., Choe, Y.: Attention augmentation with multi-residual in bidirectional LSTM. Neurocomputing 385, 340–347 (2020)
Miao, F., Zhang, P., Jin, L., Wu, H.: Chinese news text classification based on machine learning algorithm. In: 2018 10th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), vol. 2, pp. 48–51. IEEE (2018)
González-Carvajal, S., Garrido-Merchán, E.C.: Comparing bert against traditional machine learning text classification. arXiv preprint arXiv:2005.13012 (2020)
Wang, Y., Wang, H., Zhang, X., Chaspari, T., Choe, Y., Lu, M.: An attention-aware bidirectional multi-residual recurrent neural network (abmrnn): a study about better short-term text classification. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3582–3586. IEEE (2019)
Wang, Y., Liao, J., Yu, H., Leng, J.: Semantic-aware conditional variational autoencoder for one-to-many dialogue generation. Neural Comput. Appl. 34(16), 13683–13695 (2022)
Li, Y., Wang, X., Pengjian, X.: Chinese text classification model based on deep learning. Future Internet 10(11), 113 (2018)
Li, Q., et al.: A survey on text classification: From traditional to deep learning. ACM Trans. Intell. Syst. Technol. (TIST) 13(2), 1–41 (2022)
Hongxia, L., Ehwerhemuepha, L., Rakovski, C.: A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance. BMC Med. Res. Methodol. 22(1), 181 (2022)
Hajibabaee, P., et al.: Offensive language detection on social media based on text classification. In: 2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0092–0098. IEEE (2022)
Wan, C.-X., Li, B.: Financial causal sentence recognition based on bert-cnn text classification. J. Supercomput., 1–25 (2022)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Radford, A., Jeffrey, W., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: Albert: a lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(1), 5485–5551 (2020)
Lewis, M., et al.: Bart: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune BERT for text classification? In: Sun, M., Huang, X., Ji, H., Liu, Z., Liu, Y. (eds.) CCL 2019. LNCS (LNAI), vol. 11856, pp. 194–206. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32381-3_16
Hu, Y., Ding, J., Dou, Z., Chang, H.: Short-text classification detector: a bert-based mental approach. Computational Intelligence and Neuroscience 2022 (2022)
Cai, F., Ye, H.: Chinese medical text classification with roberta. In: International Symposium on Biomedical and Computational Biology, pp. 223–236. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-25191-7_17
Qing, L., Linhong, W., Xuehai, D.: A novel neural network-based method for medical text classification. Future Internet 11(12), 255 (2019)
Ali Saleh Alammary: Bert models for Arabic text classification: a systematic review. Appl. Sci. 12(11), 5720 (2022)
Kulkarni, A., Mandhane, M., Likhitkar, M., Kshirsagar, G., Jagdale, J., Joshi, R.: Experimental evaluation of deep learning models for Marathi text classification. In: Gunjan, V.K., Zurada, J.M. (eds.) Proceedings of the 2nd International Conference on Recent Trends in Machine Learning, IoT, Smart Cities and Applications. LNNS, vol. 237, pp. 605–613. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-6407-6_53
Briskilal, J., Subalalitha, C.N.: An ensemble model for classifying idioms and literal texts using bert and roberta. Inf. Process. Manage. 59(1), 102756 (2022)
Acknowledgements
This work was partly supported by the National Key R &D Program of China (2021YFF0704100), the National Natural Science Foundation of China (62136002, 61876027, 61936001), the Science and Technology Research Program of Chongqing Municipal Education Commission (KJQN202100627 and KJQN202100629), and the National Natural Science Foundation of Chongqing (cstc2022ycjh-bgzxm0004, cstc2019jcyj-cxttX0002), respectively.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yu, Z., Wang, Y., Lei, D. (2023). Research on Chinese Diabetes Question Classification with the Integration of Different BERT Models. In: Zhang, H., et al. International Conference on Neural Computing for Advanced Applications. NCAA 2023. Communications in Computer and Information Science, vol 1870. Springer, Singapore. https://doi.org/10.1007/978-981-99-5847-4_41
Download citation
DOI: https://doi.org/10.1007/978-981-99-5847-4_41
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-5846-7
Online ISBN: 978-981-99-5847-4
eBook Packages: Computer ScienceComputer Science (R0)