Abstract
SNS24 is a telephone service for triage, counselling, and referral service provided by the Portuguese National Health Service. Currently, following the predefined 59 Clinical Pathways, the selection of the most appropriate one is manually done by nurses. This paper presents a study on using automatic text classification to aid on the clinical pathway selection. The experiments were carried out on 3 months calls data containing 269,669 records and a selection of the best combination of ten text representations and four machine learning algorithm was pursued by building 40 different models. Then, fine-tuning of the algorithm parameters and the text embedding model were performed achieving a final accuracy of 78.80% and F1 of 78.45%. The best setup was then used to calculate the accuracy of the top-3 and top-5 most probable clinical pathways, reaching values of 94.10% and 96.82%, respectively. These results suggest that using a machine learning approach to aid the clinical triage in phone call services is effective and promising.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aggarwal, C.C., Clustering, C.R.D.: Algorithms and applications (2014)
Akbik, A., Blythe, D., Vollgraf, R.: Contextual string embeddings for sequence labeling. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 1638–1649. Association for Computational Linguistics, Santa Fe (2018). https://www.aclweb.org/anthology/C18-1139
Björnberg, A., Phang, A.Y.: Euro health consumer index 2018 report. In: Health Consumer Powerhouse Euro Health Consumer Index, pp. 1–90 (2019)
Consoli, B.S., Santos, J., Gomes, D., Cordeiro, F., Vieira, R., Moreira, V.: Embeddings for named entity recognition in geoscience Portuguese literature. In: Proceedings of the 12th Language Resources and Evaluation Conference, pp. 4625–4630 (2020)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding (2018)
Flores, C.A., Figueroa, R.L., Pezoa, J.E.: Active learning for biomedical text classification based on automatically generated regular expressions. IEEE Access 9, 38767–38777 (2021)
Gibson, A., et al.: Emergency department attendance after telephone triage: a population-based data linkage study. Health Serv. Res. 53(2), 1137–1162 (2018)
Gómez, D., Rojas, A.: An empirical overview of the no free lunch theorem and its effect on real-world machine learning classification. Neural Comput. 28(1), 216–228 (2016)
Kaakinen, P., Kyngäs, H., Tarkiainen, K., Kääriäinen, M.: The effects of intervention on quality of telephone triage at an emergency unit in Finland: nurses’ perspective. Int. Emerg. Nurs. 26, 26–31 (2016)
Kadhim, A.I.: Survey on supervised machine learning techniques for automatic text classification. Artif. Intell. Rev. 52(1), 273–292 (2019). https://doi.org/10.1007/s10462-018-09677-1
Kavuluru, R., Rios, A., Lu, Y.: An empirical evaluation of supervised learning approaches in assigning diagnosis codes to electronic medical records. Artif. Intell. Med. 65(2), 155–166 (2015)
Lake, R., et al.: The quality, safety and governance of telephone triage and advice services-an overview of evidence from systematic reviews. BMC Health Serv. Res. 17(1), 1–10 (2017). https://doi.org/10.1186/s12913-017-2564-x
Li, Y., Yang, T.: Word Embedding for Understanding Natural Language: A Survey, vol. 26 (2017). https://doi.org/10.1007/978-3-319-53817-4
Marafino, B.J., Boscardin, W.J., Dudley, R.A.: Efficient and sparse feature selection for biomedical text classification via the elastic net: application to ICU risk stratification from nursing notes. J. Biomed. Inf. 54, 114–120 (2015)
Mascio, A., et al.: Comparative analysis of text classification approaches in electronic health records. arXiv preprint arXiv:2005.06624 (2020)
McNemar, Q.: Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157 (1947)
Mujtaba, G., et al.: Clinical text classification research trends: systematic literature review and open issues. Expert Syst. Appl. 116, 494–520 (2019)
Mullenbach, J., Wiegreffe, S., Duke, J., Sun, J., Eisenstein, J.: Explainable prediction of medical codes from clinical text. arXiv preprint arXiv:1802.05695 (2018)
Mustafa, A., Rahimi Azghadi, M.: Automated machine learning for healthcare and clinical notes analysis. Computers 10(2), 24 (2021)
Navratil-Strawn, J.L., Ozminkowski, R.J., Hartley, S.K.: An economic analysis of a nurse-led telephone triage service. J. Telemedicine Telecare 20(6), 330–338 (2014)
North, F., et al.: Clinical decision support improves quality of telephone triage documentation-an analysis of triage documentation before and after computerized clinical decision support. BMC Med. Inf. Decis. Making 14(1), 1–10 (2014)
Santos, J., Consoli, B., dos Santos, C., Terra, J., Collonini, S., Vieira, R.: Assessing the impact of contextual embeddings for Portuguese named entity recognition. In: Proceedings of the 8th Brazilian Conference on Intelligent Systems, pp. 437–442 (2019)
Santos, J., dos Santos, H.D.P., Vieira, R.: Fall detection in clinical notes using language models and token classifier. In: Proceedings of the 33rd International Symposium on Computer-Based Medical Systems, CBMS 2020, Rochester, MN, USA, 28–30 July 2020, pp. 283–288 (2020)
Santos, J., Terra, J., Consoli, B.S., Vieira, R.: Multidomain contextual embeddings for named entity recognition. In: Proceedings of the 35th Conference of the Spanish Society for Natural Language Processing, pp. 434–441 (2019)
Shao, Y., Taylor, S., Marshall, N., Morioka, C., Zeng-Treitler, Q.: Clinical text classification with word embedding features vs. bag-of-words features. In: 2018 IEEE International Conference on Big Data (Big Data), pp. 2874–2878. IEEE (2018)
Shickel, B., Tighe, P.J., Bihorac, A., Rashidi, P.: Deep EHR: a survey of recent advances in deep learning techniques for electronic health record (EHR) analysis. IEEE J. Biomed. Health Inf. 22(5), 1589–1604 (2017)
Souza, F., Nogueira, R., Lotufo, R.: BERTimbau: pretrained BERT models for Brazilian Portuguese. In: Cerri, R., Prati, R.C. (eds.) BRACIS 2020. LNCS (LNAI), vol. 12319, pp. 403–417. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-61377-8_28
Stein, R.A., Jaques, P.A., Valiati, J.F.: An analysis of hierarchical text classification using word embeddings. Inf. Sci. 471, 216–232 (2019)
Topaz, M., Woo, K., Ryvicker, M., Zolnoori, M., Cato, K.: Home healthcare clinical notes predict patient hospitalization and emergency department visits. Nursing Res. 69(6), 448–454 (2020)
Tran, D.T., et al.: Compliance with telephone triage advice among adults aged 45 years and older: an Australian data linkage study. BMC Health Serv. Res. 17(1), 1–13 (2017). https://doi.org/10.1186/s12913-017-2458-y
Wagner Filho, J.A., Wilkens, R., Idiart, M., Villavicencio, A.: The brWaC corpus: a new open resource for Brazilian Portuguese. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan (2018). https://www.aclweb.org/anthology/L18-1686
Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. IEEE Comput. Intell. Mag. 13(3), 55–75 (2018)
Acknowledgement
This research work was funded by FCT – Fundaçño para Ciência e Tecnologia, I.P, within the project SNS24.Scout.IA: Aplicaçño de Metodologias de Inteligência Artificial e Processamento de Linguagem Natural no Serviço de Triagem, Aconselhamento e Encaminhamento do SNS24 (ref. DSAIPA/AI/0040/2019).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Veladas, R. et al. (2021). Aiding Clinical Triage with Text Classification. In: Marreiros, G., Melo, F.S., Lau, N., Lopes Cardoso, H., Reis, L.P. (eds) Progress in Artificial Intelligence. EPIA 2021. Lecture Notes in Computer Science(), vol 12981. Springer, Cham. https://doi.org/10.1007/978-3-030-86230-5_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-86230-5_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86229-9
Online ISBN: 978-3-030-86230-5
eBook Packages: Computer ScienceComputer Science (R0)