Abstract
Coronavirus disease 2019 (COVID-19) is an acute disease, which can rapidly become severe. Hence, it is of great significance to realize the automatic diagnosis of COVID-19. However, existing models are often inapplicable for fusing patients’ demographic information due to its low dimensionality. To address this, we propose a COVID-19 patient diagnosis method with feature fusion and a model based on Swin Transformer. Specifically, two auxiliary tasks are added for fusing computed tomography (CT) images and patients’ demographic information, which utilizes the patients’ demographic information as the label for the auxiliary tasks. Besides, our approach involves designing a Swin Transformer model with Enhanced Multi-head Self-Attention (EMSA) to capture different features from CT data. Meanwhile, the EMSA module is able to extract and fuse attention information in different representation subspaces, further enhancing the performance of the model. Furthermore, we evaluate our model in COVIDx CT-3 dataset with different tasks to classify Normal Controls (NC), COVID-19 cases and community-acquired pneumonia (CAP) cases and compare the performance of our method with other models, which show the effectiveness of our model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xu, Z., Shi, L., Wang, Y., Zhang, J., Huang, L., Zhang, C., et al.: Pathological findings of COVID-19 associated with acute respiratory distress syndrome. Lancet Respir. Med. 8(4), 420–422 (2020). https://doi.org/10.1016/S2213-2600(20)30076-X
Risch, H.A.: Early outpatient treatment of symptomatic, high-risk COVID-19 patients that should be ramped up immediately as key to the pandemic crisis. Am. J. Epidemiol. 189(11), 1218–1226 (2020). https://doi.org/10.1093/aje/kwaa093
Lunz, D., Batt, G., Ruess, J.: To isolate, or not to isolate: a theoretical framework for disease control via contact tracing. medRxiv 1–9 (2020)
Kong, W., Agarwal, P.P.: Chest imaging appearance of COVID-19 infection. Radiol. Cardiothorac. Imaging 2(1), e200028 (2020). https://doi.org/10.1148/ryct.2020200028
China NHCotPsRo: New diagnosis and treatment of coronary pneumonia. (2020)
Zhang, K., Liu, X., Shen, J., Li, Z., Sang, Y., Wu, X., et al.: Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell 181(6), 1423-1433.e11 (2020). https://doi.org/10.1016/j.cell.2020.04.045
Suzuki, K.: Overview of deep learning in medical imaging. Radiol. Phys. Technol. 10(3), 257–273 (2017). https://doi.org/10.1007/s12194-017-0406-5
Mei, X., et al.: Artificial intelligence–enabled rapid diagnosis of patients with COVID-19. Nat. Med. 26(8), 1224–1228 (2020). https://doi.org/10.1038/s41591-020-0931-3
Chen, J., Wu, L., Zhang, J., Zhang, L., Gong, D., Zhao, Y., et al.: Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography. Sci. Rep. 10(1), 19196 (2020). https://doi.org/10.1038/s41598-020-76282-0
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., et al.: Using artificial intelligence to detect COVID-19 and community-acquired pneumonia based on pulmonary CT: evaluation of the diagnostic accuracy. Radiology 296(2), E65–E71 (2020). https://doi.org/10.1148/radiol.2020200905
Silva, P., Luz, E., Silva, G., Moreira, G., Silva, R., Lucio, D., et al.: COVID-19 detection in CT images with deep learning: a voting-based scheme and cross-datasets analysis. Inform. Med. Unlocked. 20, 100427 (2020). https://doi.org/10.1016/j.imu.2020.100427
Liu, M., Zhang, J., Adeli, E., Shen, D.: Joint classification and regression via deep multi-task multi-channel learning for Alzheimer’s disease diagnosis. IEEE Trans. Biomed. Eng. 66(5), 1195–1206 (2019). https://doi.org/10.1109/TBME.2018.2869989
Hazarika, D., Poria, S., Zimmermann, R., Mihalcea, R.: Conversational transfer learning for emotion recognition. Inf. Fusion. 65, 1–12 (2021). https://doi.org/10.1016/j.inffus.2020.06.005
Jaiswal, A., Gianchandani, N., Singh, D., Kumar, V., Kaur, M.: Classification of the COVID-19 infected patients using DenseNet201 based deep transfer learning. J. Biomol. Struct. Dyn. 39(15), 5682–5689 (2021). https://doi.org/10.1080/07391102.2020.1788642
Ali, F., El-Sappagh, S., Islam, S.M.R., Kwak, D., Ali, A., Imran, M., et al.: A smart healthcare monitoring system for heart disease prediction based on ensemble deep learning and feature fusion. Inf. Fusion. 63, 208–222 (2020). https://doi.org/10.1016/j.inffus.2020.06.008
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows, pp. 10012–10022
Jiang, M.-x, Deng, C., Shan, J.-s, Wang, Y.-y, Jia, Y.-j, Sun, X.: Hierarchical multi-modal fusion FCN with attention model for RGB-D tracking. Inf. Fusion 50, 1–8 (2019). https://doi.org/10.1016/j.inffus.2018.09.014
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., et al.: Attention is all you need. Adv. Neural. Inf. Process. Syst. 30, 5998–6008 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Tuinstra, T., Gunraj, H., Wong, A.: COVIDx CT-3: a large-scale, multinational, open-source benchmark dataset for computer-aided COVID-19 screening from chest CT Images (2022)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Zhang, H., Wu, C., Zhang, Z., Zhu, Y., Lin, H., Zhang, Z., et al.: ResNest: split-attention networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2736–2746 (2022)
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., Xie, S.: A convnet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11976–11986 (2022)
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al.: An image is worth 16Ă—16 words: transformers for image recognition at scale. arXiv preprint https://arxiv.org/abs/2010.11929 (2020)
Acknowledgements
This work was supported partly by National Natural Science Foundation of China (Nos. U22A2024, U1902209 and 62271328), National Natural Science Foundation of Guangdong Province (Nos. 202020A1515110605, and 2022A1515012326), Shenzhen Science and Technology Program (Nos. JCYJ20220818095809021).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, Y., Liu, Y., Qu, J., Dong, X., Song, X., Lei, B. (2024). COVID-19 Diagnosis Based on Swin Transformer Model with Demographic Information Fusion and Enhanced Multi-head Attention Mechanism. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14349. Springer, Cham. https://doi.org/10.1007/978-3-031-45676-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-031-45676-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45675-6
Online ISBN: 978-3-031-45676-3
eBook Packages: Computer ScienceComputer Science (R0)
