Skip to main content
SpringerLink
Log in

COVID-19 Classification from Chest X-rays Based on Attention and Knowledge Distillation

  • Conference paper
  • First Online:
Intelligent Computing Theories and Application (ICIC 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13393))

Included in the following conference series:

  • 1529 Accesses

Abstract

The Coronavirus Disease 2019 (COVID-19) is the pandemic that has had the greatest impact on world economic development in recent years. Early detection is critical to identify patients with COVID-19, chest x-ray is used for early detection is a rapid, extensive and cost-effective method. The existing technology use deep learning methods, and have achieved very good results. However, the training time of deep learning method is long, and the model size makes it difficult to deploy on hardware system. In this work, we have proposed an attention-based ResNet50v2 network, and taken the network as the teacher network to transfer the knowledge to the student network by knowledge distillation. Thus, the student network has higher accuracy and sensitivity to the positive samples of COVID-19 under the condition of low model parameters, high training speed. The experimental results show that our network of teacher and student have achieved 100% accuracy and sensitivity in both COVID-19 and Normal binary classification. In addition, the accuracy rate of teacher network is 98.20%, the sensitivity is 99.58%, the accuracy rate of student network is 97.68%, the sensitivity is 99.17% in the COVID-19, Viral pneumonia and Normal multiple classification, and the parameters of the student network are only 0.269M.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Liu, X., Zhang, S.: COVID-19: face masks and human-to-human transmission. Influenza Other Respir. Viruses 14(4), 472 (2020)

    Article  Google Scholar 

  2. Organization, WHO: WHO Director-General’s opening remarks at the media briefing on COVID-19-11 March 2020, Geneva, Switzerland (2020).

    Google Scholar 

  3. Organization, WHO: COVID-19 weekly epidemiological update, edn. 84, 22 March 2022 (2022)

    Google Scholar 

  4. Wang, W., Xu, Y., Gao, R., Lu, R., Han, K., Wu, G., et al.: Detection of SARS-CoV-2 in different types of clinical specimens. JAMA 323(18), 1843–1844 (2020)

    Google Scholar 

  5. Yang, T., Wang, Y.-C., Shen, C.-F., Cheng, C.-M.: Point-of-care RNA-based diagnostic device for COVID-19, vol. 3, p. 165. Multidisciplinary Digital Publishing Institute (2020)

    Google Scholar 

  6. Rousan, L.A., Elobeid, E., Karrar, M., Khader, Y.: Chest X-ray findings and temporal lung changes in patients with COVID-19 pneumonia. BMC Pulm. Med. 20(1), 1–9 (2020)

    Article  Google Scholar 

  7. Cleverley, J., Piper, J., Jones, M.M.: The role of chest radiography in confirming covid-19 pneumonia. bmj 370 (2020)

    Google Scholar 

  8. Wang, L., Lin, Z.Q., Wong, A.: COVID-NET: a tailored deep convolutional neural network design for detection of covid-19 cases from chest X-ray images. Sci. Rep. 10(1), 1–12 (2020)

    Google Scholar 

  9. Shoeibi, A., Khodatars, M., Alizadehsani, R., Ghassemi, N., Jafari, M., Moridian, P., et al.: Automated detection and forecasting of covid-19 using deep learning techniques: A review. arXiv preprint arXiv:2007.10785 (2020)

  10. Pereira, R.M., Bertolini, D., Teixeira, L.O., Silla, C.N., Jr., Costa, Y.M.: COVID-19 identification in chest X-ray images on flat and hierarchical classification scenarios. Comput. Methods Programs Biomed. 194, 105532 (2020)

    Article  Google Scholar 

  11. Hammoudi, K., Benhabiles, H., Melkemi, M., Dornaika, F., Arganda-Carreras, I., Collard, D., et al.: Deep learning on chest X-ray images to detect and evaluate pneumonia cases at the era of COVID-19. J. Med. Syst. 45(7), 1–10 (2021). https://doi.org/10.1007/s10916-021-01745-4

    Article  Google Scholar 

  12. Wang, B., Chen, P., Zhang, J., Zhao, G., Zhang, X.: Inferring protein-protein interactions using a hybrid genetic algorithm/support vector machine method. Protein Pept. Lett. 17(9), 1079–1084 (2010)

    Article  Google Scholar 

  13. Wang, B., Valentine, S., Plasencia, M., Raghuraman, S., Zhang, X.: Artificial neural networks for the prediction of peptide drift time in ion mobility mass spectrometry. BMC Bioinf. 11(1), 1–11 (2010). https://doi.org/10.1186/1471-2105-11-182

    Article  Google Scholar 

  14. Wang, B., Fang, A., Shi, X., Kim, S.H., Zhang, X.: DISCO2: a comprehensive peak alignment algorithm for two-dimensional gas chromatography time-of-flight mass spectrometry. In: Huang, D.-S., Gan, Y., Premaratne, P., Han, K. (eds.) ICIC 2011. LNCS, vol. 6840, pp. 486–491. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-24553-4_64

    Chapter  Google Scholar 

  15. Chen, P., Hu, S., Wang, B., Zhang, J.: A random projection ensemble approach to drug-target interaction prediction. In: Huang, D.-S., Han, K. (eds.) ICIC 2015. LNCS (LNAI), vol. 9227, pp. 693–699. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22053-6_72

    Chapter  Google Scholar 

  16. Narin, A., Kaya, C., Pamuk, Z.: Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks. Pattern Anal. Appl. 24(3), 1207–1220 (2021). https://doi.org/10.1007/s10044-021-00984-y

    Article  Google Scholar 

  17. Jaiswal, A.K., Tiwari, P., Rathi, V.K., Qian, J., Pandey, H.M., Albuquerque, V.H.C.: COVIDPEN: A novel COVID-19 detection model using chest X-rays and CT scans. Medrxiv (2020)

    Google Scholar 

  18. Minaee, S., Kafieh, R., Sonka, M., Yazdani, S., Soufi, G.J.: Deep-COVID: predicting COVID-19 from chest X-ray images using deep transfer learning. Med. Image Anal. 65, 101794 (2020)

    Article  Google Scholar 

  19. Heidari, M., Mirniaharikandehei, S., Khuzani, A.Z., Danala, G., Qiu, Y., Zheng, B.: Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-ray images with preprocessing algorithms. Int. J. Med. Informatics 144, 104284 (2020)

    Article  Google Scholar 

  20. Hemdan, E.E.-D., Shouman, M.A., Karar, M.E.: COVIDX-Net: A framework of deep learning classifiers to diagnose covid-19 in X-ray images. ArXiv preprint arXiv:2003.11055 (2020)

  21. Afshar, P., Heidarian, S., Naderkhani, F., Oikonomou, A., Plataniotis, K.N., Mohammadi, A.: COVID-CAPS: a capsule network-based framework for identification of COVID-19 cases from X-ray images. Pattern Recogn. Lett. 138, 638–643 (2020)

    Article  Google Scholar 

  22. Chowdhury, M.E., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M.A., Mahbub, Z.B., et al.: Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 8, 132665–132676 (2020)

    Article  Google Scholar 

  23. Mahmud, T., Rahman, M.A., Fattah, S.A.: CovXNet: a multi-dilation convolutional neural network for automatic COVID-19 and other pneumonia detection from chest X-ray images with transferable multi-receptive feature optimization. Comput. Biol. Med. 122, 103869 (2020)

    Article  Google Scholar 

  24. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38

    Chapter  Google Scholar 

  25. Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1

    Chapter  Google Scholar 

  26. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012)

    Google Scholar 

  27. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv2(7):1503.02531 (2015)

  28. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)

    Google Scholar 

  29. Maghdid, H.S., Asaad, A.T., Ghafoor, K.Z., Sadiq, A.S., Mirjalili, S., Khan, M.K.: Diagnosing COVID-19 pneumonia from X-ray and CT images using deep learning and transfer learning algorithms’. In: Multimodal Image Exploitation and Learning 2021. International Society for Optics and Photonics, p. 117340E (2021)

    Google Scholar 

  30. Tangudu, V., Kakarla, J., Venkateswarlu, I.B.: COVID-19 detection from chest X-ray using MobileNet and residual separable convolution block. Soft Comput. 1–12 (2022)

    Google Scholar 

  31. Kaggle COVID-19 radiography database. https://www.kaggle.com/tawsifurrahman/covid19-radiography-database. Accessed 18 May 2022

  32. Hu, S., Chen, P., Zhang, J., Wang, B.: Prediction of hot spots based on physicochemical features and relative accessible surface area of amino acid sequence. In: Huang, D.-S., Bevilacqua, V., Premaratne, P. (eds.) ICIC 2016. LNCS, vol. 9771, pp. 422–431. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-42291-6_42

    Chapter  Google Scholar 

  33. Wang, B., Du, L., Zhang, J., Chen, P.: A hierarchical model for identifying mild cognitive impairment. In: 2015 11th International Conference on Natural Computation (ICNC), pp. 599–604. IEEE (2015)

    Google Scholar 

Download references

Acknowledgment

This work was supported by the National Natural Science Foundation of China (Nos. 62172004, 62072002, and 61872004), Educational Commission of Anhui Province (No. KJ2019ZD05).

Author information

Authors and Affiliations

  1. School of Electrical and Information Engineering, Anhui University of Technology, Ma’anshan, 243032, Anhui, China

    Jiaxing Lv, Fazhan Zhu, Kun Lu, Wenyan Wang, Yuan Zhao & Ziheng Wu

  2. Ministry of Education, Key Laboratory of Metallurgical Emission Reduction and Resources Recycling (Anhui University of Technology), Ma’anshan, 243002, China

    Kun Lu, Wenyan Wang, Yuan Zhao & Ziheng Wu

  3. School of Materials Science and Engineering, Anhui University of Technology, Ma’anshan, 243032, Anhui, China

    Wenyan Wang

  4. School of Electrical Engineering and Automation, Anhui University, Hefei, 230601, Anhui, China

    Jun Zhang

  5. National Engineering Research Center for Agro-Ecological Big Data Analysis and Application, School of Internet and Institutes of Physical Science and Information Technology, Anhui University, Hefei, 230601, Anhui, China

    Peng Chen

Authors
  1. Jiaxing Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fazhan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenyan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Peng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yuan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Ziheng Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ziheng Wu .

Editor information

Editors and Affiliations

  1. Tongji University, Shanghai, China

    De-Shuang Huang

  2. University of Ulsan, Ulsan, Korea (Republic of)

    Kang-Hyun Jo

  3. Xi'an Polytechnic University, Xi'an, China

    Junfeng Jing

  4. The University of Wollongong, North Wollongong, NSW, Australia

    Prashan Premaratne

  5. Polytecnic of Bari, Bari, Italy

    Vitoantonio Bevilacqua

  6. Liverpool John Moores University, Liverpool, UK

    Abir Hussain

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lv, J. et al. (2022). COVID-19 Classification from Chest X-rays Based on Attention and Knowledge Distillation. In: Huang, DS., Jo, KH., Jing, J., Premaratne, P., Bevilacqua, V., Hussain, A. (eds) Intelligent Computing Theories and Application. ICIC 2022. Lecture Notes in Computer Science, vol 13393. Springer, Cham. https://doi.org/10.1007/978-3-031-13870-6_64

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-13870-6_64

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-13869-0

  • Online ISBN: 978-3-031-13870-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation