Skip to main content
Springer Nature Link
Log in

An X-ray image classification method with fine-grained features for explainable diagnosis of pneumoconiosis

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Medical image classification has become popular in computer-aided diagnosis (CAD) of pneumoconiosis. However, most current work focuses on improving the accuracy of classification results and has overlooked the corresponding medical explanations. With the expectation to achieve these two sub-goals simultaneously, we propose an explainable X-ray image classification method with fine-grained features to diagnose pneumoconiosis. The proposed method consists of three consecutive stages. First, we generate a highlighted discriminative region by gradient-weighted class activation mapping (Grad-CAM) for each sample. Thus, we can give a visual explanation for the basis of classification. Then, we utilize selective convolutional descriptor aggregation (SCDA) to extract fine-grained features from the obtained discriminative region. After dimension reduction of obtained fine-grained features, we finally make a classification with these features to discover which samples are diseased. Extensive experiments on actual pneumoconiosis X-ray image datasets have shown the validity and superiority of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Availability of data and materials

The datasets generated during and/or analyzed during the current study are not publicly available due to personal privacy protection, but are available from the corresponding author on reasonable request.

References

  1. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35(8):1798–1828

    Article  Google Scholar 

  2. Zhang D, Yin J, Zhu X, Zhang C (2018) Network representation learning: a survey. IEEE transactions on Big Data 6(1):3–28

    Article  Google Scholar 

  3. Wang Z, Du B, Guo Y (2019) Domain adaptation with neural embedding matching. IEEE transactions on neural networks and learning systems 31(7):2387–2397

    Article  MathSciNet  Google Scholar 

  4. Arevalo J, González FA, Ramos-Pollán R, Oliveira JL, Lopez MAG (2016) Representation learning for mammography mass lesion classification with convolutional neural networks. Computer methods and programs in biomedicine 127:248–257

    Article  Google Scholar 

  5. Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., Greenspan, H.: Synthetic data augmentation using gan for improved liver lesion classification. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 289–293 (2018). IEEE

  6. Ranjan R, Patel VM, Chellappa R (2017) Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE transactions on pattern analysis and machine intelligence 41(1):121–135

    Article  Google Scholar 

  7. Payer C, Štern D, Bischof H, Urschler M (2019) Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Medical image analysis 54:207–219

    Article  Google Scholar 

  8. Zhang L, Rong R, Li Q, Yang DM, Yao B, Luo D, Zhang X, Zhu X, Luo J, Liu Y et al (2021) A deep learning-based model for screening and staging pneumoconiosis. Scientific reports 11(1):1–7

    Google Scholar 

  9. Morris ED, Ghanem AI, Dong M, Pantelic MV, Walker EM, Glide-Hurst CK (2020) Cardiac substructure segmentation with deep learning for improved cardiac sparing. Medical physics 47(2):576–586

    Article  Google Scholar 

  10. Haq R, Hotca A, Apte A, Rimner A, Deasy JO, Thor M (2020) Cardio-pulmonary substructure segmentation of radiotherapy computed tomography images using convolutional neural networks for clinical outcomes analysis. Physics and imaging in radiation oncology 14:61–66

    Article  Google Scholar 

  11. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural computation 1(4):541–551

    Article  Google Scholar 

  12. Pei Y, Huang Y, Zou Q, Zhang X, Wang S (2019) Effects of image degradation and degradation removal to cnn-based image classification. IEEE transactions on pattern analysis and machine intelligence 43(4):1239–1253

    Article  Google Scholar 

  13. Sun H, Zheng X, Lu X (2021) A supervised segmentation network for hyperspectral image classification. IEEE Transactions on Image Processing 30:2810–2825

    Article  Google Scholar 

  14. Hao C, Jin N, Qiu C, Ba K, Wang X, Zhang H, Zhao Q, Huang B (2021) Balanced convolutional neural networks for pneumoconiosis detection. International Journal of Environmental Research and Public Health 18(17):9091

    Article  Google Scholar 

  15. Zheng, R., Deng, K., Jin, H., Liu, H., Zhang, L.: An improved CNN-based pneumoconiosis diagnosis method on X-ray chest film. In: International Conference on Human Centered Computing, pp. 647–658 (2019). Springer

  16. Zheng R, Zhang L, Jin H (2021) Pneumoconiosis identification in chest X-ray films with CNN-based transfer learning. CCF Transactions on High Performance Computing 3(2):186–200

    Article  Google Scholar 

  17. Han, Y., Yang, X., Pu, T., Peng, Z.: Fine-grained recognition for oriented ship against complex scenes in optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing (2021)

  18. Zhang Y (2021) Computer-aided diagnosis for pneumoconiosis staging based on multi-scale feature mapping. International Journal of Computational Intelligence Systems 14(1):1–11

    Article  Google Scholar 

  19. Wang, D., Arzhaeva, Y., Devnath, L., Qiao, M., Amirgholipour, S., Liao, Q., McBean, R., Hillhouse, J., Luo, S., Meredith, D., et al.: Automated pneumoconiosis detection on chest x-rays using cascaded learning with real and synthetic radiographs. In: 2020 Digital Image Computing: Techniques and Applications (DICTA), pp. 1–6 (2020). IEEE

  20. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

  21. Jiang, H., Xu, J., Shi, R., Yang, K., Zhang, D., Gao, M., Ma, H., Qian, W.: A multi-label deep learning model with interpretable grad-cam for diabetic retinopathy classification. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 1560–1563 (2020). IEEE

  22. Basu, S., Mitra, S., Saha, N.: Deep learning for screening COVID-19 using chest X-ray images. In: 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 2521–2527 (2020). IEEE

  23. Moujahid, H., Cherradi, B., Al-Sarem, M., Bahatti, L., Eljialy, A.B.A.M.Y., Alsaeedi, A., Saeed, F.: Combining CNN and Grad-CAM for COVID-19 disease prediction and visual explanation. Intelligent Automation and Soft Computing, 723–745 (2022)

  24. Seerala, P.K., Krishnan, S.: Grad-cam-based classification of chest X-ray images of pneumonia patients. In: International Symposium on Signal Processing and Intelligent Recognition Systems, pp. 161–174 (2020). Springer

  25. Kim J-K, Jung S, Park J, Han SW (2022) Arrhythmia detection model using modified DenseNet for comprehensible Grad-CAM visualization. Biomedical Signal Processing and Control 73:103408

    Article  Google Scholar 

  26. Wei X-S, Luo J-H, Wu J, Zhou Z-H (2017) Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE Transactions on Image Processing 26(6):2868–2881

    Article  MathSciNet  Google Scholar 

  27. Zhang, N., Donahue, J., Girshick, R., Darrell, T.: Part-based R-CNNs for fine-grained category detection. In: European Conference on Computer Vision, pp. 834–849 (2014). Springer

  28. Zhang Y, Wei X-S, Wu J, Cai J, Lu J, Nguyen V-A, Do MN (2016) Weakly supervised fine-grained categorization with part-based image representation. IEEE Transactions on Image Processing 25(4):1713–1725

    Article  MathSciNet  Google Scholar 

  29. Lin, T.-Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1449–1457 (2015)

  30. Zhang, C., Yao, Y., Xu, X., Shao, J., Song, J., Li, Z., Tang, Z.: Extracting useful knowledge from noisy web images via data purification for fine-grained recognition. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 4063–4072 (2021)

  31. Xu, F., Wang, M., Zhang, W., Cheng, Y., Chu, W.: Discrimination-aware mechanism for fine-grained representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 813–822 (2021)

  32. Kruger RP, Thompson WB, Turner AF (1974) Computer diagnosis of pneumoconiosis. IEEE Transactions on Systems, Man, and Cybernetics 1:40–49

    Article  Google Scholar 

  33. Sundararajan R, Xu H, Annangi P, Tao X, Sun X, Mao L (2010) A multiresolution support vector machine based algorithm for pneumoconiosis detection from chest radiographs. In: 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, IEEE, pp 1317–1320

  34. Okumura E, Kawashita I, Ishida T (2017) Computerized classification of pneumoconiosis on digital chest radiography artificial neural network with three stages. Journal of digital imaging 30(4):413–426

    Article  Google Scholar 

  35. Yu P, Xu H, Zhu Y, Yang C, Sun X, Zhao J (2011) An automatic computer-aided detection scheme for pneumoconiosis on digital chest radiographs. Journal of digital imaging 24(3):382–393

    Article  Google Scholar 

  36. Zhu B, Luo W, Li B, Chen B, Yang Q, Xu Y, Wu X, Chen H, Zhang K (2014) The development and evaluation of a computerized diagnosis scheme for pneumoconiosis on digital chest radiographs. Biomedical engineering online 13(1):1–14

    Article  Google Scholar 

  37. Wang, Z., Hu, M., Zeng, M., Wang, G.: Intelligent image diagnosis of pneumoconiosis based on wavelet transform-derived texture features. Computational and Mathematical Methods in Medicine 2022 (2022)

  38. Devnath L, Luo S, Summons P, Wang D (2021) Automated detection of pneumoconiosis with multilevel deep features learned from chest x-ray radiographs. Computers in Biology and Medicine 129:104125

    Article  Google Scholar 

  39. Yang F, Tang Z-R, Chen J, Tang M, Wang S, Qi W, Yao C, Yu Y, Guo Y, Yu Z (2021) Pneumoconiosis computer aided diagnosis system based on X-rays and deep learning. BMC Medical Imaging 21(1):1–7

    Article  Google Scholar 

  40. Nguyen H, Huynh H, Tran T, Huynh H (202) Explanation of the convolutional neural network classifying chest X-ray images supporting pneumonia diagnosis. EAI Endorsed Transactions on Context-aware Systems and Applications 7(21)

  41. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2921–2929

  42. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4700–4708

  43. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. Advances in neural information processing systems 25:1097–1105

    Google Scholar 

  44. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  45. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 770–778

  46. Zhang L, Lim CP, Yu Y (2021) Intelligent human action recognition using an ensemble model of evolving deep networks with swarm-based optimization. Knowledge-Based Systems 220:106918

    Article  Google Scholar 

  47. Dinakaran, R., Zhang, L.: Object detection using deep convolutional generative adversarial networks embedded single shot detector with hyper-parameter optimization. In: 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–6 (2021). IEEE

  48. Lawrence T, Zhang L, Lim CP, Phillips E-J (2021) Particle swarm optimization for automatically evolving convolutional neural networks for image classification. IEEE Access 9:14369–14386

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Jia He and Lin Shang are authors contributed equally to this work.

Authors and Affiliations

  1. School of Artificial Intelligence, Nanjing Vocational College of Information Technology, 99 Wenlan Road, Qixia District, Nanjing, 210023, Jiangsu, People’s Republic of China

    Chunmei Zhang

  2. Department of Computer Science and Technology, Nanjing University, 163 Xianlin Road, Qixia District, Nanjing, 210023, Jiangsu, People’s Republic of China

    Jia He & Lin Shang

Authors
  1. Chunmei Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jia He
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lin Shang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Chunmei Zhang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., He, J. & Shang, L. An X-ray image classification method with fine-grained features for explainable diagnosis of pneumoconiosis. Pers Ubiquit Comput 28, 403–415 (2024). https://doi.org/10.1007/s00779-023-01730-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-023-01730-3

Keywords