Skip to main content
SpringerLink
Log in

Modeling global and local label correlation with graph convolutional networks for multi-label chest X-ray image classification

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

The diagnosis of chest diseases is a challenging task for assessing thousands of radiology subjects. Their diagnosis decisions heavily rely on the expert radiologists’ manual annotations. It is important to develop automated analysis methods for the computer-aided diagnosis of chest diseases on chest radiography. To explore the label relationship and improve the diagnosis performance, we present an end-to-end multi-label learning framework for jointly modeling the global and local label correlation, called GL-MLL that (1) explores the label correlation from a globally static view and a locally adaptive view, (2) considers the imbalanced class distribution, and (3) focuses on capturing label-specific features in image-level representation. We validate the performance of the proposed framework on the CheXpert dataset. The results demonstrate that the proposed GL-MLL outperforms state-of-the-art approaches. The code is available at https://github.com/llt1836/GL-MLL.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and materials

The CheXpert dataset used in this study is freely available and can be downloaded from the corresponding project website: https://stanfordmlgroup.github.io/competitions/chexpert. The code for the ML-GCN is available at https://github.com/llt1836/GL-MLL.

References

  1. Kieu STH, Bade A, Hijazi MHA, Kolivand H (2020) A survey of deep learning for lung disease detection on medical images: state-of-the-art, taxonomy, issues and future directions. J Imag 6(12):131

    Article  Google Scholar 

  2. Huang S, Lee F, Miao R, Si Q, Lu C, Chen Q (2020) A deep convolutional neural network architecture for interstitial lung disease pattern classification. Med Biol Eng Comput:1–13

  3. Luo L, Yu L, Chen H, Liu Q, Wang X, Xu J, Heng PA (2020) Deep mining external imperfect data for chest x-ray disease screening. IEEE Trans Med Imag 39(11):3583–3594

    Article  Google Scholar 

  4. Firmino M, Morais AH, Mendoça RM, Dantas MR, Hekis HR, Valentim R (2014) Computer-aided detection system for lung cancer in computed tomography scans: review and future prospects. Biomed Eng Online 13(1):1–16

    Article  Google Scholar 

  5. Zhang W, Wang X, Li X, Chen J (2018) 3d skeletonization feature based computer-aided detection system for pulmonary nodules in ct datasets. Comput Biol Med 92:64–72

    Article  Google Scholar 

  6. Wang J, Yang Y, Mao J, Huang Z, Huang C, Xu W (2016) CNN-RNN: a unified framework for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition. p 2285–2294

  7. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  8. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  Google Scholar 

  9. Tsoumakas G, Katakis I, Vlahavas I (2009) Mining multi-label data. In: Data mining and knowledge discovery handbook. Springer, Berlin, p 667–685

  10. Zhang ML, Zhou ZH (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  11. Zhang ML, Zhang K (2010) Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. p 999–1008

  12. Wosiak A, Glinka K, Zakrzewska D (2017) Multi-label classification methods for improving comorbidities identification. Comput Biol Med: 100

  13. Chen ZM, Wei XS, Wang P, Guo Y (2019) Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. p 5177–5186

  14. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907

  15. Hao JA, Peng C, Myx A, Jy A, Oz C (2020) Hi-GCN: a hierarchical graph convolution network for graph embedding learning of brain network and brain disorders prediction. Comput Biol Med:127

  16. Mazurowski MA, Habas PA, Zurada JM, Lo JY, Baker JA, Tourassi GD (2008) Training neural network classifiers for medical decision making: the effects of imbalanced datasets on classification performance. Neural Netw 21(2–3):427–436

    Article  Google Scholar 

  17. Zhang J, Xie Y, Pang G, Liao Z, Verjans J, Li W, Sun Z, He J, Li Y, Shen C, etal. (2020) Viral pneumonia screening on chest x-ray images using confidence-aware anomaly detection. arXiv:2003.12338

  18. Ke A, Ellsworth W, Banerjee O, Ng AY, Rajpurkar P (2021) Chextransfer: performance and parameter efficiency of imagenet models for chest x-ray interpretation. In: Proceedings of the conference on health, inference, and learning, pp 116–124

  19. Irvin J, Rajpurkar P, Ko M, Yu Y, Ciurea-Ilcus S, Chute C, Marklund H, Haghgoo B, Ball R, Shpanskaya K et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. Proc AAAI Conf Artif Intell 33:590–597

    Article  Google Scholar 

  20. Bressem KK, Adams LC, Erxleben C, Hamm B, Niehues SM, Vahldiek JL (2020) Comparing different deep learning architectures for classification of chest radiographs. Sci Rep 10(1):1–16

    Article  Google Scholar 

  21. Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K, etal. (2017) CheXNet: radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv:1711.05225

  22. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) ChestX-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2097–2106

  23. Wang H, Jia H, Lu L, Xia Y (2019) Thorax-Net: an attention regularized deep neural network for classification of thoracic diseases on chest radiography. IEEE J Biomed Health Inform 24(2):475–485

    Article  Google Scholar 

  24. Guan Q, Huang Y, Zhong Z, Zheng Z, Zheng L, Yang Y (2018) Diagnose like a radiologist: attention guided convolutional neural network for thorax disease classification. arXiv:1801.09927

  25. Yan R, Tesic J, Smith JR (2007) Model-shared subspace boosting for multi-label classification. In: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 834–843

  26. Ji S, Tang L, Yu S, Ye J (2008) Extracting shared subspace for multi-label classification. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 381–389

  27. Cheng W, Hüllermeier E (2009) Combining instance-based learning and logistic regression for multilabel classification. Mach Learn 76(2–3):211–225

    Article  Google Scholar 

  28. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. In: European conference on machine learning, Springer, pp 406–417

  29. Yang YY, Lin YA, Chu HM, Lin HT (2019) Deep learning with a rethinking structure for multi-label classification. In: Asian conference on machine learning, PMLR, pp 125–140

  30. Chen SF, Chen YC, Yeh CK, Wang YCF (2018) Order-free rnn with visual attention for multi-label classification. In: Thirty-second AAAI conference on artificial intelligence

  31. Zhu F, Li H, Ouyang W, Yu N, Wang X (2017) Learning spatial regularization with image-level supervisions for multi-label image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5513–5522

  32. Chen B, Li J, Lu G, Yu H, Zhang D (2020) Label co-occurrence learning with graph convolutional networks for multi-label chest x-ray image classification. IEEE J Biomed Health Inform 24(8):2292–2302

    Article  Google Scholar 

  33. Maas AL, Hannun AY, Ng AY, etal. (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proc. icml, Citeseer, vol 30, p 3

  34. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

  35. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  36. Yan Y, Guo Y (2020) Partial label learning with batch label correction. Proc AAAI Conf Artif Intell 34:6575–6582

    Article  Google Scholar 

  37. Geng X (2016) Label distribution learning. IEEE Trans Knowl Data Eng 28(7):1734–1748

    Article  Google Scholar 

  38. Ye J, He J, Peng X, Wu W, Qiao Y (2020) Attention-driven dynamic graph convolutional network for multi-label image recognition. In: European conference on computer vision, Springer, pp 649–665

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (No.62076059) and the Science Project of Liaoning Province (2021-MS-105).

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Northeastern University, Shenyang, China

    Lanting Li, Peng Cao & Jinzhu Yang

  2. Key Laboratory of Intelligent Computing in Medical Image of Ministry of Education, Northeastern University, Shenyang, China

    Lanting Li, Peng Cao & Jinzhu Yang

  3. Alberta Machine Intelligence Institute, University of Alberta, Edmonton, Alberta, Canada

    Osmar R. Zaiane

Authors
  1. Lanting Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peng Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinzhu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Osmar R. Zaiane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Cao.

Ethics declarations

This article does not contain any studies with human participants or animals performed by any of the authors.

Conflict of interest

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Cao, P., Yang, J. et al. Modeling global and local label correlation with graph convolutional networks for multi-label chest X-ray image classification. Med Biol Eng Comput 60, 2567–2588 (2022). https://doi.org/10.1007/s11517-022-02604-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02604-1

Keywords

Search

Navigation