Abstract
Chest radiography remain the global standard for diagnosing pulmonary diseases. Despite numerous research efforts, medical professionals still face challenges in rapidly and accurately analyzing multiple diseases on a single chest radiography. Moreover, traditional deep learning methods suffer from complexities in design and prolonged processing times. To address these issues, we propose a multi-label neural architecture search (MLNAS) approach. Primarily intended for multi-label chest radiography image classification, MLNAS employs automated modeling, data augmentation, and threshold calculation strategies to improve the accuracy of chest radiography image classification and enhance result interpretation. Furthermore, MLNAS demonstrates the potential for application in other multi-label medical image classification domains. Experimental results indicate that MLNAS achieves state-of-the-art prediction accuracy for 9 out of 14 lung diseases. This novel approach presents a new solution for computer-aided diagnosis of chest X-rays.













Similar content being viewed by others
Data availibility
The data and material that has been used in this article is cited with references at their respective place.
References
Ahmadi, Z., Kramer, S.: A label compression method for online multi-label classification. Pattern Recogn. Lett. 111, 64–71 (2018). https://doi.org/10.1016/j.patrec.2018.04.015
Annarumma, M., Withey, S.J., Bakewell, R.J., et al.: Automated triaging of adult chest radiographs with deep artificial neural networks. Radiology 291(1), 196–202 (2019). https://doi.org/10.1148/radiol.2018180921
Baemani, M.J., Monadjemi, A., Moallem, P.: Detection of respiratory abnormalities using artificial neural networks. J. Comput. Sci. 4(8), 663 (2008)
Baker, B., Gupta, O., Naik, N., et al.: Designing neural network architectures using reinforcement learning (2016). arXiv preprint arXiv:1611.02167
Boutell, M.R., Luo, J., Shen, X., et al.: Learning multi-label scene classification. Pattern Recogn. 37(9), 1757–1771 (2004). https://doi.org/10.1016/j.patcog.2004.03.009
Charte, F., Rivera, A.J., del Jesus, M.J., et al.: Mlsmote: approaching imbalanced multilabel learning through synthetic instance generation. Knowl.-Based Syst. 89, 385–397 (2015). https://doi.org/10.1016/j.knosys.2015.07.019
Chawla, N.V., Bowyer, K.W., Hall, L.O., et al.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002). https://doi.org/10.1613/jair.953
Chen, X., Xie, L., Wu, J., et al.: Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1294–1303 (2019a)
Chen, ZM., Wei, XS., Wang, P., et al.: Multi-label image recognition with graph convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019b)
Dey, R., Lu, Z., Hong, Y.: Diagnostic classification of lung nodules using 3d neural networks. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 774–778 (2018). https://doi.org/10.1109/ISBI.2018.8363687
Elisseeff, A., Weston, J.: A kernel method for multi-labelled classification. Adv. Neural Inf. Process. Syst. 14 (2001)
Gao, M., Bagci, U., Lu, L., et al.: Holistic classification of ct attenuation patterns for interstitial lung diseases via deep convolutional neural networks. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 6(1), 1–6 (2018). https://doi.org/10.1080/21681163.2015.1124249
Grauman, K., Darrell, T.: The pyramid match kernel: Discriminative classification with sets of image features. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) vol. 12, pp. 1458–1465 (2005)
Guan, Q., Huang, Y.: Multi-label chest x-ray image classification via category-wise residual attention learning. Pattern Recogn. Lett. 130, 259–266 (2020). https://doi.org/10.1016/j.patrec.2018.10.027
He, Y., Yang, D., Roth, H., et al.: Dints: Differentiable neural network topology search for 3d medical image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5841–5850 (2021)
Hermoza, R., Maicas, G., Nascimento, J.C., et al.: Region proposals for saliency map refinement for weakly-supervised disease localisation and classification. Med. Image Comput. Comput. Assist. Interv. MICCAI 2020, 539–549 (2020). https://doi.org/10.1007/978-3-030-59725-2_52
Hua, Y., Mou, L., Zhu, X.X.: Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional lstm network for multi-label aerial image classification. ISPRS J. Photogramm. Remote. Sens. 149, 188–199 (2019). https://doi.org/10.1016/j.isprsjprs.2019.01.015
Iandola, F., Moskewicz, M., Karayev, S., et al.: Densenet: implementing efficient convnet descriptor pyramids (2014). arXiv preprint arXiv:1404.1869
Jin, J., Nakayama, H.: Annotation order matters: Recurrent image annotator for arbitrary length image tagging. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 2452–2457 (2016)
Jozefowicz, R., Zaremba, W., Sutskever, I.: An empirical exploration of recurrent network architectures. In: International Conference on Machine Learning, pp. 2342–2350 (2015)
Kelly, B.: The chest radiograph. Ulst. Med. J. 81(3), 143 (2012)
Khobragade, S., Tiwari, A., Patil, C., et al.: Automatic detection of major lung diseases using chest radiographs and classification by feed-forward artificial neural network. In: 2016 IEEE 1st International Conference on Power Electronics, Intelligent Control and Energy Systems (ICPEICES), IEEE, pp. 1–5 (2016)
Kim, E., Kim, S., Seo, M., et al.: Xprotonet: diagnosis in chest radiography with global and local explanations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15719–15728 (2021)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural Inf. Process. Syst. (2012). https://doi.org/10.1145/3065386
Li, Z., Wang, C., Han, M., et al.: Thoracic disease identification and localization with limited supervision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8290–8299 (2018a)
Li, Z., Wang, C., Han, M., et al.: Thoracic disease identification and localization with limited supervision. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8290–8299 (2018b)
Li, L., Cao, P., Yang, J., et al.: Modeling global and local label correlation with graph convolutional networks for multi-label chest x-ray image classification. Med. Biol. Eng. Comput. 60(9), 2567–2588 (2022). https://doi.org/10.1007/s11517-022-02604-1
Liu, C., Chen, LC., Schroff, F., et al.: Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 82–92 (2019a)
Liu, H., Simonyan, K., Yang, Y.: DARTS: Differentiable architecture search. In: International Conference on Learning Representations (2019b)
Liu, J., Zhao, G., Fei, Y., et al.: Align, attend and locate: Chest x-ray diagnosis via contrast induced attention network with limited supervision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10632–10641 (2019c)
Loza Mencia, E., Furnkranz, J .: Pairwise learning of multilabel classifications with perceptrons. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 2899–2906 (2008). https://doi.org/10.1109/IJCNN.2008.4634206
Lyu, F., Wu, Q., Hu, F., et al.: Attend and imagine: multi-label image classification with visual attention and recurrent neural networks. IEEE Trans. Multimedia 21(8), 1971–1981 (2019). https://doi.org/10.1109/TMM.2019.2894964
Ma, C., Wang, H., Hoi, SC.: Multi-label thoracic disease image classification with cross-attention networks. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 730–738 (2019a)
Ma, C., Wang, H., Hoi, S.C.H.: Multi-label thoracic disease image classification with cross-attention networks. Med. Image Comput. Comput. Assist. Interv. MICCAI 2019, 730–738 (2019). https://doi.org/10.1007/978-3-030-32226-7_81
Miranda, E., Aryuni, M., Irwansyah, E.: A survey of medical image classification techniques. In: 2016 International Conference on Information Management and Technology (ICIMTech), pp. 56–61 (2016). https://doi.org/10.1109/ICIMTech.2016.7930302
Oliveira, H., dos Santos, J.: Deep transfer learning for segmentation of anatomical structures in chest radiographs. In: 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 204–211 (2018). https://doi.org/10.1109/SIBGRAPI.2018.00033
O’Neill, D., Xue, B., Zhang, M.: Evolutionary neural architecture search for high-dimensional skip-connection structures on densenet style networks. IEEE Trans. Evol. Comput. 25(6), 1118–1132 (2021). https://doi.org/10.1109/TEVC.2021.3083315
Pauletto, L., Amini, M.R., Babbar, R., et al.: Neural Architecture Search for extreme multi-label classification: an evolutionary approach. In: The Fourth International Workshop on Automation in Machine Learning (AutoML 2020) (2020)
Pham, H., Guan, M., Zoph, B., et al.: Efficient neural architecture search via parameters sharing. In: International Conference on Machine Learning, pp. 4095–4104 (2018)
Real, E., Moore, S., Selle, A., et al.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning, pp. 2902–2911 (2017)
Real, E., Aggarwal, A., Huang, Y., et al.: Regularized evolution for image classifier architecture search. Proc. AAAI Conf. Artif. Intell. 33(01), 4780–4789 (2019). https://doi.org/10.1609/aaai.v33i01.33014780
Ruuskanen, O., Lahti, E., Jennings, L.C., et al.: Viral pneumonia. Lancet 377(9773), 1264–1275 (2011). https://doi.org/10.1016/S0140-6736(10)61459-6
Shen, Y., Gao, M.: Dynamic routing on deep neural network for thoracic disease classification and sensitive area localization. In: Machine Learning in Medical Imaging, pp. 389–397 (2018). https://doi.org/10.1007/978-3-030-00919-9_45
Sim, Y., Chung, M.J., Kotter, E., et al.: Deep convolutional neural network-based software improves radiologist detection of malignant lung nodules on chest radiographs. Radiology 294(1), 199–209 (2020). https://doi.org/10.1148/radiol.2019182465
Tai, Y.: A deep learning based workflow for detection of lung nodules with chest radiograph (2021). arXiv preprint arXiv:2112.10184
Tang, Y., Wang, X., Harrison, A.P., et al.: Attention-guided curriculum learning for weakly supervised classification and localization of thoracic diseases on chest radiographs. In: Shi, Y., Suk, H.I., Liu, M. (eds.) Machine Learning in Medical Imaging, pp. 249–258. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_29
Tarekegn, A.N., Giacobini, M., Michalak, K.: A review of methods for imbalanced multi-label classification. Pattern Recogn. 118, 107965 (2021). https://doi.org/10.1016/j.patcog.2021.107965
Targ, S., Almeida, D., Lyman, K.: Resnet in resnet: generalizing residual architectures (2016). arXiv preprint arXiv:1603.08029
Tsoumakas, G., Vlahavas, I.: Random k-labelsets: an ensemble method for multilabel classification. Mach. Learn. ECML 2007, 406–417 (2007)
Wang, J., Yang, Y., Mao, J., et al.: Cnn-rnn: a unified framework for multi-label image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Wang, X., Peng, Y., Lu, L., et al.: Chestx-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)
Wang, W., Liang, D., Chen, Q., et al.: Medical image classification using deep learning. Healthc. Paradigms Appl, Deep Learn (2020). https://doi.org/10.1007/978-3-030-32606-7_3
Xie, L., Yuille, A.: Genetic cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1379–1388 (2017)
Xu, Y., Xie, L., Zhang, X., et al.: Pc-darts: Partial channel connections for memory-efficient architecture search. In: International Conference on Learning Representations (2020)
Yang, Z., Wang, Y., Chen, X., et al.: Cars: continuous evolution for efficient neural architecture search. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)
Yang, Y., Wei, J., Yu, Z., et al.: A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology. J. Supercomput. (2023). https://doi.org/10.1007/s11227-023-05541-4
Yao, L., Prosky, J., Poblenz, E., et al.: Weakly supervised medical diagnosis and localization from multiple resolutions (2018). arXiv preprint arXiv:1803.07703
Zhang, ML., Zhang, K.: Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining—KDD ’10. Association for Computing Machinery, New York, NY, USA, KDD ’10, pp. 999–1008 (2010). https://doi.org/10.1145/1835804.1835930
Zhang, M.L., Zhou, Z.H.: A k-nearest neighbor based algorithm for multi-label classification. In: 2005 IEEE International Conference on Granular Computing, vol. 2, pp. 718–721 (2005). https://doi.org/10.1109/GRC.2005.1547385
Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26(8), 1819–1837 (2014). https://doi.org/10.1109/TKDE.2013.39
Zhang, X., Hou, P., Zhang, X., et al.: Neural architecture search with random labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10907–10916 (2021)
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning (2016). arXiv preprint arXiv:1611.01578
Zoph, B., Vasudevan, V., Shlens, J., et al.: Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)
Funding
This work was supported by the Natural Science Foundation of Gansu Province, Grant number no. 22JR11RA042. We would like to thank the Editor-in-Chief for handling our paper. We would also like to thank the reviewers for their time and effort in providing constructive suggestions for our paper.
Author information
Authors and Affiliations
Contributions
Yi Yang: development or design of methodology, designing computer programs, implementation of the computer code and supporting algorithms and preparation, creation and presentation of the published work, specifically writing the initial draft. Jiaxuan Wei and Zhixuan Yu: preparation, creation and presentation of the published work by those from the original research group, specifically critical review, commentary or revision. Ruisheng Zhang: oversight and leadership responsibility for the research activity planning and execution, including mentorship external to the core team.
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
This work does not involve any work related to ethics.
Consent to publish
All authors consent to publication.
Additional information
Communicated by F. Wu.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Yang, Y., Wei, J., Yu, Z. et al. Multi-label neural architecture search for chest radiography image classification. Multimedia Systems 30, 8 (2024). https://doi.org/10.1007/s00530-023-01215-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00530-023-01215-6