Skip to main content

Advertisement

SpringerLink
Log in

Contour-aware multi-label chest X-ray organ segmentation

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Segmentation of organs from chest X-ray images is an essential task for an accurate and reliable diagnosis of lung diseases and chest organ morphometry. In this study, we investigated the benefits of augmenting state-of-the-art deep convolutional neural networks (CNNs) for image segmentation with organ contour information and evaluated the performance of such augmentation on segmentation of lung fields, heart, and clavicles from chest X-ray images.

Methods

Three state-of-the-art CNNs were augmented, namely the UNet and LinkNet architecture with the ResNeXt feature extraction backbone, and the Tiramisu architecture with the DenseNet. All CNN architectures were trained on ground-truth segmentation masks and additionally on the corresponding contours. The contribution of such contour-based augmentation was evaluated against the contour-free architectures, and 20 existing algorithms for lung field segmentation.

Results

The proposed contour-aware segmentation improved the segmentation performance, and when compared against existing algorithms on the same publicly available database of 247 chest X-ray images, the UNet architecture with the ResNeXt50 encoder combined with the contour-aware approach resulted in the best overall segmentation performance, achieving a Jaccard overlap coefficient of 0.971, 0.933, and 0.903 for the lung fields, heart, and clavicles, respectively.

Conclusion

In this study, we proposed to augment CNN architectures for CXR segmentation with organ contour information and were able to significantly improve segmentation accuracy and outperform all existing solution using a public chest X-ray database.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Chen S, Zhong S, Yao L, Shang Y, Suzuki K (2016) Enhancement of chest radiographs obtained in the intensive care unit through bone suppression and consistent processing. Phys Med Biol 61:2283

    Article  Google Scholar 

  2. Candemir S, Antani S (2019) A review on lung boundary detection in chest X-rays. Int J Comput Assist Radiol Surg 14:563–576

    Article  Google Scholar 

  3. Miniati M, Coppini G, Monti S, Bottai M, Paterni M, Ferdeghini E (2011) Computer-aided recognition of emphysema on digital chest radiography. Eur J Radiol 80:169–175

    Article  Google Scholar 

  4. Candemir S, Jaeger S, Lin W, Xue Z, Antani S, Thoma G (2016) Automatic heart localization and radiographic index computation in chest X-rays. In: SPIE medical imaging, 9785

  5. Finnegan R, Dowling J, Koh E-S, Tang S, Otton J, Delaney G, Batumalai V, Luo C, Atluri P, Satchithanandha A (2019) Feasibility of multi-atlas cardiac segmentation from thoracic planning CT in a probabilistic framework. Phys Med Biol 64:085006

    Article  Google Scholar 

  6. Gordienko Y, Gang P, Hui J, Zeng W, Kochura Y, Alienin O, Rokovyi O, Stirenko S (2018) Deep learning with lung segmentation and bone shadow exclusion techniques for chest x-ray analysis of lung cancer. In: International conference on ICAFS, pp 638–647

    Google Scholar 

  7. Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR Am J Roentgenol 174:71–4

    Article  CAS  Google Scholar 

  8. van Ginneken B, Stegmann MB, Loog M (2006) Segmentation of anatomical structures in chest radiographs using supervised methods: a comparative study on a public database. Med Image Anal 10:19–40

    Article  Google Scholar 

  9. Vittitoe NF, Vargas-Voracek R, Floyd CF Jr (1998) Identification of lung regions in chest radiographs using Markov random field modeling. Med Phys 25:976–85

    Article  CAS  Google Scholar 

  10. Shi Z, Zhou P, He L, Nakamura T, Yao Q, Itoh H (2009) Lung segmentation in chest radiographs by means of Gaussian kernel-based FCM with spatial constraints. In: 6th international conference on fuzzy systems and knowledge discovery (FSKD). IEEE, pp 428–432

  11. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Computer vision and pattern recognition (CVPR). IEEE, pp 248–55

  12. Wang C (2017) Segmentation of multiple structures in chest radiographs using multi-task fully convolutional networks. In: Scandinavian conference on image analysis (SCIA). Volume 10270 of Lecture notes in computer science. Springer, Cham, pp 282–289

    Chapter  Google Scholar 

  13. Hwang S, Park S (2017) Accurate lung segmentation via network-wise training of convolutional networks. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, vol 10553 lecture notes in computer science. Springer, Cham, pp 92–99

    Chapter  Google Scholar 

  14. Dai W, Dong N, Wang Z, Liang X, Zhang H, Xing EP (2018) SCAN: structure correcting adversarial network for organ segmentation in chest X-rays. In: Deep learning in medical image analysis and multimodal learning for clinical decision support, volume 11045 of lecture notes in computer science. Springer, Berlin, pp 263–73

    Chapter  Google Scholar 

  15. Bi L, Feng D, Kim J (2018) Dual-path adversarial learning for fully convolutional network (FCN)-based medical image segmentation. Vis Comput 34:1043–52

    Article  Google Scholar 

  16. Novikov AA, Lenis D, Major D, Hladůvka J, Wimmer M, Bühler K (2018) Fully convolutional architectures for multi-class segmentation in chest radiographs. IEEE Trans Med Imaging 37:1865–76

    Article  Google Scholar 

  17. Mittal A, Hooda R, Sofat S (2018) LF-SegNet: a fully convolutional encoder-decoder network for segmenting lung fields from chest radiographs. Wirel Pers Commun 101:511–29

    Article  Google Scholar 

  18. Frid-Adar M, Ben-Cohen A, Amer R, Greenspan H (2018) Improving the segmentation of anatomical structures in chest radiographs using U-Net with an ImageNet pre-trained encoder. In: Image analysis for moving organ. breast, and thoracic images, volume 11040 of lecture notes in computer science. Springer, Cham, pp 159–68

    Chapter  Google Scholar 

  19. Bonheur S, Stern D, Payer C, Pienn M, Olschewski H, Urschler M (2019) Matwo-capsnet: a multi-label semantic segmentation capsules network. MICCAI 2019:664–672

    Google Scholar 

  20. Ngo T A, Carneiro G (205) Lung segmentation in chest radiographs using distance regularized level set and deep-structured learning and inference. In: International conference on image processing (ICIP). IEEE, pp 2140–2143

  21. Kumar N, Verma R, Sharma S, Bhargava S, Vahadane A, Sethi A (2017) A dataset and a technique for generalized nuclear segmentation for computational pathology. IEEE Trans Med Imaging 36:1550–60

    Article  Google Scholar 

  22. Cui Y, Zhang G, Liu Z, Xiong Z, Hu J (2018) A deep learning algorithm for one-step contour aware nuclei segmentation of histopathological images. arXiv, p 1803.02786

  23. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention (MICCAI), volume 9351 of lecture notes in computer science. Springer, Cham, pp 234–241

    Google Scholar 

  24. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 5987–5995

  25. Chaurasia A, Culurciello E (2017) LinkNet: exploiting encoder representations for efficient semantic segmentation. In: Visual communications and image processing (VCIP). IEEE

  26. Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: fully convolutional DenseNets for semantic segmentation. In: Computer vision and pattern recognition workshops (CVPRW). IEEE, pp 1175–1783

  27. Huang G, Liu Z, van der Maaten L, Weinberger K Q (2017) Densely connected convolutional networks. In: Computer vision and pattern recognition (CVPR). IEEE, pp 2261–2269

  28. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Computer vision and pattern recognition (CVPR). IEEE, pp 3431–3440

  29. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) UNet++: a nested U-Net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support (DLMIA / ML-CDS), volume 11045 of lecture notes in computer science. Springer, Cham, pp 3–11

    Chapter  Google Scholar 

  30. Wang CW, Huang CT, Lee JH, Li CH, Chang SW, Siao MJ, Lai TM, Ibragimov B, Vrtovec T, Ronneberger O, Fischer P, Cootes TF, Lindner C (2016) A benchmark for comparison of dental radiography analysis algorithms. Med Image Anal 31:63–76

    Article  CAS  Google Scholar 

  31. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Computer vision and pattern recognition (CVPR). IEEE, pp 770–788

  32. Shaikh M, Anand G, Acharya G, Amrutkar A, Varghese A, Krishnamurthi G (2018) Brain tumor segmentation using dense fully convolutional neural network. Brainlesion: Glioma. In: Multiple sclerosis, stroke and traumatic brain injuries (BrainLes), volume 10670 of lecture notes in computer Science. Springer, Cham, pp 309–319

    Chapter  Google Scholar 

  33. Xu C, Xu L, Brahm G, Zhang, Li S (2018) Mutgan: simultaneous segmentation and quantification of myocardial infarction without contrast agents via joint adversarial learning. In: Conference on Medical Image Computing and Computer-Assisted Intervention, pp 525–534, 2018

    Chapter  Google Scholar 

  34. Arik SO, Ibragimov B, Xing L (2017) Fully automated quantitative cephalometry using convolutional neural networks. J Med Imaging 4:014501

    Article  Google Scholar 

  35. Milletari F, Navab N, Ahmadi S-A (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 3D vision (3DV). IEEE, pp 565–571

  36. DSBowl2018 (2018) 2018 data science bowl. https://www.kaggle.com/c/data-science-bowl-2018/overview. Accessed 15 Oct 2019

  37. KiTS2019 (2019) 2019 kidney and kidney tumor segmentation challenge. https://kits19.grand-challenge.org/. Accessed 15 Oct 2019

  38. Isensee F, Petersen J, Kohl S A A, Jager PF, Maier-Hein KH (2019) Nnu-net: breaking the spell on successful medical image segmentation. arXiv:1904.08128

  39. Kingma DP, Ba LJ (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations (ICLR)

  40. Shi Y, Qi F, Xue Z, Chen L, Ito K, Matsuo H, Shen D (2008) Segmenting lung fields in serial chest radiographs using both population-based and patient-specific shape statistics. IEEE Trans Med Imaging 27:481–94

    Article  CAS  Google Scholar 

  41. Li X, Luo S, Hu Q, Li J, Wang D, Chiong F (2016) Automatic lung field segmentation in X-ray radiographs using statistical shape and appearance models. J Med Imaging Health Inform 6:338–48

    Article  Google Scholar 

  42. Dawoud A (2010) Fusing shape information in lung segmentation in chest radiographs. In: Image analysis and recognition—ICIAR 2010, volume 6112 of lecture notes in computer science. Springer, Berlin, Heidelberg, pp 70–78

    Google Scholar 

  43. Candemir S, Jaeger S, Palaniappan K, Musco JP, Singh RK, Zhiyun X, Karargyris A, Antani S, Thoma G, McDonald CJ (2014) Lung segmentation in chest radiographs using anatomical atlases with nonrigid registration. IEEE Trans Med Imaging 33:577–90

    Article  Google Scholar 

  44. Ibragimov B, Likar B, Pernuš F, Vrtovec T (2012) A game-theorteic framework for landmark-based image segmentation. IEEE Trans. Med. Imaging 31:1761–76

    Article  Google Scholar 

  45. Seghers D, Loeckx D, Maes F, Vandermeulen D, Suetens P (2007) Minimal shape and intensity cost path segmentation. IEEE Trans Med Imaging 26:1115–29

    Article  Google Scholar 

  46. Wu G, Zhang X, Luo S, Hu Q (2015) Lung segmentation based on customized active shape model from digital radiography chest images. J Med Imaging Health Inform 5:184–91

    Article  Google Scholar 

  47. Yang W, Liu Y, Lin L, Yun Z, Lu Z, Feng Q, Chen W (2018) Lung field segmentation in chest radiographs from boundary maps by a structured edge detector. IEEE J Biomed Health Inform 22:842–51

    Article  Google Scholar 

  48. Ibragimov B, Likar B, Pernuš F, Vrtovec T (2016) Accurate landmark-based segmentation by incorporating landmark misdetections. In: 13th international symposium on biomedical imaging (ISBI). IEEE, pp 1072–1075

  49. Chondro P, Yao C-Y, Ruan S-J, Chien L-C (2018) Low order adaptive region growing for lung segmentation on plain chest radiographs. Neurocomputing 275:1002–11

    Article  Google Scholar 

  50. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of CVPR 2017

  51. RSNA (2018) Kaggle pneumonia detection challenge. https://www.kaggle.com/c/rsna-pneumonia-detection-challenge. Accessed 15 Oct 2019

  52. Karargyris A, Siegelman J, Tzortzis D, Jaeger S, Candemir S, Xue Z, Santosh K, Vajda S, Antani S, Folio L, Thoma G (2016) Combination of texture and shape features to detect pulmonary abnormalities in digital chest X-rays. Int J Comput Assist Radiol Surg 11:99–106

    Article  Google Scholar 

  53. Zheng G, Chu C, Belavy DL, Ibragimov B, Korez R, Vrtovec T, Hutt H, Everson R, Meakin J, Andrade IL, Glocker B, Chen H, Qi Dou Q, Heng PA, Wang C, Forsberg D, Neubert A, Fripp J, Urschler M, Stern D, Wimmer M, Novikov AA, Cheng H, Armbrecht G, Felsenberg D, Li S (2017) Evaluation and comparison of 3d intervertebral disc localization and segmentation methods for 3d t2mr data: a grand challenge. Med Image Anal 35:327–344

    Article  Google Scholar 

  54. Cong J, Zheng Y, Xue W, Cao B, Li S (2019) Ma-shape: modality adaptation shape regression for left ventricle segmentation on mixed MR and CT images. IEEE Access 7:16584–16593

    Article  Google Scholar 

  55. Kholiavchenko M (2019) https://github.com/dirtmaxim/ca-ml-organ-segmentation. Accessed 15 Oct 2019

  56. Jeager S, Candemir S, Antani S, Wang Y, Lu P, G Thoma (2014) Two public chest X-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4:475–477

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Russian Science Foundation (Grant No. 18-71-10072), Russian Foundation for Basic Research (Grant No. 18-47-160015), and Slovenian Research Agency—ARRS (Grant No. P2-0232). Due to the requirements of the Russian Science Foundation, we explicitly state that the methodology development and implementation were solely supported by the Russian Science Foundation (Grant No. 18-71-10072).

Author information

Authors and Affiliations

  1. Innopolis University, Innopolis, Russia

    M. Kholiavchenko, I. Sirazitdinov, K. Kubrak, R. Kuleev & B. Ibragimov

  2. Kazan Federal University, Kazan, Russia

    R. Badrutdinova

  3. Department of Computer Science, University of Copenhagen, Copenhagen, Denmark

    B. Ibragimov

  4. Department of Electronic Engineering, City University of Hong Kong, Hong Kong, China

    Y. Yuan

  5. Faculty of Electrical Engineering, University of Ljubljana, Ljubljana, Slovenia

    T. Vrtovec

Authors
  1. M. Kholiavchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. I. Sirazitdinov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. K. Kubrak
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. R. Badrutdinova
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. R. Kuleev
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Y. Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. T. Vrtovec
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. B. Ibragimov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to B. Ibragimov.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This study has not been supported by any industrial company and does not serve to promote any commercial product. Anonymized publicly available databases of CXR were used in the conducted experiments.

Informed consent

For this type of study, formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kholiavchenko, M., Sirazitdinov, I., Kubrak, K. et al. Contour-aware multi-label chest X-ray organ segmentation. Int J CARS 15, 425–436 (2020). https://doi.org/10.1007/s11548-019-02115-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-019-02115-9

Keywords

Search

Navigation