Skip to main content
SpringerLink
Log in

Rib segmentation algorithm for X-ray image based on unpaired sample augmentation and multi-scale network

  • S.I. : Babel Fish for Feature-driven Machine Learning to Maximise Societal Value
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Rib segmentation based on chest X-ray images is essential in the computer-aided diagnosis systems of lung cancer, which serves as an important step in the quantitative analysis of various types of lung diseases. However, the traditional methods are unable to segment ribs effectively due to the unclear edges and overlapping regions in X-ray images. A novel rib segmentation framework based on Unpaired Sample Augmentation and Multi-Scale Network is presented in this paper, aiming to improve the accuracy of ribs segmentation with limited labeled samples. First, the algorithm learns pneumonia-related texture changes via unpaired chest x-ray images and generates various augmented samples. Then, a multi-scale network attempts to learn hierarchical features using global supervision. Finally, the refined segmentation result of each organ is achieved by using a deep separation module and a comprehensive loss function. Specifically, the hierarchical features can greatly improve the robustness of multi-organ segmentation networks. The complex multi-organ segmentation task with limited labeled data is simplified with the designed deep separation module. We justify the proposed framework through extensive experiments. It achieves good performance with DSC, Precision, Recall, and Jaccard of 88.03, 88.25, 88.36, and 79.02%, respectively. The DSC value increases nearly by 3% compared to other popular methods. The experimental results show that our algorithm presents better segmentation performance for the overlapping region and fuzzy region of multiple organs, which holds research value and prospects for application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Tabik S, Gómez-Ríos A, Martín-Rodríguez JL, Sevillano-García I, Rey-Area M, Charte D, Guirado E, Suárez JL, Luengo J, Valero-González MA, García-Villanova P, Olmedo-Sánchez E, Herrera F (2020) Covidgr dataset and covid-sdnet methodology for predicting covid-19 based on chest x-ray images. IEEE J Biomed Health Inform 24(12):3595–3605

    Article  Google Scholar 

  2. Hassantabar S, Ahmadi M, Sharifi A (2020) Diagnosis and detection of infected tissue of covid-19 patients based on lung x-ray image using convolutional neural network approaches. Chaos, Solitons and Fractals 140:110170

    Article  MathSciNet  Google Scholar 

  3. Wessel J, Heinrich M.P, von Berg J, Franz A, Saalbach A (2019) Sequential rib labeling and segmentation in chest x-ray using mask r-cnn, arXiv preprint arXiv:1908.08329,

  4. Li H, Han H, Li Z, Wang L, Wu Z, Lu J, Zhou S.K (2020) High-Resolution Chest X-ray Bone Suppression Using Unpaired CT Structural Priors, IEEE Transactions on Medical Imaging, pp. 1–1, . [Online]. Available: https://ieeexplore.ieee.org/document/9058664/

  5. Oliveira H,Mota V, Machado A.M.C,Santos J.A.d, (2020)From 3D to 2D: Transferring knowledge for rib segmentation in chest X-rays, Pattern Recognition Letters, vol. 140, pp. 10–17,

  6. Juhász S, Horváth A, Nikházy L, Horváth G,Horváth A (2010)Segmentation of Anatomical Structures on Chest Radiographs, in XII Mediterranean Conference on Medical and Biological Engineering and Computing 2010.Berlin, Heidelberg: Springer Berlin Heidelberg, , vol.29, pp. 359–362

  7. Peng T, Wang Y, Xu TC, Chen X (2019) Segmentation of lung in chest radiographs using hull and closed polygonal line method. IEEE Access 7:137794–137810

    Article  Google Scholar 

  8. Zhang Y,Miao S, Mansi T, Liao R (2018) Task driven generative modeling for unsupervised domain adaptation: Application to x-ray image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, pp. 599–607

  9. van Ginneken B, ter HaarRomeny BM (2000) Automatic delineation of ribs in frontal chest radiographs. Med Imag 2000 Image Process SPIE 3979:825–836

    Article  Google Scholar 

  10. Lee J, Reeves A.P.(2010) Segmentation of individual ribs from low-dose chest CT, in Medical Imaging : Computer-Aided Diagnosis, vol. 7624, International Society for Optics and Photonics. SPIE, 2010, pp. 1001–1008

  11. Candemir S, Jaeger S, Antani S, Bagci U, Folio LR, Xu Z, Thoma G (2016) Atlas-based rib-bone detection in chest x-rays. Comput Med Imaging Graph 51:32–9

    Article  Google Scholar 

  12. Ali A, Zhu Y, Zakarya M (2021) Exploiting dynamic spatio-temporal correlations for citywide traffic flow prediction using attention based neural networks. Inf Sci 577:852–870

    Article  MathSciNet  Google Scholar 

  13. Ali A, Zhu Y, Zakarya M (2021)A data aggregation based approach to exploit dynamic spatio-temporal correlations for citywide crowd flows prediction in fog computing. Multim Tools Appl. No 2

  14. Ali A, Zhu Y, Chen Q, Yu J, Cai H(2019) Leveraging spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks. In: 2019 IEEE 25th International Conference on Parallel and Distributed Systems (ICPADS), pp. 125–132

  15. Liu D, Chen B, Chin T-J, Rutten MG (2020) Topological sweep for multi-target detection of geostationary space objects. IEEE Trans Signal Process 68:5166–5177

    Article  MathSciNet  MATH  Google Scholar 

  16. Ding S, Qu S, Xi Y, Wan S (2020) Stimulus-driven and concept-driven analysis for image caption generation. Neurocomputing 398:520–530

    Article  Google Scholar 

  17. Hesamian MH, Jia W, He X, Kennedy P (2019) Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging 32(4):582–596

    Article  Google Scholar 

  18. Zhao Y, Li H, Wan S, Sekuboyina A, Hu X, Tetteh G, Piraud M, Menze B (2019) Knowledge-aided convolutional neural network for small organ segmentation. IEEE J Biomed Health Inform 23(4):1363–1373

    Article  Google Scholar 

  19. Wang W, Feng H, Bu Q, Cui L, Xie Y, Zhang A, Feng J, Zhu Z, Chen Z (2020) Mdu-net: a convolutional network for clavicle and rib segmentation from a chest radiograph. J Healthc Eng 07:1–9

    Google Scholar 

  20. Huang L, Pan W, Zhang Y, Qian L, Gao N, Wu Y (2019) Data augmentation for deep learning-based radio modulation classification. IEEE Access 8:1498–1506

    Article  Google Scholar 

  21. Jaeger Stefan Xu, Ziyue Thoma George, Sema Candemir, Les Folio (2016) Atlas-based rib-bone detection in chest x-rays. Comput Med Imaging Graph 51:32–39

    Article  Google Scholar 

  22. Zhang G, Wu H, Guo W (2016)Rib segmentation in chest radiographs by support vector machine. In: Proceedings of the 2016 International Conference on Education, Management, Computer and Society. Atlantis Press, pp. 1564–1567

  23. Liu Y, Zhang X, Cai G, Chen Y, Yun Z, Feng Q, Yang W (2019) Automatic delineation of ribs and clavicles in chest radiographs using fully convolutional densenets. Comput Methods Programs Biomed 180:105014

    Article  Google Scholar 

  24. Zhao A, Balakrishnan G, Durand F, Guttag JV, Dalca AV (2019) Data augmentation using learned transformations for one-shot medical image segmentation. In: The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), June

  25. Ogul BB, Sümer E, Ogul H (2015) Unsupervised rib delineation in chest radiographs by an integrative approach. In: Proceedings of the 10th International Conference on Computer Vision Theory and Applications. SCITEPRESS - Science and and Technology Publications, Berlin, Germany, pp 260–265

  26. Loog M, Ginneken B (2006) Segmentation of the posterior ribs in chest radiographs using iterated contextual pixel classification. IEEE Trans Med Imaging 25(5):602–611

    Article  Google Scholar 

  27. Li X, Luo S, Hu Q (2015) An automatic rib segmentation method on X-ray Radiographs, in multimedia modeling. Cham: Springer International Publishing, , pp. 128–139

  28. Liu Y, Liu M, Xi Y, Qin G, Shen D, Yang W(2020) Xray-Generating Dual-Energy Subtraction Soft-Tissue Images from Chest Radiographs via Bone Edge-Guided GAN. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020. Cham: Springer International Publishing, , vol. 12262, pp. 678–687, series Title: Lecture Notes in Computer Science

  29. Wang H, Cao J, Feng J, Xie Y, Yang D, Chen B (2021) Mixed 2d and 3d convolutional network with multi-scale context for lesion segmentation in breast dce-mri. Biomed Signal Process Control 68:102607

    Article  Google Scholar 

  30. Lei Y, Fu Y, Wang T, Qiu R.L.J, Curran W.J, Liu T, Yang X (2020)Deep Learning in Multi-organ Segmentation, arXiv:2001.10619 [physics], Jan. , arXiv: 2001.10619

  31. Xu X, Lian C, Wang S, Wang A, Royce T, Chen R, Lian J, Shen D(2020) Asymmetrical multi-task attention U-net for the segmentation of prostate bed in CT image. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020. Cham: Springer International Publishing, , vol. 12264, pp. 470–479, series Title: Lecture Notes in Computer Science

  32. Zhang J, Xie Y, Xia Y, Shen C (2021)Dodnet: Learning to segment multi-organ and tumors from multiple partially labeled datasets. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  33. Saleh HM, Saad NH, Isa NAM (2019) Unet-overlapping chromosome segmentation using U-net: convolutional networks with test time augmentation. Proc Comput Sci 159:524–533

    Article  Google Scholar 

  34. Hauberg S, Freifeld O, Larsen A.B.L, Fisher J, Hansen L (2016)Dreaming more data\(:\) class-dependent distributions over diffeomorphisms for learned data augmentation. Artif Intell Stat, pp. 342–350

  35. Saini M, Susan S (2019) Data augmentation of minority class with transfer learning for classification of imbalanced breast cancer dataset using inception-v3. In: Iberian Conference on Pattern Recognition and Image Analysis. Springer, , pp. 409–420

  36. Khan S, Islam N, Jan Z, Din IU, Rodrigues JJC (2019) A novel deep learning based framework for the detection and classification of breast cancer using transfer learning. Pattern Recogn Lett 125:1–6

    Article  Google Scholar 

  37. Guan S, Loew M (2019) Breast cancer detection using synthetic mammograms from generative adversarial networks in convolutional neural networks. J Med Imag 6(3):031411

    Article  Google Scholar 

  38. Alyafi B, Diaz O, Martí R (2020) Dcgans for realistic breast mass augmentation in x-ray mammography, in Medical Imaging 2020: Computer-Aided Diagnosis. Int Soc Op Photonics 11314:1131420

    Google Scholar 

  39. Zhu J.Y, Park T, Isola P, Efros A.A (2017)Unpaired image-to-image translation using cycle-consistent adversarial networks. In:Proceedings of the IEEE international conference on computer vision, pp. 2223–2232

  40. Zhou Z, Siddiquee M.M.R, Tajbakhsh N, Liang J(2018) Unet++: A nested u-net architecture for medical image segmentation, CoRR, vol. abs/1807.10165, . [Online]. Available: http://arxiv.org/abs/1807.10165

  41. Ma J, Chen J, Ng M, Huang R, Li Y, Li C, Yang X, Martel A.L (2021)Loss odyssey in medical image segmentation, Medical Image Analysis, vol.71, p. 102035, . [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1361841521000815

  42. Berman M, Triki A.R, Blaschko M.B (2018)The Lov’asz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks, arXiv:1705.08790 [cs], Apr. , arXiv: 1705.08790. [Online]. Available: http://arxiv.org/abs/1705.08790

  43. Ronneberger O, Fischer P, Brox T(2015) U-net: Convolutional networks for biomedical image segmentation, CoRR, vol. abs/1505.04597, . [Online]. Available: http://arxiv.org/abs/1505.04597

  44. Shelhamer E, Long J, Darrell T (2016)Fully convolutional networks for semantic segmentation, CoRR, vol. abs/1605.06211, . [Online]. Available: http://arxiv.org/abs/1605.06211

  45. Chen L, Zhu Y, Papandreou G, Schroff F, Adam H(2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. Vol. abs/1802.02611. [Online]. Available: http://arxiv.org/abs/1802.02611

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (NO.62001380, 62172438), the fundamental research funds for the central universities (31732111303, 31512111310) and by the open project from the State Key Laboratory for Novel Software Technology, Nanjing University, under Grant No. KFKT2019B17.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Hongyu Wang & Songtao Ding

  2. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an, 710121, China

    Hongyu Wang & Songtao Ding

  3. Xi’an Key Laboratory of Big Data and Intelligent Computing, Xi’an, 710121, China

    Hongyu Wang & Songtao Ding

  4. Department of Information Science and Technology, Northwest University, Xi’an, 710127, China

    Dandan Zhang & Jun Feng

  5. Cangzhou Hospital of Integrated TCM-WM.Hebei, Cangzhou, 061000, China

    Zhanyi Gao

  6. School of Information and Safety Engineering, Zhongnan University of Economics and Law, Wuhan, 430073, China

    Shaohua Wan

Authors
  1. Hongyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dandan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Songtao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhanyi Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shaohua Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaohua Wan.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Zhang, D., Ding, S. et al. Rib segmentation algorithm for X-ray image based on unpaired sample augmentation and multi-scale network. Neural Comput & Applic 35, 11583–11597 (2023). https://doi.org/10.1007/s00521-021-06546-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06546-x

Keywords

Search

Navigation