Skip to main content
SpringerLink
Log in

Difficulty-aware bi-network with spatial attention constrained graph for axillary lymph node segmentation

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Axillary lymph node (ALN) segmentation in ultrasound images is important for the diagnosis and treatment of breast cancer. Recently, deep learning methods for automatic medical image segmentation have improved significantly. However, two problems arise. (1) A unified model is often employed to segment all images without considering the difficulty diversity. (2) The relationship between elements in the learned class probability map is disregarded. To address these two issues, we propose a novel difficulty-aware bi-network with a spatial attention constrained graph. First, a difficulty grading module (DGM) is developed to learn the difficulty grade of input images. Based on the difficulty grade of images, a novel bi-network architecture is proposed to segment the image adaptively using different branches. In complex branches, a novel spatial attention module (SAM) and graph-based energy with spatial attention constraint are proposed. The learned spatial attention map can provide additional discriminative information. Moreover, the graph-based segmentation framework can capture the relationship between pixels, further improving the segmentation performance for complex images. We conducted an experiment on our ultrasound database using 216 cases. The overall dice similarity coefficient, Jaccard coefficient, volumetric overlap error, and false positive rate are 83.41%, 74.4%, 12.02%, and 13.36% for ALN segmentation, respectively. The comparison results demonstrated that the proposed method outperforms other deep learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang J H, Wang Y Y, Shi X L. An improved graph cut segmentation method for cervical lymph nodes on sonograms and its relationship with node’s shape assessment. Comput Med Imag Graph, 2009, 33: 602–607

    Article  Google Scholar 

  2. Zhang Y Z, Ying M, Lin Y, et al. Coarse-to-fine stacked fully convolutional nets for lymph node segmentation in ultrasound images. In: Proceedings of IEEE International Conference on Bioinformatics and Biomedicine, 2016. 443–448

  3. Chmielewski A, Dufort P, Scaranelo A M. A computerized system to assess axillary lymph node malignancy from sonographic images. Ultrasound Med Biol, 2015, 41: 2690–2699

    Article  Google Scholar 

  4. Diepstraten S C E, Sever A R, Buckens C F M, et al. Value of preoperative ultrasound-guided axillary lymph node biopsy for preventing completion axillary lymph node dissection in breast cancer: a systematic review and meta-analysis. Ann Surg Oncol, 2014, 21: 51–59

    Article  Google Scholar 

  5. Guo Q, Dong Z W, Zhang L, et al. Ultrasound features of breast cancer for predicting axillary lymph node metastasis. J Ultrasound Med, 2018, 37: 1354–1353

    Article  Google Scholar 

  6. Cheng H D, Shan J, Ju W, et al. Automated breast cancer detection and classification using ultrasound images: a survey. Pattern Recogn, 2010, 43: 299–317

    Article  MATH  Google Scholar 

  7. Zheng X Y, Yao Z, Huang Y N, et al. Deep learning radiomics can predict axillary lymph node status in early-stage breast cancer. Nat Commun, 2020, 11: 1236

    Article  Google Scholar 

  8. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431–3440

  9. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, 2015. 234–241

  10. Li C M, Huang R, Ding Z H, et al. A level set method for image segmentation in the presence of intensity inhomogeneities with application to MRI. IEEE Trans Image Process, 2011, 20: 2007–2016

    Article  MathSciNet  MATH  Google Scholar 

  11. Hu J, Shen L, Albanie S, et al. Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell, 2020, 42: 2011–2023

    Article  Google Scholar 

  12. Debats O A, Litjens G J S, Barentsz J O, et al. Automated 3-dimensional segmentation of pelvic lymph nodes in magnetic resonance images. Med Phys, 2011, 38: 6178–6187

    Article  Google Scholar 

  13. Zhy C M, Gu G C, Liu H B, et al. Segmentation of ultrasound image based on texture feature and graph cut. In: Proceedings of International Conference on Computer Science and Software Engineering, 2008. 795–798

  14. Kuo J, Mamou J, Wang Y, et al. A novel nested graph cuts method for segmenting human lymph nodes in 3D high frequency ultrasound images. In: Proceedings of International Symposium on Biomedical Imaging, 2015. 372–375

  15. Kuo J W, Mamou J, Wang Y, et al. Segmentation of 3-D high-frequency ultrasound images of human lymph nodes using graph cut with energy functional adapted to local intensity distribution. IEEE Trans Ultrason Ferroelect Freq Control, 2017, 64: 1514–1525

    Article  Google Scholar 

  16. Zhang J H, Wang Y Y, Dong Y, et al. Sonographic feature extraction of cervical lymph nodes and its relationship with segmentation methods. J Ultrasound Med, 2006, 25: 995–1008

    Article  Google Scholar 

  17. Bnouni N, Mechi O, Rekik I, et al. Semi-automatic lymph node segmentation and classification using cervical cancer MR imaging. In: Proceedings of International Conference on Advanced Technologies for Signal And Image Processing, 2018

  18. Zhang Q, Huang C C, Li C L, et al. Ultrasound image segmentation based on multi-scale fuzzy c-means and particle swarm optimization. In: Proceedings of International Conference on Information Science and Control Engineering, 2012

  19. Meinel L A, Bergtholdt M, Abe H, et al. Multi-modality computer-aided diagnosis system for axillary lymph node (ALN) staging: segmentation of ALN on ultrasound images. In: Proceedings of International Society for Optical Engineering, 2009

  20. Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 2481–2495

    Article  Google Scholar 

  21. Chen L C, Papandreou G, Kokkinos I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs. 2014. ArXiv:1412.7062

  22. Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40: 834–848

    Article  Google Scholar 

  23. Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. 2017. ArXiv:1706.05587

  24. Chen L C, Zhu Y K, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of European Conference on Computer Vision, 2018. 833–851

  25. Zhou Z W, Siddiquee M M R, Tajbakhsh N, et al. Unet++: a nested u-net architecture for medical image segmentation. 2018. ArXiv:1807.10165

  26. Oktay O, Schlemper J, Folgoc L L, et al. Attention U-Net: learning where to look for the pancreas. 2018. ArXiv:1804.03999

  27. Alom M Z, Hasan M, Yakopcic C, et al. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation. 2018. ArXiv:1802.06955

  28. Gu Z W, Cheng J, Fu H Z, et al. CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans Med Imag, 2019, 38: 2281–2292

    Article  Google Scholar 

  29. Wang Y F, Yue W W, Li X L, et al. Comparison study of radiomics and deep learning-based methods for thyroid nodules classification using ultrasound images. IEEE Access, 2020, 8: 52010–52017

    Article  Google Scholar 

  30. He K M, Zhang X Y, Ren S Q, et al. Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2016. 770–778

  31. Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell, 2001, 23: 1222–1239

    Article  Google Scholar 

  32. Boykov Y, Kolmogorov V. An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell, 2004, 26: 1124–1137

    Article  MATH  Google Scholar 

  33. Steiner B, DeVito Z, Chintala S, et al. PyTorch: an imperative style, high-performance deep learning library. In: Proceedings of Neural Information Processing Systems, 2019. 8026–8037

  34. Pohlen T, Hermans A, Mathias M, et al. Full-resolution residual networks for semantic segmentation in street scenes. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 3309–3318

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61701280, 61801263, 61703235, 61701281), National Key R&D Program of China (Grant Nos. 2018YFC0830100, 2018YFC0830102), Natural Science Foundation of Shandong Province (Grant No. ZR2018BF012), and Foundation of Distinguished Associate Professor in Shandong Jianzhu University. The authors would like to greatly thank the anonymous reviewers for their valuable comments and suggestions.

Author information

Authors and Affiliations

  1. School of Software, Shandong University, Jinan, 250101, China

    Qing Xu, Zheyun Qin, Yongjian Wu & Yilong Yin

  2. School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China

    Xiaoming Xi, Xiushan Nie & Chenglong Li

  3. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, 250014, China

    Xianjing Meng

  4. Shandong Provinvial Qianfoshan Hospital, The First Hospital Affiliated with Shandong First Medical University, Jinan, 250014, China

    Dongsheng Zhou

  5. Department of Geriatrics, Qilu Hospital of Shandong University, Jinan, 250012, China

    Yi Qu

Authors
  1. Qing Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoming Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianjing Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zheyun Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiushan Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yongjian Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dongsheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yi Qu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Chenglong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Yilong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiaoming Xi or Yilong Yin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Q., Xi, X., Meng, X. et al. Difficulty-aware bi-network with spatial attention constrained graph for axillary lymph node segmentation. Sci. China Inf. Sci. 65, 192102 (2022). https://doi.org/10.1007/s11432-020-3079-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-3079-8

Keywords

Search

Navigation