Skip to main content
SpringerLink
Log in

Difficulty-aware prior-guided hierarchical network for adaptive segmentation of breast tumors

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Breast tumor segmentation is vital to tumor detection at the early stages. Deep learning methods are typically used in automatic tumor segmentation tasks. However, in existing methods, the difference between pixels is disregarded, and the union network architecture is used to segment all pixels; these methods involve a tradeoff between accuracy and efficiency. A novel, difficulty-aware, prior-guided hierarchical network for the adaptive segmentation of breast tumors is presented herein. A difficulty prior learning module is proposed to learn the pixel’s difficulty prior to guild adaptive segmentation in the proposed network. To achieve a more accurate segmentation of hard pixels, a hard pixel processing unit is presented to learn more discriminative features for hard pixels. Experiments are conducted based on three datasets. The experimental results show that the proposed methods outperform traditional deep learning methods and achieve a balance between accuracy and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kuen J, Kong X, Wang G, et al. DelugeNets: deep networks with efficient and flexible cross-layer information inflows. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017. 958–966

  2. Lei Y, He X, Yao J, et al. Breast tumor segmentation in 3D automatic breast ultrasound using Mask scoring R-CNN. Med Phys, 2021, 48: 204–214

    Article  Google Scholar 

  3. Xi X, Xu H, Shi H, et al. Robust texture analysis of multi-modal images using local structure preserving ranklet and multi-task learning for breast tumor diagnosis. Neurocomputing, 2017, 259: 210–218

    Article  Google Scholar 

  4. Rani V M K, Dhenakaran S S. Classification of ultrasound breast cancer tumor images using neural learning and predicting the tumor growth rate. Multimed Tools Appl, 2020, 79: 16967–16985

    Article  Google Scholar 

  5. Guo R, Lu G, Qin B, et al. Ultrasound imaging technologies for breast cancer detection and management: a review. Ultrasound Med Biol, 2018, 44: 37–70

    Article  Google Scholar 

  6. Yang X, Yu L, Wu L, et al. Fine-grained recurrent neural networks for automatic prostate segmentation in ultrasound images. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2017

  7. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of International Conference on Medical Image Computing and Computer-assisted Intervention, 2015. 234–241

  8. Xue C, Zhu L, Fu H, et al. Global guidance network for breast lesion segmentation in ultrasound images. Med Image Anal, 2021, 70: 101989

    Article  Google Scholar 

  9. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015. 3431–3440

  10. Badrinarayanan V, Kendall A, Cipolla R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell, 2017, 39: 2481–2495

    Article  Google Scholar 

  11. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 770–778

  12. Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the impact of residual connections on learning. 2016. ArXiv:1602.07261

  13. Zhang Z, Wu C, Coleman S, et al. DENSE-INception U-net for medical image segmentation. Comput Methods Programs Biomed, 2020, 192: 105395

    Article  Google Scholar 

  14. Ghosh P, Mitchell M, Tanyi J A, et al. Incorporating priors for medical image segmentation using a genetic algorithm. Neurocomputing, 2016, 195: 181–194

    Article  Google Scholar 

  15. Ilesanmi A E, Chaumrattanakul U, Makhanov S S. Methods for the segmentation and classification of breast ultrasound images: a review. J Ultrasound, 2021, 24: 367–382

    Article  Google Scholar 

  16. Ilhan U, Ilhan A. Brain tumor segmentation based on a new threshold approach. Procedia Comput Sci, 2017, 120: 580–587

    Article  Google Scholar 

  17. Nayak T, Bhat N, Bhat V, et al. Automatic segmentation and breast density estimation for cancer detection using an efficient watershed algorithm. In: Data Analytics and Learning. Berlin: Springer, 2019. 347–358

    Chapter  Google Scholar 

  18. Raja N S M, Fernandes S L, Dey N, et al. Contrast enhanced medical MRI evaluation using Tsallis entropy and region growing segmentation. J Ambient Intell Human Comput, 2018. doi: https://doi.org/10.1007/s12652-018-0854-8

  19. Punitha S, Amuthan A, Joseph K S. Benign and malignant breast cancer segmentation using optimized region growing technique. Future Computing Inf J, 2018, 3: 348–358

    Article  Google Scholar 

  20. Fang L, Pan X, Yao Y, et al. A hybrid active contour model for ultrasound image segmentation. Soft Comput, 2020, 24: 18611–18625

    Article  Google Scholar 

  21. Niaz A, Memon A A, Rana K, et al. Inhomogeneous image segmentation using hybrid active contours model with application to breast tumor detection. IEEE Access, 2020, 8: 186851

    Article  Google Scholar 

  22. Chowdhary C L, Mittal M, Kumaresan P, et al. An efficient segmentation and classification system in medical images using intuitionist possibilistic fuzzy c-mean clustering and fuzzy SVM algorithm. Sensors, 2020, 20: 3903

    Article  Google Scholar 

  23. Li Y, Qi H, Dai J, et al. Fully convolutional instance-aware semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2359–2367

  24. Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell, 2018, 40: 834–848

    Article  Google Scholar 

  25. Zhao H, Shi J, Qi X, et al. Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 2881–2890

  26. Ullah I, Jian M, Hussain S, et al. Global context-aware multi-scale features aggregative network for salient object detection. Neurocomputing, 2021, 455: 139–153

    Article  Google Scholar 

  27. Ragab D A, Sharkas M, Marshall S, et al. Breast cancer detection using deep convolutional neural networks and support vector machines. PeerJ, 2019, 7: e6201

    Article  Google Scholar 

  28. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2015. ArXiv:1409.1556

  29. Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1492–1500

  30. Chollet F. Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1251–1258

  31. Yap M H, Pons G, Marti J, et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform, 2018, 22: 1218–1226

    Article  Google Scholar 

  32. Lecun Y, Bottou L, Bengio Y, et al. Gradient-based learning applied to document recognition. Proc IEEE, 1998, 86: 2278–2324

    Article  Google Scholar 

  33. Huang K, Cheng H D, Zhang Y, et al. Medical knowledge constrained semantic breast ultrasound image segmentation. In: Proceedings of the 24th International Conference on Pattern Recognition (ICPR), 2018. 1193–1198

  34. Zhang J, Saha A, Zhu Z, et al. Hierarchical convolutional neural networks for segmentation of breast tumors in MRI with application to radiogenomics. IEEE Trans Med Imag, 2019, 38: 435–447

    Article  Google Scholar 

  35. Chiang T C, Huang Y S, Chen R T, et al. Tumor detection in automated breast ultrasound using 3-D CNN and prioritized candidate aggregation. IEEE Trans Med Imag, 2019, 38: 240–249

    Article  Google Scholar 

  36. Al-antari M A, Al-masni M A, Choi M T, et al. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inf, 2018, 117: 44–54

    Article  Google Scholar 

  37. Zhou Y, Chen H, Li Y, et al. Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Med Image Anal, 2021, 70: 101918

    Article  Google Scholar 

  38. Ho D J, Yarlagadda D V K, D’Alfonso T M, et al. Deep multi-magnification networks for multi-class breast cancer image segmentation. Computized Med Imag Graph, 2021, 88: 101866

    Article  Google Scholar 

  39. Ahmed L, Iqbal M M, Aldabbas H, et al. Images data practices for semantic segmentation of breast cancer using deep neural network. J Ambient Intell Human Comput, 2020. doi: https://doi.org/10.1007/s12652-020-01680-1

  40. He K, Gkioxari G, Dollár P, et al. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 2017. 2961–2969

  41. Chen L C, Papandreou G, Schroff F, et al. Rethinking atrous convolution for semantic image segmentation. 2017. ArXiv:1706.05587

  42. Lv E, Wang X, Cheng Y, et al. Deep ensemble network based on multi-path fusion. Artif Intell Rev, 2019, 52: 151–168

    Article  Google Scholar 

  43. Ullah I, Jian M, Hussain S, et al. A brief survey of visual saliency detection. Multimed Tools Appl, 2020, 79: 34605–34645

    Article  Google Scholar 

  44. Shareef B, Vakanski A, Xian M, et al. Estan: enhanced small tumor-aware network for breast ultrasound image segmentation. 2020. ArXiv:2009.12894

  45. Nie D, Wang L, Xiang L, et al. Difficulty-aware attention network with confidence learning for medical image segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2019. 1085–1092

  46. Xie S, Feng Z, Chen Y, et al. DEAL: difficulty-aware active learning for semantic segmentation. In: Proceedings of the Asian Conference on Computer Vision, 2020

  47. Ullah I, Jian M, Hussain S, et al. DSFMA: deeply supervised fully convolutional neural networks based on multi-level aggregation for saliency detection. Multimed Tools Appl, 2021, 80: 7145–7165

    Article  Google Scholar 

  48. Mathai T S, Lathrop K L, Galeotti J. Learning to segment corneal tissue interfaces in OCT images. In: Proceedings of IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019. 1432–1436

  49. Al-Dhabyani W, Gomaa M, Khaled H, et al. Dataset of breast ultrasound images. Data Brief, 2020, 28: 104863

    Article  Google Scholar 

  50. Shareef B, Xian M, Vakanski A. Stan: small tumor-aware network for breast ultrasound image segmentation. In: Proceedings of IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020. 1–5

  51. Xu Q, Xi X M, Meng X J, et al. Difficulty-aware bi-network with spatial attention constrained graph for axillary lymph node segmentation. Sci China Inf Sci, 2022, 65: 192102

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. 61701280, 61801263, 61703235, 61701281), National Key R&D Program of China (Grant Nos. 2018YFC0830100, 2018YFC0830102), Natural Science Foundation of Shandong Province (Grant No. ZR2018BF012), Foundation of Distinguished Associate Professor in Shandong Jianzhu University. The authors would like to thank all the anonymous reviewers for their valuable time, comments, and suggestions.

Author information

Authors and Affiliations

  1. School of Software Engineering, Shandong University, Jinan, 250101, China

    Sumaira Hussain, Inam Ullah & Yilong Yin

  2. Department of Computer Science, Sindh Madressatul Islam University, Karachi, 74000, Pakistan

    Sumaira Hussain & Syeda Wajiha Naim

  3. School of Computer Science and Technology, Shandong Jianzhu University, Jinan, 250101, China

    Xiaoming Xi

  4. School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China

    Kashif Shaheed

  5. School of Medicine, Shandong University, Jinan, 250100, China

    Cuihuan Tian

  6. Health Management Center, Qilu Hospital of Shandong University, Jinan, 250000, China

    Cuihuan Tian

Authors
  1. Sumaira Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoming Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Inam Ullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Syeda Wajiha Naim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kashif Shaheed
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Cuihuan Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yilong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiaoming Xi, Cuihuan Tian or Yilong Yin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hussain, S., Xi, X., Ullah, I. et al. Difficulty-aware prior-guided hierarchical network for adaptive segmentation of breast tumors. Sci. China Inf. Sci. 66, 122104 (2023). https://doi.org/10.1007/s11432-021-3340-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3340-y

Keywords

Search

Navigation