Abstract
Purpose The automatic analysis of ultrasound images facilitates the diagnosis of breast cancer effectively and objectively. However, due to the characteristics of ultrasound images, it is still a challenging task to achieve analyzation automatically. We suppose that the algorithm will extract lesion regions and distinguish categories easily if it is guided to focus on the lesion regions.Method We propose a multi-task learning (SHA-MTL) model based on soft and hard attention mechanisms for breast ultrasound (BUS) image simultaneous segmentation and binary classification. The SHA-MTL model consists of a dense CNN encoder and an upsampling decoder, which are connected by attention-gated (AG) units with soft attention mechanism. Cross-validation experiments are performed on BUS datasets with category and mask labels, and multiple comprehensive analyses are performed on the two tasks.Results We assess the SHA-MTL model on a public BUS image dataset. For the segmentation task, the sensitivity and DICE of the SHA-MTL model to the lesion regions increased by 2.27% and 1.19% compared with the single task model, respectively. The classification accuracy and F1 score increased by 2.45% and 3.82%, respectively.Conclusion The results validate the effectiveness of our model and indicate that the SHA-MTL model requires less a priori knowledge to achieve better results by comparing with other recent models. Therefore, we can draw the conclusion that paying more attention to the lesion region of BUS is conducive to the discrimination of lesion types.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Global Burden of Disease Cancer Collaboration (2015) The Global Burden of Cancer 2013. JAMA Oncol 1(4):505–527
Cheng Heng-Da, Shan Juan, Wen Ju, Guo Yanhui, Zhang Ling (2010) Automated breast cancer detection and classification using ultrasound images: a survey. Pattern Recognit 43(1):299–317
Xue C, Zhu L, Huazhu F, Xiaowei H, Li X, Zhang H, Heng P-A (2021) Global guidance network for breast lesion segmentation in ultrasound images. Med Image Anal 70:101989
Huang Q, Huang Y, Luo Y, Yuan F, Li X (2020) Segmentation of breast ultrasound image with semantic classification of superpixels. Med Image Anal 61:101657
Moon WK, Lee Y-W, Ke H-H, Lee SH, Huang C-S, Chang, R-F (2020) Computer-aided diagnosis of breast ultrasound images using ensemble learning from convolutional neural networks. Computer Methods and Programs in Biomedicine, 190:105361
Wang Y, Choi EJ, Choi Y, Zhang H, Jin GY, Ko S-B (2020) Breast cancer classification in automated breast ultrasound using multiview convolutional neural network with transfer learning. Ultrasound Med Biol 46(5):1119–1132
Xian M, Zhang Y, Cheng H-D, Xu F, Huang K, Zhang B, Ding J, Ning C, Wang Y (2018) A benchmark for breast ultrasound image segmentation (BUSIS). Infinite Study
Amiri Mina, Brooks Rupert, Behboodi Bahareh, Rivaz Hassan (2020) Two-stage ultrasound image segmentation using u-net and test time augmentation. Int J Comput Assist Radiol Surg 15(6):981–988
Han Zhongyi, Wei Benzheng, Hong Yanfei, Li Tianyang, Cong Jinyu, Zhu Xue, Wei Haifeng, Zhang Wei (2020) Accurate screening of Covid-19 using attention based deep 3d multiple instance learning. IEEE Trans Med Imaging 39(8):2584–2594
Schlemper Jo, Oktay Ozan, Schaap Michiel, Heinrich Mattias, Kainz Bernhard, Glocker Ben, Rueckert Daniel (2019) Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53:197–207
Zhuang Z, Li N, Raj ANJ, Mahesh VGV, Qiu S (2019) An rdau-net model for lesion segmentation in breast ultrasound images. PLoS ONE 14(8):e0221535
Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data Brief 28:104863
Xu Y, Wang Y, Yuan J, Cheng Q, Wang X, Carson PL (2019) Medical breast ultrasound image segmentation by machine learning. Ultrasonics, 91:1–9
Acknowledgements
This work was partly funded by Natural Science Foundation of China (No. 61872225); Introduction and Cultivation Program for Young Creative Talents in Colleges and Universities of Shandong Province (No. 173); the Natural Science Foundation of Shandong Province (No. ZR2019ZD04, No. ZR2015FM010); the Project of Science and technology plan of Shandong higher education institutions Program (No. J15LN20); the Project of Shandong Province Medical and Health Technology Development Program (No. 2016WS0577).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Human and animal rights
The paper does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent is obtained from all individual participants included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, G., Zhao, K., Hong, Y. et al. SHA-MTL: soft and hard attention multi-task learning for automated breast cancer ultrasound image segmentation and classification. Int J CARS 16, 1719–1725 (2021). https://doi.org/10.1007/s11548-021-02445-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-021-02445-7