Skip to main content
SpringerLink
Log in

Identification method of thyroid nodule ultrasonography based on self-supervised learning dual-branch attention learning framework

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Thyroid ultrasound is a widely used diagnostic technique for thyroid nodules in clinical practice. However, due to the characteristics of ultrasonic imaging, such as low image contrast, high noise levels, and heterogeneous features, detecting and identifying nodules remains challenging. In addition, high-quality labeled medical imaging datasets are rare, and thyroid ultrasound images are no exception, posing a significant challenge for machine learning applications in medical image analysis. In this study, we propose a Dual-branch Attention Learning (DBAL) convolutional neural network framework to enhance thyroid nodule detection by capturing contextual information. Leveraging jigsaw puzzles as a pretext task during network training, we improve the network’s generalization ability with limited data. Our framework effectively captures intrinsic features in a global-to-local manner. Experimental results involve self-supervised pre-training on unlabeled ultrasound images and fine-tuning using 1216 clinical ultrasound images from a collaborating hospital. DBAL achieves accurate discrimination of thyroid nodules, with a 88.5% correct diagnosis rate for malignant and benign nodules and a 93.7% area under the ROC curve. This novel approach demonstrates promising potential in clinical applications for its accuracy and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data are not publicly available due to restrictions .Their containing information that could compromise the privacy of research participants.

References

  1. DeSantis CE, Ma J, Gaudet MM, Newman LA, Miller KD, Goding Sauer A, Jemal A, Siegel RL. Breast cancer statistics, 2019. CA Cancer J Clin. 2019;69(6):438–51.

    Article  Google Scholar 

  2. Ospina NS, Iñiguez-Ariza NM, Castro MR. Thyroid nodules: diagnostic evaluation based on thyroid cancer risk assessment. BMJ. 2020;368:l6670.

    Article  Google Scholar 

  3. Filetti S, Durante C, Hartl D, Leboulleux S, Locati LD, Newbold K, Papotti MG, Berruti A. Thyroid cancer: ESMO clinical practice guidelines for diagnosis, treatment and follow-up. Ann Oncol. 2019;30(12):1856–83.

    Article  Google Scholar 

  4. Pedraza L, Vargas C, Narváez F, Durán O, Muñoz E, Romero E. An open access thyroid ultrasound image database. In: 10th international symposium on medical information processing and analysis, vol 9287. SPIE; 2015. p. 188–93.

  5. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim EK. Thyroid imaging reporting and data system for us features of nodules: a step in establishing better stratification of cancer risk. Radiology. 2011;260(3):892–9.

    Article  Google Scholar 

  6. Lingam RK, Qarib MH, Tolley NS. Evaluating thyroid nodules: predicting and selecting malignant nodules for fine-needle aspiration (FNA) cytology. Insights Imaging. 2013;4(5):617–24.

    Article  Google Scholar 

  7. Acharya UR, Swapna G, Sree SV, Molinari F, Gupta S, Bardales RH, Witkowska A, Suri JS. A review on ultrasound-based thyroid cancer tissue characterization and automated classification. Technol Cancer Res Treat. 2014;13(4):289–301.

    Article  Google Scholar 

  8. Acharya UR, Chowriappa P, Fujita H, Bhat S, Dua S, Koh JE, Eugene LW, Kongmebhol P, Ng KH. Thyroid lesion classification in 242 patient population using Gabor transform features from high resolution ultrasound images. Knowl Based Syst. 2016;107:235–45.

    Article  Google Scholar 

  9. Raghavendra U, Acharya UR, Gudigar A, Tan JH, Fujita H, Hagiwara Y, Molinari F, Kongmebhol P, Ng KH. Fusion of spatial gray level dependency and fractal texture features for the characterization of thyroid lesions. Ultrasonics. 2017;77:110–20.

    Article  Google Scholar 

  10. Kattenborn T, Leitloff J, Schiefer F, Hinz S. Review on convolutional neural networks (CNN) in vegetation remote sensing. ISPRS J Photogramm Remote Sens. 2021;173:24–49.

    Article  Google Scholar 

  11. Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, Hao A. Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition. IEEE J Biomed Health Inform. 2018;23(3):1215–24.

    Article  Google Scholar 

  12. Liu Z, Yang C, Huang J, Liu S, Zhuo Y, Xu L. Deep learning framework based on integration of S-Mask R-CNN and Inception-v3 for ultrasound image-aided diagnosis of prostate cancer. Future Gener Comput Syst. 2021;114:358–67.

    Article  Google Scholar 

  13. Liu T, Guo Q, Lian C, Ren X, Liang S, Jing Yu, Niu L, Sun W, Shen D. Automated detection and classification of thyroid nodules in ultrasound images using clinical-knowledge-guided convolutional neural networks. Med Image Anal. 2019;58:101555.

    Article  Google Scholar 

  14. Guo Y, Jiang S-Q, Sun B, Siuly S, Şengür A, Tian J-W. Using neutrosophic graph cut segmentation algorithm for qualified rendering image selection in thyroid elastography video. Health Inf Sci Syst. 2017;5:1–7.

    Article  Google Scholar 

  15. Taleb A, Loetzsch W, Danz N, Severin J, Gaertner T, Bergner B, Lippert C. 3D self-supervised methods for medical imaging. Adv Neural Inf Process Syst. 2020;33:18158–72.

    Google Scholar 

  16. Shurrab S, Duwiari R. Self-supervised learning methods and applications in medical imaging analysis: a survey. PeerJ Comput Sci 2022;8:e1045.

  17. Chen L, Bentley P, Mori K, Misawa K, Fujiwara M, Rueckert D. Self-supervised learning for medical image analysis using image context restoration. Med Image Anal. 2019;58:101539.

    Article  Google Scholar 

  18. Zhang P, Wang F, Zheng Y. Self supervised deep representation learning for fine-grained body part recognition. In: 2017 IEEE 14th international symposium on biomedical imaging (ISBI 2017). IEEE; 2017. p. 578–82.

  19. Bai W, Chen C, Tarroni G, Duan J, Guitton F, Petersen SE, Guo Y, Matthews PM, Rueckert D. Self-supervised learning for cardiac MR image segmentation by anatomical position prediction. In: Medical image computing and computer assisted intervention—MICCAI 2019: 22nd international conference, Shenzhen, China, October 13–17, 2019, proceedings, part II 22. Springer, 2019. p. 541–9.

  20. Taleb A, Lippert C, Klein T, Nabi M. Multimodal self-supervised learning for medical image analysis. In: Information processing in medical imaging: 27th international conference, IPMI 2021, virtual event, June 28–June 30, 2021, proceedings. Springer; 2021. p. 661–73.

  21. Noroozi M, Favaro P. Unsupervised learning of visual representations by solving jigsaw puzzles. In: Computer vision–ECCV 2016: 14th European conference, Amsterdam, The Netherlands, October 11–14, 2016, proceedings, part VI. Springer; 2016. p. 69–84.

  22. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770–8.

  23. Moon HJ, Kwak JY, Kim EK, Kim MJ. A taller-than-wide shape in thyroid nodules in transverse and longitudinal ultrasonographic planes and the prediction of malignancy. Thyroid. 2011;21(11):1249–53.

    Article  Google Scholar 

  24. Anil G, Hegde A, Chong FHV. Thyroid nodules: risk stratification for malignancy with ultrasound and guided biopsy. Cancer Imaging. 2011;11(1):209.

    Google Scholar 

  25. Wei X-S, Luo J-H, Jianxin W, Zhou Z-H. Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE Trans Image Process. 2017;26(6):2868–81.

    Article  MathSciNet  Google Scholar 

  26. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y. Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv Preprint. 2013. http://arxiv.org/abs/1312.6229.

  27. Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A. Dataset of breast ultrasound images. Data Brief. 2020;28:104863.

    Article  Google Scholar 

  28. Rodrigues PS. “Breast ultrasound image” Mendeley data v1. 2019. Available from: https://data.mendeley.com/datasets/wmy84gzngw/1. Accessed 3 Apr 2023.

  29. Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.

    Article  Google Scholar 

  30. Guo X, Zhao H, Tang Z. An improved deep learning approach for thyroid nodule diagnosis. In: 2020 IEEE 17th international symposium on biomedical imaging (ISBI). IEEE; 2020. p. 296–9.

  31. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv Preprint. 2014. http://arxiv.org/abs/1409.1556.

  32. Wang L, Zhang L, Zhu M, Qi X, Yi Z. Automatic diagnosis for thyroid nodules in ultrasound images by deep neural networks. Med Image Anal. 2020;61:101665.

    Article  Google Scholar 

  33. Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 31. 2017.

Download references

Acknowledgements

This work was guided and supported by Basic and Applied Basic Research Foundation of Guangdong Province, China (Grant No. 2022A1515140033), Guangdong Provincial Drug Administration (Grant No. 2022YDZ06), National Natural Science Foundation of China (62073086) and Natural Science Foundation of Guangdong Province, China (2022A 1515011445)

Funding

This work was guided and supported by Basic and Applied Basic Research Foundation of Guangdong Province, China (Grant No. 2022A1515140033), Guangdong Provincial Drug Administration (Grant No. 2022YDZ06), National Natural Science Foundation of China (62073086) and Natural Science Foundation of Guangdong Province, China (2022A 1515011445)

Author information

Author notes

  1. Yifei Xie and Zhengfei Yang have contributed equally to this work and are co-first authors.

Authors and Affiliations

  1. Guangzhou Panyu Central Hospital, Guangzhou, 510006, Guangdong, People’s Republic of China

    Yifei Xie, Lin Yang & Jiaxun Wang

  2. Guangdong University of Technology, Guangzhou, 510006, Guangdong, People’s Republic of China

    Yifei Xie, Qiyu Yang, Dongning Liu, Shuzhuang Tang, Xuan Duan & Yu-Jing Lu

  3. Smart Medical Innovation Technology Center, Guangdong University of Technology, Guangzhou, 510006, Guangdong, People’s Republic of China

    Yu-Jing Lu

  4. Guangdong Medical Device Quality Supervision and Inspection Institute, Guangzhou, 510006, Guangdong, People’s Republic of China

    Changming Hu

  5. Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, 510006, Guangdong, People’s Republic of China

    Zhengfei Yang

Authors
  1. Yifei Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhengfei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongning Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shuzhuang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Xuan Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Changming Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Yu-Jing Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jiaxun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qiyu Yang, Yu-Jing Lu or Jiaxun Wang.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Consent to participate

Not applicable.

Consent for publication

All authors have checked the manuscript and have agreed to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Y., Yang, Z., Yang, Q. et al. Identification method of thyroid nodule ultrasonography based on self-supervised learning dual-branch attention learning framework. Health Inf Sci Syst 12, 7 (2024). https://doi.org/10.1007/s13755-023-00266-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00266-3

Keywords

Search

Navigation