Skip to main content
SpringerLink
Log in

Dual-branch network via pseudo-label training for thyroid nodule detection in ultrasound image

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Automated nodule detection in the ultrasound image is essential for computer-aided thyroid tumor diagnosis. However, in the ultrasound image, the solid nodule has imaging characteristics similar to other tissues, making it challenging to detect such nodules. Therefore, we proposed a feature-enhanced dual-branch network (FDnet) to complete the nodule detection task by adding a semantic segmentation branch and a feature enhancement mechanism into the detection network. This design improves the target area’s proposal score and suppresses the interference of similar tissues, which can reduce the false-positive rate of the proposed bounding box and finally obtain more reliable detection results. Additionally, to solve the lack of fine-grained mask information for semantic segmentation branch training in the actual scenario, we also proposed an iterative training strategy that combines the ground-truth boundary box with the branch results to generate a pseudo-label mask. Finally, we carried out various comparative experiments to verify the feasibility of the proposed network and training strategy. A series of experiments showed that FDnet could achieve competitive detection performance (mAP: 61.8/92.5/65.9), which metrics are better than the state-of-the-art detection methods. Besides, the performance using the pseudo-label mask training is also close to using the ground-truth mask in the public dataset, and the inference speed per image is also comparable to that of other networks both in the two datasets. This result shows that our method can improve the efficiency of thyroid nodule detection without fine-grained annotation, and the output result of the trained semantic segmentation branch can guide the further segmentation of nodule edge, which has practical clinical significance. We will release the source code and the public dataset at https://github.com/songruoning/Thyroid_Solid_Nodule.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Song R, Zhang L, Zhu C, Liu J, Yang J, Zhang T (2020) Thyroid nodule ultrasound image classification through hybrid feature cropping network. IEEE Access 8:64,064–64,074. https://doi.org/10.1109/ACCESS.2020.2982767

    Article  Google Scholar 

  2. Vander JB, Gaston EA, Dawber TR (1968) The significance of nontoxic thyroid nodules: final report of a 15-year study of the incidence of thyroid malignancy. Ann Intern Med 69(3):537–540

    Article  Google Scholar 

  3. Tan GH, Gharib H (1997) Thyroid incidentalomas: management approaches to nonpalpable nodules discovered incidentally on thyroid imaging. Ann Intern Med 126(3):226–231

    Article  Google Scholar 

  4. Qin P, Wu K, Hu Y, Zeng J, Chai X (2019) Diagnosis of benign and malignant thyroid nodules using combined conventional ultrasound and ultrasound elasticity imaging. IEEE J Biomed Health Inform 24 (4):1028–1036

    Article  Google Scholar 

  5. Kwak JY, Han KH, Yoon JH, Moon HJ, Son EJ, Park SH, Jung HK, Choi JS, Kim BM, Kim EK (2011) Thyroid imaging reporting and data system for us features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3):892–899

    Article  Google Scholar 

  6. Smith-Bindman R, Lebda P, Feldstein VA, Sellami D, Goldstein RB, Brasic N, Jin C, Kornak J (2013) Risk of thyroid cancer based on thyroid ultrasound imaging characteristics: results of a population-based study. JAMA Int Med 173(19):1788–1795

    Article  Google Scholar 

  7. Russ G, Bonnema SJ, Erdogan MF, Durante C, Ngu R, Leenhardt L (2017) European thyroid association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the eu-tirads. Eur Thyroid J 6(5):225–237

    Article  Google Scholar 

  8. Noble JA, Boukerroui D (2006) Ultrasound image segmentation: a survey. IEEE Trans Med Imaging 25(8):987–1010

    Article  Google Scholar 

  9. Maroulis DE, Savelonas MA, Iakovidis DK, Karkanis SA, Dimitropoulos N (2007) Variable background active contour model for computer-aided delineation of nodules in thyroid ultrasound images. IEEE J Biomed Health Inform 11(5):537– 543

    Google Scholar 

  10. Cheng HD, Shan J, Ju W, Guo Y, Zhang L (2010) Automated breast cancer detection and classification using ultrasound images: A survey. Pattern Recognit 43(1):299–317

    Article  Google Scholar 

  11. Lee YH, Kim DW, In HS, Park JS, Kim SH, Eom JW, Kim B, Lee EJ, Rho MH (2011) Differentiation between benign and malignant solid thyroid nodules using an us classification system. Korean J Radiol 12(5):559–567

    Article  Google Scholar 

  12. Unsal O, Akpinar M, Turk B, Ucak I, Ozel A, Kayaoglu S, Coskun BU (2017) Sonographic scoring of solid thyroid nodules: effects of nodule size and suspicious cervical lymph node. Braz J Otorhinolaryngol 83(1):73–79

    Article  Google Scholar 

  13. Chang RF, Wu WJ, Moon WK, Chen DR (2005) Automatic ultrasound segmentation and morphology based diagnosis of solid breast tumors. Breast Cancer Res Treat 89(2):179

    Article  Google Scholar 

  14. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  Google Scholar 

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  Google Scholar 

  16. Girshick R, Donahue J, Darrell T, Malik J (2016) Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1):142–158. https://doi.org/10.1109/TPAMI.2015.2437384

    Article  Google Scholar 

  17. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: Unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  18. Song W, Li S, Liu J, Qin H, Zhang B, Zhang S, Hao A (2018) Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition. IEEE J Biomed Health Inf 23 (3):1215–1224

    Article  Google Scholar 

  19. Liu R, Zhou S, Guo Y, Wang Y, Chang C (2020) Nodule localization in thyroid ultrasound images with a joint-training convolutional neural network. J Digit Imaging 33(5):1266– 1279

    Article  Google Scholar 

  20. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: A survey. Int J Comput Vis 128(2):261–318

    Article  Google Scholar 

  21. Liu H, Fang S, Zhang Z, Li D, Lin K, Wang J (2021) Mfdnet: Collaborative poses perception and matrix fisher distribution for head pose estimation. IEEE Trans Multimed

  22. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  23. Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  24. Dai J, He K, Sun J (2016) Instance-aware semantic segmentation via multi-task network cascades. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3150–3158

  25. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969

  26. Huang Z, Huang L, Gong Y, Huang C, Wang X (2019) Mask scoring r-cnn. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6409–6418

  27. Wu Y, Chen Y, Yuan L, Liu Z, Wang L, Li H, Fu Y (2020) Rethinking classification and localization for object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10,186–10,195

  28. Zhang H, Chang H, Ma B, Wang N, Chen X (2020) Dynamic r-cnn: Towards high quality object detection via dynamic training. In: European Conference on Computer Vision. Springer, pp 260–275

  29. Tian Z, Shen C, Chen H, He T (2019) Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9627–9636

  30. Chen H, Sun K, Tian Z, Shen C, Huang Y, Yan Y (2020) Blendmask: Top-down meets bottom-up for instance segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8573–8581

  31. Tong K, Wu Y, Zhou F (2020) Recent advances in small object detection based on deep learning: A review. Image Vision Comput 97:103910

    Article  Google Scholar 

  32. Lee DH, et al. (2013) Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol. 3

  33. Zheng Z, Zheng L, Yang Y (2017) Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In: Proceedings of the IEEE international conference on computer vision, pp 3754–3762

  34. Oliver A, Odena A, Raffel C, Cubuk ED, Goodfellow IJ (2018) Realistic evaluation of deep semi-supervised learning algorithms. arXiv:1804.09170

  35. Liu X, Van De Weijer J, Bagdanov AD (2019) Exploiting unlabeled data in cnns by self-supervised learning to rank. IEEE Trans Pattern Anal Mach Intell 41(8):1862–1878

    Article  Google Scholar 

  36. Li X, Chen W, Xie D, Yang S, Yuan P, Pu S, Zhuang Y (2020) A free lunch for unsupervised domain adaptive object detection without source data. arXiv:2012.05400

  37. Xie Q, Luong MT, Hovy E, Le QV (2020) Self-training with noisy student improves imagenet classification. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10,687–10,698

  38. Li S, Huang J, Hua XS, Zhang L (2021) Category dictionary guided unsupervised domain adaptation for object detection. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 1949–1957

  39. Li Z, Liu H, Zhang Z, Liu T, Xiong NN (2021) Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Trans Neural Netw Learn Syst

  40. Zhao X, Liang S, Wei Y (2018) Pseudo mask augmented object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4061–4070

  41. Zhang Y, Bai Y, Ding M, Li Y, Ghanem B (2018) Weakly-supervised object detection via mining pseudo ground truth bounding-boxes. Pattern Recogn 84:68–81

    Article  Google Scholar 

  42. Yan P, Li G, Xie Y, Li Z, Wang C, Chen T, Lin L (2019) Semi-supervised video salient object detection using pseudo-labels. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7284–7293

  43. Liu T, Liu H, Li YF, Chen Z, Zhang Z, Liu S (2019) Flexible ftir spectral imaging enhancement for industrial robot infrared vision sensing. IEEE Trans Industr Inform 16(1):544–554

    Article  Google Scholar 

  44. Liu J, Wang X, Wang R, Xu C, Zhao R, Li H, Zhang S, Yao X (2020) Near-infrared auto-fluorescence spectroscopy combining with fisher’s linear discriminant analysis improves intraoperative real-time identification of normal parathyroid in thyroidectomy. BMC Surgery 20(1):1–7

    Article  Google Scholar 

  45. Wu X, Tan G, Zhu N, Chen Z, Yang Y, Wen H, Li K (2021) Cachetrack-yolo: Real-time detection and tracking for thyroid nodules and surrounding tissues in ultrasound videos. IEEE J Biomed Health Inf:1–1. https://doi.org/10.1109/JBHI.2021.3084962

  46. Kumar V, Webb J, Gregory A, Meixner DD, Knudsen JM, Callstrom M, Fatemi M, Alizad A (2020) Automated segmentation of thyroid nodule, gland, and cystic components from ultrasound images using deep learning. IEEE Access 8:63,482–63,496

    Article  Google Scholar 

  47. Gong H, Chen G, Wang R, Xie X, Mao M, Yu Y, Chen F, Li G (2021) Multi-task learning for thyroid nodule segmentation with thyroid region prior. In: 2021 IEEE 18th International symposium on biomedical imaging (ISBI). IEEE, pp 257–261

  48. Kesarkar XA, Kulhalli K (2021) Thyroid nodule detection using artificial neural network. In: 2021 International conference on artificial intelligence and smart systems (ICAIS). IEEE, pp 11–15

  49. Chi J, Walia E, Babyn P, Wang J, Groot G, Eramian M (2017) Thyroid nodule classification in ultrasound images by fine-tuning deep convolutional neural network. J Digital Imaging 30(4):477–486

    Article  Google Scholar 

  50. Gomes Ataide EJ, Ponugoti N, Illanes A, Schenke S, Kreissl M, Friebe M (2020) Thyroid nodule classification for physician decision support using machine learning-evaluated geometric and morphological features. Sensors 20(21):6110

    Article  Google Scholar 

  51. Vadhiraj VV, Simpkin A, O’Connell J, Singh Ospina N, Maraka S, O’Keeffe DT (2021) Ultrasound image classification of thyroid nodules using machine learning techniques. Medicina 57(6):527

    Article  Google Scholar 

  52. Avola D, Cinque L, Fagioli A, Filetti S, Grani G, Rodolà E (2021) Multimodal feature fusion and knowledge-driven learning via experts consult for thyroid nodule classification. IEEE Trans Circuits Syst Video Technol

  53. Zhu C, Tao S, Chen H, Li M, Wang Y, Liu J, Jin M (2021) Hybrid model enabling highly efficient follicular segmentation in thyroid cytopathological whole slide image. Intelligent Medicine

  54. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  55. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  56. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  57. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  58. Rezatofighi H, Tsoi N, Gwak J, Sadeghian A, Reid I, Savarese S (2019) Generalized intersection over union: A metric and a loss for bounding box regression. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 658–666

  59. Yu J, Jiang Y, Wang Z, Cao Z, Huang T (2016) Unitbox: An advanced object detection network. In: Proceedings of the 24th ACM international conference on multimedia, pp 516–520

  60. Chen K, Wang J, Pang J, Cao Y, Xiong Y, Li X, Sun S, Feng W, Liu Z, Xu J et al (2019) Mmdetection: Open mmlab detection toolbox and benchmark. arXiv:1906.07155

  61. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. In: Advances in neural information processing systems, pp 8026–8037

  62. Pedraza L, Vargas C, Narváez F, Durán O, Muñoz E, Romero E (2015) An open access thyroid ultrasound image database. In: 10th International symposium on medical information processing and analysis, vol 9287. International Society for Optics and Photonics, p 92870W

  63. Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. In: Proceedings of the european conference on computer vision (ECCV), pp 734–750

  64. Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S (2020) End-to-end object detection with transformers. In: ECCV

  65. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision. Springer, pp 21–37

  66. Chen Q, Wang Y, Yang T, Zhang X, Cheng J, Sun J (2021) You only look one-level feature. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 13,039–13,048

  67. Cai Z, Vasconcelos N (2019) Cascade r-cnn: High quality object detection and instance segmentation. IEEE Trans Pattern Anal Mach Intell

Download references

Acknowledgements

The authors thank Taolue Feng and Yanting Zhang for their assistance with data analysis and language check.

In addition, the work is conducted on the platform of Center for Data Science of Beijing University of Posts and Telecommunications.

Author information

Author notes

    Authors and Affiliations

    1. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China

      Ruoning Song, Chuang Zhu, Long Zhang, Yihao Luo, Jun Liu & Jie Yang

    2. Ultrasonic Medical Center, Chenzhou No.1 People’s Hospital, Chenzhou, China

      Tong Zhang

    Authors
    1. Ruoning Song
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Chuang Zhu
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Long Zhang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Tong Zhang
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Yihao Luo
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Jun Liu
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Jie Yang
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Tong Zhang.

    Ethics declarations

    Ethics approval

    All data were evaluated retrospectively. All studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee. The number of ethical approval file is No. 2020056.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Ruoning Song and Chuang Zhu contributed equally to this work.

    Rights and permissions

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Song, R., Zhu, C., Zhang, L. et al. Dual-branch network via pseudo-label training for thyroid nodule detection in ultrasound image. Appl Intell 52, 11738–11754 (2022). https://doi.org/10.1007/s10489-021-02967-2

    Download citation

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s10489-021-02967-2

    Keywords

    Search

    Navigation