Skip to main content

Advertisement

Springer Nature Link
Log in

Cascade marker removal algorithm for thyroid ultrasound images

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

During thyroid ultrasound diagnosis, radiologists add markers such as pluses or crosses near a nodule’s edge to indicate the location of a nodule. For computer-aided detection, deep learning models achieve classification, segmentation, and detection by learning the thyroid’s texture in ultrasound images. Experiments show that manual markers are strong prior knowledge for data-driven deep learning models, which interferes with the judgment mechanism of computer-aided detection systems. Aiming at this problem, this paper proposes cascade marker removal algorithm for thyroid ultrasound images to eliminate the interference of manual markers. The algorithm consists of three parts. First, in order to highlight marked features, the algorithm extracts salient features in thyroid ultrasound images through feature extraction module. Secondly, mask correction module eliminates the interference of other features besides markers’ features. Finally, the marker removal module removes markers without destroying the semantic information in thyroid ultrasound images. Experiments show that our algorithm enables classification, segmentation, and object detection models to focus on the learning of pathological tissue features. At the same time, compared with mainstream image inpainting algorithms, our algorithm shows better performance on thyroid ultrasound images. In summary, our algorithm is of great significance for improving the stability and performance of computer-aided detection systems.

During thyroid ultrasound diagnosis, doctors add markers such as pluses or crosses near nodule's edge to indicate the location of nodule. Manual markers are strong prior knowledge for data-driven deep learning models, which interferes the judgment mechanism of computer-aided diagnosis system based on deep learning. Markers make models overfit the specific labeling forms easily, and performs poorly on unmarked thyroid ultrasound images. Aiming at this problem, this paper proposes a cascade marker removal algorithm to eliminate the interference of manual markers. Our algorithm make deep learning models pay attention on nodules’ features of thyroid ultrasound images, which make computer-aided diagnosis system performs good in both marked imaging and unmarked imaging.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Patt Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  2. Barnes C, Shechtman E, Finkelstein A, Goldman DB (2009) Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proc SIGGRAPH) 28(3)

  3. Dekel T, Rubinstein M, Liu C, Freeman WT (2017) On the effectiveness of visible watermarks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2146–2154

  4. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2007) The pascal visual object classes challenge 2007 (voc2007) results

  5. Geirhos R, Rubisch P, Michaelis C, Bethge M, Wichmann FA, Brendel W (2018) Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv:1811.12231

  6. Girshick R (2015) Fast R-CNN. In: The IEEE international conference on computer vision (ICCV)

  7. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: The IEEE conference on computer vision and pattern recognition (CVPR)

  8. Grant EG, Tessler FN, Hoang JK, Langer JE, Beland MD, Berland LL, Cronan JJ, Desser TS, Frates MC, Hamper UM et al (2015) Thyroid ultrasound reporting lexicon: white paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) committee. J Am Coll Radiol 12(12):1272–1279

    Article  Google Scholar 

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  10. Hegedüs L (2004) The thyroid nodule. N Engl J Med 351(17):1764–1771

    Article  Google Scholar 

  11. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  12. Iizuka S, Simo-Serra E, Ishikawa H (2017) Globally and locally consistent image completion. ACM Trans Graph (ToG) 36(4):107

    Article  Google Scholar 

  13. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  14. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks Advances in neural information processing systems 25(2)

  15. Lin TY, Dollar P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR)

  16. Lin TY, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. In: The IEEE international conference on computer vision (ICCV)

  17. Liu G, Reda FA, Shih KJ, Wang TC, Tao A, Catanzaro B (2018) Image inpainting for irregular holes using partial convolutions. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 85–100

  18. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) Ssd: single shot multibox detector. In: European conference on computer vision. Springer, New York, pp 21–37

  19. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. pp 3431–3440

  20. Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv:1508.04025

  21. Mandel SJ (2004) A 64-year-old woman with a thyroid nodule. Jama 292(21):2632–2642

    Article  CAS  Google Scholar 

  22. Neubeck A, Gool LJV (2006) Efficient non-maximum suppression. In: 18th International conference on pattern recognition (ICPR 2006), 20-24 august 2006, Hong Kong, China

  23. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR)

  24. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  25. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv:1804.02767

  26. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN:towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91–99

  27. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation pp 234–241

  28. RUMELHART DE (1986) Learning representations by back-propagating errors. Nature 23

  29. Shelhamer E, Long J, Darrell T Fully convolutional networks for semantic segmentation

  30. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  31. Telea A (2004) An image inpainting technique based on the fast marching method. J Graph Tools 9(1):23–34

    Article  Google Scholar 

  32. Tessler FN, Middleton WD, Grant EG, Hoang JK, Berland LL, Teefey SA, Cronan JJ, Beland MD, Desser TS, Frates MC et al (2017) ACR thyroid imaging, reporting and data system (ti-rads): white paper of the ACR TI-RADS committee. J Am Coll Radiol 14(5):587–595

    Article  Google Scholar 

  33. Tunbridge W, Evered D, Hall R, Appleton D, Brewis M, Clark F, Evans JG, Young E, Bird T, Smith P (1977) The spectrum of thyroid disease in a community: the Whickham Survey. Clinical Endocrinology 7(6):481–493

    Article  CAS  Google Scholar 

  34. Vander JB, Gaston EA, Dawber TR (1968) The significance of nontoxic thyroid nodules: final report of a 15-year study of the incidence of thyroid malignancy. Annals Intern Med 69(3):537–540

    Article  CAS  Google Scholar 

  35. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need arxiv: Computation and Language

  36. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3156–3164

  37. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (Grant No. 61877043, No. 61976155), Major Scientific and Technological Projects for A New Generation of Artificial Intelligence of Tianjin (Grant No. 18ZXZNSY00300).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin Univeristy, Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China

    Xiang Ying, Yulin Zhang, Mei Yu, Jie Gao, Zhiqiang Liu, Hongqian Shen, Ruixuan Zhang, Xuewei Li & Ruiguo Yu

  2. Tianjin Key Laboratory of Advanced Networking, Tianjin, China

    Xiang Ying, Yulin Zhang, Mei Yu, Jie Gao, Zhiqiang Liu, Hongqian Shen, Ruixuan Zhang, Xuewei Li & Ruiguo Yu

  3. Tianjin Medical University Cancer Institute and Hospital, Tianjin, China

    Xi Wei & Jialin Zhu

Authors
  1. Xiang Ying
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yulin Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Mei Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xi Wei
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jialin Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Jie Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Zhiqiang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Hongqian Shen
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Ruixuan Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Xuewei Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Ruiguo Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Xuewei Li or Ruiguo Yu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ying, X., Zhang, Y., Yu, M. et al. Cascade marker removal algorithm for thyroid ultrasound images. Med Biol Eng Comput 58, 2641–2656 (2020). https://doi.org/10.1007/s11517-020-02216-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02216-7

Keywords