Skip to main content
Springer Nature Link
Log in

An Anti-biased TBSRTC-Category Aware Nuclei Segmentation Framework with a Multi-label Thyroid Cytology Benchmark

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 (MICCAI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14225))

  • 5816 Accesses

Abstract

The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC) has been widely accepted as a reliable criterion for thyroid cytology diagnosis, where extensive diagnostic information can be deduced from the allocation and boundary of cell nuclei. However, two major challenges hinder accurate nuclei segmentation from thyroid cytology. Firstly, unbalanced distribution of nuclei morphology across different TBSRTC categories can lead to a biased model. Secondly, the insufficiency of densely annotated images results in a less generalized model. In contrast, image-wise TBSRTC labels, while containing lightweight information, can be deeply explored for segmentation guidance. To this end, we propose a TBSRTC-category aware nuclei segmentation framework (TCSegNet). To top up the small amount of pixel-wise annotations and eliminate the category preference, a larger amount of image-wise labels are taken in as the complementary supervision signal in TCSegNet. This integration of data can effectively guide the pixel-wise nuclei segmentation task with a latent global context. We also propose a semi-supervised extension of TCSegNet that leverages images with only TBSRTC-category labels. To evaluate the proposed framework and also for further cytology cell studies, we curated and elaborately annotated a multi-label thyroid cytology benchmark, collected clinically from 2019 to 2022, which will be made public upon acceptance. Our TCSegNet outperforms state-of-the-art segmentation approaches with an improvement of 2.0% Dice and 2.7% IoU; besides, the semi-supervised extension can further boost this margin. In conclusion, our study explores the weak annotations by constructing an image-wise-label-guided nuclei segmentation framework, which has the potential medical importance to assist thyroid abnormality examination. Code is available at https://github.com/Junchao-Zhu/TCSegNet.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Similar content being viewed by others

References

  1. Ferlay, J., et al.: Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012. Int. J. Cancer 136(5), E359–E386 (2015)

    Article  Google Scholar 

  2. Haugen, B.R., et al.: 2015 American thyroid association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the american thyroid association guidelines task force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1), 1–133 (2016)

    Article  MathSciNet  Google Scholar 

  3. Cibas, E.S., Ali, S.Z.: The bethesda system for reporting thyroid cytopathology. Thyroid 19(11), 1159–1165 (2009)

    Article  Google Scholar 

  4. Kumar, N., et al.: A multi-organ nucleus segmentation challenge. IEEE Trans. Med. Imaging 39(5), 1380–1391 (2019)

    Article  Google Scholar 

  5. Veta, M., et al.: Prognostic value of automatically extracted nuclear morphometric features in whole slide images of male breast cancer. Mod. Pathol. 25(12), 1559–1565 (2012)

    Article  Google Scholar 

  6. Kakudo, K.: Thyroid FNA Cytology: Differential Diagnoses and Pitfalls. Springer, Heidelberg (2019). https://doi.org/10.1007/978-981-13-1897-9

    Book  Google Scholar 

  7. Xing, F., Yang, L.: Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev. Biomed. Eng. 9, 234–263 (2016)

    Article  Google Scholar 

  8. Cibas, E.S., Ali, S.Z.: The 2017 bethesda system for reporting thyroid cytopathology. Thyroid 27(11), 1341–1346 (2017)

    Article  Google Scholar 

  9. Graham, S., et al.: Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Med. Image Anal. 58, 101563 (2019)

    Article  Google Scholar 

  10. Gamper, J., Alemi Koohbanani, N., Benet, K., Khuram, A., Rajpoot, N.: PanNuke: an open pan-cancer histology dataset for nuclei instance segmentation and classification. In: Reyes-Aldasoro, C.C., Janowczyk, A., Veta, M., Bankhead, P., Sirinukunwattana, K. (eds.) ECDP 2019. LNCS, vol. 11435, pp. 11–19. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23937-4_2

    Chapter  Google Scholar 

  11. Greenwald, N.F., et al.: Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotechnol. 40(4), 555–565 (2022)

    Article  Google Scholar 

  12. Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 111, 98–136 (2015)

    Article  Google Scholar 

  13. Zhou, Y., Onder, O.F., Dou, Q., Tsougenis, E., Chen, H., Heng, P.-A.: CIA-Net: robust nuclei instance segmentation with contour-aware information aggregation. In: Chung, A.C.S., Gee, J.C., Yushkevich, P.A., Bao, S. (eds.) IPMI 2019. LNCS, vol. 11492, pp. 682–693. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20351-1_53

    Chapter  Google Scholar 

  14. Huang, J., Shen, Y., Shen, D., Ke, J.: CA2.5-net nuclei segmentation framework with a microscopy cell benchmark collection. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 445–454. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_43

    Chapter  Google Scholar 

  15. Ke, J., et al.: Clusterseg: a crowd cluster pinpointed nucleus segmentation framework with cross-modality datasets. Med. Image Anal. 85, 102758 (2023)

    Article  Google Scholar 

  16. Peng, Z., et al.: Conformer: local features coupling global representations for visual recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 367–376 (2021)

    Google Scholar 

  17. Hou, Q., Cheng, M.M., Hu, X., Borji, A., Tu, Z., Torr, P.H.: Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3203–3212 (2017)

    Google Scholar 

  18. Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst. 30, 1–10 (2017)

    Google Scholar 

  19. Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016)

  20. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961–2969 (2017)

    Google Scholar 

  21. Cao, H., et al.: Swin-unet: unet-like pure transformer for medical image segmentation. In: Computer Vision-ECCV 2022 Workshops: Tel Aviv, Israel, 23–27 October 2022, Proceedings, Part III, pp. 205–218. Springer, Heidelberg (2023). https://doi.org/10.1007/978-3-031-25066-8_9

  22. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Article  Google Scholar 

  23. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested u-net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1

    Chapter  Google Scholar 

  24. Zou, Y., et al.: Pseudoseg: designing pseudo labels for semantic segmentation. arXiv preprint arXiv:2010.09713 (2020)

  25. Chen, X., Yuan, Y., Zeng, G., Wang, J.: Semi-supervised semantic segmentation with cross pseudo supervision. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2613–2622 (2021)

    Google Scholar 

  26. Luo, X., Hu, M., Song, T., Wang, G., Zhang, S.: Semi-supervised medical image segmentation via cross teaching between cnn and transformer. In: International Conference on Medical Imaging with Deep Learning, pp. 820–833. PMLR (2022)

    Google Scholar 

  27. Chen, Z., Zhu, L., Wan, L., Wang, S., Feng, W., Heng, P.A.: A multi-task mean teacher for semi-supervised shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5611–5620 (2020)

    Google Scholar 

  28. Liu, W., Rabinovich, A., Berg, A.C.: Parsenet: looking wider to see better. arXiv preprint arXiv:1506.04579 (2015)

Download references

Acknowledgement

This work was supported by National Natural Science Foundation of China (Grant No. 62102247) and Natural Science Foundation of Shanghai (No. 23ZR1430700).

Author information

Authors and Affiliations

  1. School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, China

    Junchao Zhu

  2. Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

    Yiqing Shen

  3. School of Engineering, Shanghai Ocean University, Shanghai, China

    Haolin Zhang

  4. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Jing Ke

  5. School of Computer Science and Engineering, University of New South Wales, Sydney, Australia

    Jing Ke

Authors
  1. Junchao Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yiqing Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haolin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Ke .

Editor information

Editors and Affiliations

  1. Icahn School of Medicine, Mount Sinai, NYC, NY, USA, Tel Aviv University, Tel Aviv, Israel

    Hayit Greenspan

  2. Emory University, Atlanta, GA, USA

    Anant Madabhushi

  3. Queen’s University, Kingston, ON, Canada

    Parvin Mousavi

  4. The University of British Columbia, Vancouver, BC, Canada

    Septimiu Salcudean

  5. Yale University, New Haven, CT, USA

    James Duncan

  6. IBM Research, San Jose, CA, USA

    Tanveer Syeda-Mahmood

  7. Johns Hopkins University, Baltimore, MD, USA

    Russell Taylor

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhu, J., Shen, Y., Zhang, H., Ke, J. (2023). An Anti-biased TBSRTC-Category Aware Nuclei Segmentation Framework with a Multi-label Thyroid Cytology Benchmark. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14225. Springer, Cham. https://doi.org/10.1007/978-3-031-43987-2_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43987-2_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43986-5

  • Online ISBN: 978-3-031-43987-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships