Skip to main content
SpringerLink
Log in

Robust co-teaching learning with consistency-based noisy label correction for medical image classification

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Deep neural networks (DNNs) have made great achievements in computer-aided diagnostic systems, but the success highly depends on massive data with high-quality labels. However, for many medical image datasets, a considerable number of noisy labels are introduced by inter- and intra-observer variability, thus hampering DNNs’ performance. To address this problem, a robust noisy label correction method with the co-teaching learning paradigm is proposed.

Methods

The proposed method aims to reduce the effect of noisy labels by correcting or removing them. It consists of two modules. An adaptive noise rate estimation module is employed to calculate the dataset’s noise rate, which is helpful to detect noisy labels but is usually unavailable in clinical applications. A consistency-based noisy label correction module aims to detect noisy labels and correct them to reduce the disturbance from noisy labels and exploit useful information in data.

Results

Experiments are conducted on the public skin lesion dataset ISIC-2017, ISIC-2019, and our constructed thyroid ultrasound image dataset. The results demonstrate that the proposed method outperforms other noisy label learning methods in medical image classification tasks. It is also evaluated on the natural image dataset CIFAR-10 to show its generalization.

Conclusion

This paper proposes a noisy label correction method to handle noisy labels in medical image datasets. Experimental results show that it can self-adapt to different datasets and efficiently correct noisy labels, which is suitable for medical image classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.isic-archive.com.

References

  1. Park VY, Han K, Seong YK, Park MH, Kim EK, Moon HJ, Yoon JH, Kwak JY (2019) Diagnosis of thyroid nodules: performance of a deep learning convolutional neural network model versus radiologists. Sci Rep 9(1):1–9

    Google Scholar 

  2. Wang L, Zhang L, Zhu M, Qi X, Yi Z (2020) Automatic diagnosis for thyroid nodules in ultrasound images by deep neural networks. Medical Image Anal 61:101665

    Article  Google Scholar 

  3. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Park S, Park S, Choi Y, Kim D, Son E, Lee H, Yoon J, Kim EK, Moon H, Kwak J (2012) Interobserver variability and diagnostic performance in us assessment of thyroid nodule according to size. Ultraschall in der Medizin-Eur J Ultrasound 33(07):E186–E190

    Article  CAS  Google Scholar 

  5. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O (2021) Understanding deep learning (still) requires rethinking generalization. Commun ACM 64(3):107–115

    Article  Google Scholar 

  6. Dgani Y, Greenspan H, Goldberger J(2018) Training a neural network based on unreliable human annotation of medical images. In: 2018 IEEE 15th International symposium on biomedical imaging (ISBI 2018), pp 39–42

  7. Xue C, Dou Q, Shi X, Chen H, Heng PA (2019) Robust learning at noisy labeled medical images: Applied to skin lesion classification. In: 2019 IEEE 16th International symposium on biomedical imaging (ISBI 2019), pp 1280–128

  8. Xue C, Yu L, Chen P, Dou Q, Heng PA (2022) Robust medical image classification from noisy labeled data with global and local representation guided co-training. IEEE Trans Med Imaging. https://doi.org/10.1109/TMI.2021.3140140

    Article  PubMed  Google Scholar 

  9. Javadi G, Samadi S, Bayat S, Sojoudi S, Hurtado A, Eshumani W, Chang S, Black P, Mousavi P, Abolmaesumi P (2022) Training deep neural networks with noisy clinical labels: toward accurate detection of prostate cancer in US data. Int J CARS 17:1697–1705. https://doi.org/10.1007/s11548-022-02707-y

    Article  Google Scholar 

  10. Sukhbaatar S, Bruna J, Paluri M, Bourdev L, Fergus R (2015) Training convolutional networks with noisy labels. In: 3rd International conference on learning representations, pp 1-11

  11. Xiao T, Xia T, Yang Y, Huang C, Wang X (2015) Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2691–2699

  12. Goldberger J, Ben-Reuven E (2017) Training deep neural-networks using a noise adaptation layer. In: International conference on learning representations, pp 1-9

  13. Jiang L, Zhou Z, Leung T, Li LJ, Fei-Fei L (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International conference on machine learning, pp 2309–2318

  14. Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I, Sugiyama M (2018) Co-teaching:Robust training of deep neural networks with extremely noisy labels. In: Advances in neural information processing systems, pp 8527–8537

  15. Huang J, Qu L, Jia R, Zhao B (2019) O2u-net: a simple noisy label detection approach for deep neural networks. In: Proceedings of the IEEE/CVF International conference on computer vision, pp 3326–3334

  16. Chen P, Liao BB, Chen G, Zhang S (2019) Understanding and utilizing deep neural networks trained with noisy labels. In: International conference on machine learning, pp 1062–1070

  17. Wei H, Feng L, Chen X, An B (2020) Combating Noisy Labels by Agreement: A Joint Training Method with Co-Regularization. Proceedings of the IEEE computer society conference on computer vision and pattern recognition pp 13723–13732

  18. Tanaka D, Ikami D, Yamasaki T, Aizawa K(2018) Joint optimization framework for learning with noisy labels. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5552–5560

  19. Yi K, Wu J (2019) Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 7017–7025

  20. Arpit D, Jastrzębski S, Ballas N, Krueger D, Bengio E, Kanwal MS, Maharaj T, Fischer A, Courville A, Bengio Y, Lacoste-Julien S (2017) A closer look at memorization in deep networks. In: Precup D, Teh YW (eds) Proceedings of the 34th International conference on machine learning, proceedings of machine learning research. 70: 233–242

  21. Codella NCF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th International symposium on biomedical imaging (ISBI 2018), pp 168–172

  22. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5(1):1–9

    Article  Google Scholar 

  23. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Handbook of systemic autoimmune diseases 1(4)

  24. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  25. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255

  26. Chollet F(2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258

  27. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, Springer, pp 630–645

Download references

Acknowledgements

This work was supported by the National Science Fund for Distinguished Young Scholars under Grants No. 62025601, the National Natural Science Foundation of China under Grants No. 62006164, and the Sichuan University Innovation Spark Project Library under Grants No. 2018SCUH0040.

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Chengdu, 610065, Sichuan, China

    Minjuan Zhu, Lei Zhang, Lituan Wang, Dong Li, Jianwei Zhang & Zhang Yi

Authors
  1. Minjuan Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lituan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

For this type of study, formal consent is not required.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, M., Zhang, L., Wang, L. et al. Robust co-teaching learning with consistency-based noisy label correction for medical image classification. Int J CARS 18, 675–683 (2023). https://doi.org/10.1007/s11548-022-02799-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02799-6

Keywords

Search

Navigation