Abstract
Training datasets for deep models inevitably contain noisy labels, such labels can seriously impair the performance of deep models. Empirically, all labels will be remembered after enough epochs, while pure labels will be remembered first and then noise labels. Inspired by this, we propose a new deep framework named “Discerning Coteaching” (DC). It can automatically identify noise and pure labels during training without additional prior knowledge. Specifically we train two networks at the same time, each network will contain an additional categorical cross entropy (CCE) loss. Then a threshold is dynamically selected based on the CCE, samples with a loss value greater than the threshold will be discarded directly, the rest will be sent to its peer network for updating. We validate the framework on Cifar10 and UCMD, and the results reveal that DC has a positive effect in dealing with noisy labels.
This work was in part supported by the National Natural Science Foundation of China under Grant no. 61801222, and in part supported by the Fundamental Research Funds for the Central Universities under Grant no. 30919011230, and in part supported by the Fundamental Research Funds for the Central Universities under Grant no. JSGP202204, and in part supported by the Natural Science Foundation of Shandong Province under Grant No. ZR2021MF039.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Korot, E., Guan, Z., Ferraz, D., et al.: Code-free deep learning for multi-modality medical image classification. Nat. Mach. Intell. 3(4), 288–298 (2021)
Lei, Z., Zeng, Y., Liu, P., et al.: Active deep learning for hyperspectral image classification with uncertainty learning. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021)
Wu, X., Sahoo, D., Hoi, S.C.H.: Recent advances in deep learning for object detection. Neurocomputing 396, 39–64 (2020)
Wang, W., Lai, Q., Fu, H., et al.: Salient object detection in the deep learning era: an in-depth survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 3239–3259 (2021)
Khan, M.Z., et al.: Deep neural architectures for medical image semantic segmentation. IEEE Access 9, 83002–83024 (2021)
Han, B., et al.: Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems, vol. 31 (2018)
Wang, Y., et al.: Symmetric cross entropy for robust learning with noisy labels. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
Ma, X., et al.: Normalized loss functions for deep learning with noisy labels. In: International Conference on Machine Learning. PMLR (2020)
Yu, X., et al.: How does disagreement help generalization against label corruption?. In: International Conference on Machine Learning. PMLR (2019)
Lyu, Y., Tsang, I.W.: Curriculum loss: robust learning and generalization against label corruption. arXiv preprint arXiv:1905.10045 (2019)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images, vol. 7 (2009)
Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems (2010)
Huang, G., et al.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Sukhbaatar, S., Fergus, R.: Learning from noisy labels with deep neural networks. arXiv preprint arXiv:1406.20802(3) (2014)
Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Feurer, M., et al.: Efficient and robust automated machine learning. In: Advances in Neural Information Processing Systems, vol. 28 (2015)
Han, B., et al.: Masking: a new perspective of noisy supervision. arXiv preprint arXiv:1805.08193 (2018)
Hendrycks, D., Mazeika, M., Wilson, D., Gimpel, K.: Using trusted data to train deep networks on labels corrupted by severe noise. arXiv preprint arXiv:1802.05300 (2018)
Xiao, T., et al.: Learning from massive noisy labeled data for image classification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015)
Vahdat, A.: Toward robustness against label noise in training deep discriminative neural networks. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Sohn, K., et al.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. In: Advances in Neural Information Processing Systems, vol. 30, pp. 596–608 (2020)
Lee, D.-H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, vol. 3, no. 2 (2013)
Zhang, B., et al.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wei, Q., Fu, P., Zhang, P., Wang, T. (2022). Discerning Coteaching: A Deep Framework for Automatic Identification of Noise Labels. In: Yu, S., et al. Pattern Recognition and Computer Vision. PRCV 2022. Lecture Notes in Computer Science, vol 13535. Springer, Cham. https://doi.org/10.1007/978-3-031-18910-4_54
Download citation
DOI: https://doi.org/10.1007/978-3-031-18910-4_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-18909-8
Online ISBN: 978-3-031-18910-4
eBook Packages: Computer ScienceComputer Science (R0)