Skip to main content
SpringerLink
Log in

Ensemble diversified learning for image classification with noisy labels

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work, we develop a new approach for learning a deep neural network for image classification with noisy labels using ensemble diversified learning. We first partition the training set into multiple subsets with diversified image characteristics. For each subset, we train a separate deep neural network image classifier. These networks are then used to encode the input image into different feature vectors, providing diversified observations of the input image. The encoded features are then fused together and further analyzed by a decision network to produce the final classification output. We study image classification on noisy labels with and without the access to clean samples. Our extensive experimental results on the CIFAR-10 and MNIST datasets demonstrate that our proposed method outperforms existing methods by a large margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Ahmed A, Yousif H, Kays R, He Z (2020) Animal species classification using deep neural networks with noise labels. Eco Inform 57:101063

    Article  Google Scholar 

  2. Azadi S, Feng J, Jegelka S, Darrell T (2015) Auxiliary image regularization for deep cnns with noisy labels. arXiv preprint arXiv:1511.07069

  3. Barandela R, Gasca E (2000) Decontamination of training samples for supervised pattern recognition methods. In: Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). Springer, Berlin, Heidelberg, pp 621–630

  4. Beigman E, Klebanov BB (2009) Learning with annotation noise. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp 280–287

  5. Brodley CE, Friedl MA (1999) Identifying mislabeled training data. J Artif Intell Res 11:131–167

    Article  Google Scholar 

  6. Chu J, Guo Z, Leng L (2018) Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access 6:19959–19967

    Article  Google Scholar 

  7. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. IEEE, Piscataway

  8. Ding Y, Wang L, Fan D, Gong B (2018) A semi-supervised two-stage approach to learning from noisy labels. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Piscataway, pp 1215–1224

  9. Fawzi A, Moosavi-Dezfooli SM, Frossard P (2017) The robustness of deep networks: A geometrical perspective. IEEE Signal Process Mag 34(6):50–62

    Article  Google Scholar 

  10. Fergus R, Fei-Fei L, Perona P, Zisserman A (2010) Learning object categories from internet image searches. Proc IEEE 98(8):1453-66

  11. Ghosh A, Manwani N, Sastry PS (2015) Making risk minimization tolerant to label noise. Neurocomputing 160:93–107

    Article  Google Scholar 

  12. Han B, Yao Q, Yu X, Niu G, Xu M, Hu W, Tsang I, Sugiyama M (2018) Co-teaching: Robust training of deep neural networks with extremely noisy labels. Adv Neural Inf Proces Syst 31:8527–8537

    Google Scholar 

  13. Han B, Niu G, Yao J, Yu X, Xu M, Tsang I, Sugiyama M (2018) Pumpout: A meta approach to robust deep learning with noisy labels. arXiv preprint arXiv:1809.11008.

  14. Hendrycks D, Mazeika M, Wilson D, Gimpel K (2018) Using trusted data to train deep networks on labels corrupted by severe noise. In: Advances in neural information processing systems, pp 10456–10465

  15. Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath TN, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  16. Jiang L, Zhou Z, Leung T, Li LJ, Fei-Fei L (2018) Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In: International Conference on Machine Learning, pp 2304–2313. PMLR.

  17. Jindal I, Nokleby M, Chen X (2016) Learning deep networks from noisy labels with dropout regularization. In: 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, Piscataway, pp 967–972

  18. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images

  19. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems

  20. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–324

  21. Lee KH, He X, Zhang L, Yang L (2018) Cleannet: Transfer learning for scalable image classifier training with label noise. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5447–5456

  22. Leng L, Zhang J (2011) Dual-key-binding cancelable palmprint cryptosystem for palmprint protection and information security. J Netw Comput Appl 34(6):1979–1989

    Article  Google Scholar 

  23. Leng L, Zhang J (2013) Palmhash code vs. palmphasor code. Neurocomputing 108:1–12

    Article  Google Scholar 

  24. Leng L, Zhang J, Chen G, Khan MK, Bai P (2011) Two dimensional PalmPhasor enhanced by multi-orientation score level fusion. In: FTRA International Conference on Secure and Trust Computing, Data Management, and Application. Springer, Berlin, Heidelberg, pp 122–129

  25. Leng L, Li M, Teoh AB (2013) Conjugate 2DPalmHash code for secure palm-print-vein verification. In: 2013 6th International Congress on Image and Signal Processing (CISP), vol 3. IEEE, Piscataway, pp 1705–1710

  26. Leng L, Teoh AB, Li M, Khan MK (2014) A remote cancelable palmprint authentication protocol based on multi-directional two‐dimensional PalmPhasor‐fusion. Secur Commun Netw 7(11):1860–1871

    Article  Google Scholar 

  27. Leng L, Li M, Kim C, Bi X (2017) Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed Tools Appl 76(1):333–354

    Article  Google Scholar 

  28. Li Y, Yang J, Song Y, Cao L, Luo J, Li LJ (2017) Learning from noisy labels with distillation. In: Proceedings of the IEEE International Conference on Computer Vision, pp 1910–1918

  29. Li J, Wong Y, Zhao Q, Kankanhalli MS (2019) Learning to learn from noisy labeled data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5051–5059

  30. Manwani N, Sastry PS (2013) Noise tolerance under risk minimization. IEEE Trans Cybern 43(3):1146–1151

    Article  Google Scholar 

  31. Masnadi-Shirazi H, Vasconcelos N (2008) On the design of loss functions for classification: theory, robustness to outliers, and savageboost. Adv Neural Inf Proces Syst 21:1049–1056

    Google Scholar 

  32. Miranda AL, Garcia LP, Carvalho AC, Lorena AC (2009) Use of classification algorithms in noise detection and elimination. In: International Conference on Hybrid Artificial Intelligence Systems. Springer, Berlin, Heidelberg, pp 417–424

  33. Nettleton DF, Orriols-Puig A, Fornells A (2010) A study of the effect of different types of noise on the precision of supervised learning techniques. Artif Intell Rev 33(4):275–306

    Article  Google Scholar 

  34. Niu L, Li W, Xu D (2015) Visual recognition by learning from web data: A weakly supervised domain generalization approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2774–2783

  35. Patrini G, Rozza A, Krishna Menon A, Nock R, Qu L (2017) Making deep neural networks robust to label noise: A loss correction approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1944–1952

  36. Reed S, Lee H, Anguelov D, Szegedy C, Erhan D, Rabinovich A (2014) Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596

  37. Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149

    Article  Google Scholar 

  38. Ren M, Zeng W, Yang B, Urtasun R (2018) Learning to reweight examples for robust deep learning. arXiv preprint arXiv:1803.09050

  39. Rolnick D, Veit A, Belongie S, Shavit N (2017) Deep learning is robust to massive label noise. arXiv preprint arXiv:1705.10694

  40. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  41. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556

  42. Sukhbaatar S, Bruna J, Paluri M, Bourdev L, Fergus R (2014) Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080

  43. Sun L, Zhao C, Yan Z, Liu P, Duckett T, Stolkin R (2018) A novel weakly-supervised approach for RGB-D-based nuclear waste object detection. IEEE Sens J 19(9):3487–3500

    Article  Google Scholar 

  44. Tang Z, Yu H, Lu C, Liu P, Jin X (2019) Single-trial classification of different movements on one arm based on ERD/ERS and corticomuscular coherence. IEEE Access 7:128185–128197

    Article  Google Scholar 

  45. Vahdat A (2017) Toward robustness against label noise in training deep discriminative neural networks. In: Advances in Neural Information Processing Systems, pp 5596–5605

  46. Van Horn G, Branson S, Farrell R, Haber S, Barry J, Ipeirotis P, Perona P, Belongie S (2015) Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 595–604

  47. Van Rooyen B, Menon A, Williamson RC (2015) Learning with symmetric label noise: The importance of being unhinged. Adv Neural Inf Process Syst 28:10–18

    Google Scholar 

  48. Veit A, Alldrin N, Chechik G, Krasin I, Gupta A, Belongie S (2017) Learning from noisy large-scale datasets with minimal supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 839–847

  49. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  50. Wang Y, Liu W, Ma X, Bailey J, Zha H, Song L, Xia ST (2018) Iterative learning with open-set noisy labels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8688–8696

  51. Xiao T, Xia T, Yang Y, Huang C, Wang X (2015) Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2691–2699

  52. Yu X, Han B, Yao J, Niu G, Tsang IW, Sugiyama M (2019) How does disagreement help generalization against label corruption? arXiv preprint arXiv:1901.04215

  53. Yuan B, Chen J, Zhang W, Tai HS, McMains S (2018) Iterative cross learning on noisy labels. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Piscataway, pp 757–765

  54. Zhang Z, Sabuncu M (2018) Generalized cross entropy loss for training deep neural networks with noisy labels. Adv Neural Inf Proces Syst 31:8778–8788

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, 65211, USA

    Ahmed Ahmed & Zhihai He

  2. Department of Electrical Power Engineering, Basra Engineering Technical College, Southern Technical University, Basra, Iraq

    Hayder Yousif

Authors
  1. Ahmed Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hayder Yousif
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhihai He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhihai He.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, A., Yousif, H. & He, Z. Ensemble diversified learning for image classification with noisy labels. Multimed Tools Appl 80, 20759–20772 (2021). https://doi.org/10.1007/s11042-021-10760-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10760-z

Keywords

Search

Navigation