Abstract
Deep learning has made significant achievements in the field of medical image processing. To train a robust model with strong generalization, a large-scale, high-quality dataset with balanced categories and correct labels is required. However, most datasets follow a long-tail distribution that some classes occupy most of the data, and other classes have only a few samples. At the same time, incorrect labels exist in the datasets. The existing methods focus on solving only one of these two problems, such as Focal Loss for class imbalance and mean-absolute error loss function for noisy labels. However, methods that try to alleviate one of the problems will aggravate the other. In order to tackle the class imbalance while avoids fitting the noisy labels, we propose a novel Batch Adaptation Weighted (BAW) loss. It uses the loss weights of known samples to guide the direction of network optimization for next batch training. BAW is easy to implement and can be extended to various deep networks to improve accuracy without any extra cost. We evaluate BAW on a general natural image dataset, CIFAR-10, and verify it on a large-scale medical image dataset, ChestX-ray14. Compared with existing algorithms, BAW gets best results on both datasets. Experiments shows that our algorithm can solve the problem of class imbalance and noisy labels at the same time. The code of our project is available at https://github.com/pansiyuan123/chestnet.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bottou L (2012) Stochastic gradient descent tricks neural networks: tricks of the trade
Chawla N, Japkowic N, Kotcz A, Japkowicz N (2004) Editorial: special issues on learning from imbalanced data sets. Ann Nucl Energy 36(3):255–257
Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2011) Smote: Synthetic minority over-sampling technique
Dan H, Lee K, Mazeika M (2019) Using pre-training can improve model robustness and uncertainty
Ding Y, Wang L, Fan D, Gong B (2018) A semi-supervised two-stage approach to learning from noisy labels. In: 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1215–1224
Durand T, Mordan T, Thome N, Cord M (2017) Wildcat: weakly supervised learning of deep convnets for image classification, pointwise localization and segmentation. In: IEEE conference on computer vision pattern recognition
Fawcett T (2006) An introduction to ROC analysis[J]. Pattern recognition letters 27(8):861–874
Ghosh A, Kumar H, Sastry PS (2017) Robust loss functions under label noise for deep neural networks
Guendel S, Grbic S, Georgescu B, Zhou K, Comaniciu D (2018) Learning to recognize abnormalities in chest x-rays with location-aware dense networks. Springer, Cham
Haynes D, Corns S, Venayagamoorthy GK (2012) An exponential moving average algorithm. In: Evolutionary computation
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Huang G, Liu Z, Laurens V, Weinberger KQ (2016) Densely connected convolutional networks. IEEE Computer Society
Huang J, Qu L, Jia R, Zhao B (2019) O2u-net: a simple noisy label detection approach for deep neural networks. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 3325–3333. https://doi.org/10.1109/ICCV.2019.00342
Hu J, Shen L, Sun G, Albanie S (2018) Squeeze-and-excitation networks[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Jindal I, Nokleby M, Pressel D, Chen X (2019) A nonlinear, noise-aware, quasi-clustering approach to learning deep cnns from noisy labels. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 64–72
Kim Y, Lee Y, Jeon M (2021) Imbalanced image classification with complement cross entropy
Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Master’s thesis, Department of Computer Science, University of Toronto
Lecun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436
Li Z, Wang C, Han M, Xue Y, Wei W, Li LJ, Fei-fei L (2017) Thoracic disease identification and localization with limited supervision
Li B, Liu Y, Wang X (2018) Gradient harmonized single-stage detector
Lin TY, Maire M, Belongie S, Hays J, Zitnick CL (2014) Microsoft coco: common objects in context. In: European conference on computer vision
Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. IEEE Transactions on Pattern Analysis Machine Intelligence PP(99):2999–3007
Liu XY (2006) Exploratory under-sampling for class-imbalance learning. In: International conference on data mining
Northcutt CG, Jiang L, Chuang IL (2021) Confident learning: estimating uncertainty in dataset labels. J Artif Intell Res (JAIR) 70:1373–1411
Patrini G, Rozza A, Menon AK, Nock R, Qu L (2017) Making deep neural networks robust to label noise: a loss correction approach. In: IEEE conference on computer vision pattern recognition
Rajpurkar P, Irvin J, Zhu K, Yang B, Mehta H, Duan T, Ding D, Bagul A, Langlotz C, Shpanskaya K (2017) Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. IEEE
Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge[J]. International journal of computer vision 115(3):211–252
Szegedy C, Wei L, Jia Y, Sermanet P, Rabinovich A (2014) Going deeper with convolutions. IEEE Computer Society
Tanaka D, Ikami D, Yamasaki T, Aizawa K (2018) Joint optimization framework for learning with noisy labels. IEEE
Vahdat A (2017) Toward robustness against label noise in training deep discriminative neural networks
Voets M, Mllersen K, Bongo LA (2018) Replication study: development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. PLos ONE
Wallace BC, Small K, Brodley CE, Trikalinos TA (2012) Class imbalance, redux. In: IEEE international conference on data mining
Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. IEEE
Yao J, Wang J, Tsang IW, Zhang Y, Sun J, Zhang C, Zhang R (2018) Deep learning from noisy image labels with quality embedding. IEEE Trans Image Process 28(4):1909–1922
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: CVPR
Zhou B, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition[J]. IEEE transactions on pattern analysis and machine intelligence 40(6):1452–1464
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of Interests
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Pan, S., Sheng, B., He, G. et al. BAW: learning from class imbalance and noisy labels with batch adaptation weighted loss. Multimed Tools Appl 81, 13593–13610 (2022). https://doi.org/10.1007/s11042-022-12323-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-12323-2