Abstract
Consistency regularization has witnessed tremendous success in the area of semi-supervised deep learning for image classification, which leverages data augmentation on unlabeled examples to encourage the model outputting the invariant predicted class distribution as before augmented. These methods have been made considerable progress in this area, but most of them are at the cost of utilizing more complex models. In this work, we propose a simple and efficient method FMixAugment, which combines the proposed MixAugment with Fourier space-based data masking and applies it on unlabeled examples to generate a strongly-augmented version. Our approach first generates a hard pseudo-label by employing a weakly-augmented version and minimizes the cross-entropy between it and the strongly-augmented version. Furthermore, to improve the robustness and uncertainty measurement of the model, we also enforce consistency constraints between the mixed augmented version and the weakly-augmented version. Ultimately, we introduce a dynamic growth of the confidence threshold for pseudo-labels. Extensive experiments are tested on CIFAR-10/100, SVHN, and STL-10 datasets, which indicate that our method outperforms the previous state-of-the-art methods. Specifically, with 40 labeled examples on CIFAR-10, we achieve 90.21% accuracy, and exceed 95% accuracy with 1000 labeled examples on STL-10.
H. Lin—Student.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Change history
22 October 2021
A co-author (Shiping Wang) was not included in the original publication due to an oversight. This co-author has been added in this correction, reflecting the original submission of the article.
Notes
- 1.
- 2.
http://ufldl.stanford.edu/housenumbers/{}_32x32.mat.
- 3.
- 4.
References
Xu, N., Liu, Y.-P., Geng, X.: Partial multi-label learning with label distribution. In: Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, vol. 34, pp. 6510–6517 (2020)
Chen, L.-Z., Lin, Z., Wang, Z., Yang, Y.-L., Cheng, M.-M.: Spatial information guided convolution for real-time RGBD semantic segmentation. IEEE Trans. Image Process. 30, 2313–2324 (2021)
Yang, X., Yan, J.: Arbitrary-oriented object detection with circular smooth label. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12353, pp. 677–694. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_40
Wang, Q., Gao, J., Yuan, Y.: A joint convolutional neural networks and context transfer for street scenes labeling. IEEE Trans. Intell. Transp. Syst. 19(5), 1457–1470 (2017)
Hendrycks, D., Mu, N., Cubuk, E.D., Zoph, B., Gilmer, J., Lakshminarayanan, B.: Augmix: a simple data processing method to improve robustness and uncertainty. In: Proceedings of the Eighth International Conference on Learning Representations (2020)
Harris, E., Marcu, A., Painter, M., Niranjan, M., Prugel-Bennett, A., Hare, J.: Fmix: enhancing mixed sample data augmentation, arXiv preprint arXiv:2002.12047, vol. 2, no. 3, p. 4 (2020)
Bachman, P., Alsharif, O., Precup, D.: Learning with pseudo-ensembles. In: Proceedings of the Twenty-Eighth Conference on Advances in Neural Information Processing Systems, pp. 3365–3373 (2014)
Rasmus, A., Valpola, H., Honkala, M., Berglund, M., Raiko, T.: Semi-supervised learning with ladder networks. In: Proceedings of the Twenty-Ninth Conference on Advances in Neural Information Processing Systems, pp. 3546–3554 (2015)
Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: Proceedings of the Thirty-First Conference on Advances in Neural Information Processing Systems, pp. 1195–1204 (2017)
Miyato, T., Maeda, S., Koyama, M., Ishii, S.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2019)
Verma, V., Lamb, A., Kannala, J., Bengio, Y., Lopez-Paz, D.: Interpolation consistency training for semi-supervised learning. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 3635–3641 (2019)
Xie, Q., Dai, Z., Hovy, E.H., Luong, T., Le, Q.: Unsupervised data augmentation for consistency training. In: Proceedings of the Thirty-Fourth Conference on Advances in Neural Information Processing Systems (2020)
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: Mixmatch: a holistic approach to semi-supervised learning. In: Proceedings of the Thirty-Third Conference on Advances in Neural Information Processing Systems, pp. 5049–5059 (2019)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. In: Proceedings of the Sixth International Conference on Learning Representations (2018)
Berthelot, D., et al.: Remixmatch: semi-supervised learning with distribution matching and augmentation anchoring. In: Proceedings of the Eighth International Conference on Learning Representations (2020)
Sohn, K., et al.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. In: Proceedings of the Thirty-Fourth Conference on Advances in Neural Information Processing Systems (2020)
Xie, Q., Luong, M.-T., Hovy, E., Le, Q.V.: Self-training with noisy student improves imagenet classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10687–10698 (2020)
Cui, S., Wang, S., Zhuo, J., Li, L., Huang, Q., Tian, Q.: Towards discriminability and diversity: batch nuclear-norm maximization under label insufficient situations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3941–3950 (2020)
Yun, S., Han, D., Chun, S., Oh, S.J., Yoo, Y., Choe, J.: Cutmix: regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6022–6031 (2019)
DeVries, T., Taylor, G.W.: Improved regularization of convolutional neural networks with cutout, arXiv preprint arXiv:1708.04552 (2017)
Oliver, A., Odena, A., Raffel, C.A., Cubuk, E.D., Goodfellow, I.: Realistic evaluation of deep semi-supervised learning algorithms. In: Proceedings of the Thirty-Second Conference on Advances in Neural Information Processing Systems, pp. 3235–3246 (2018)
Acknowledgments
This work is in part supported by the National Natural Science Foundation of China (Grant No. U1705262), the Natural Science Foundation of Fujian Province (Grant Nos. 2020J01130193 and 2018J07005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, H., Wang, S., Liu, Z., Xiao, S., Du, S., Guo, W. (2021). FMixAugment for Semi-supervised Learning with Consistency Regularization. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13020. Springer, Cham. https://doi.org/10.1007/978-3-030-88007-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-88007-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88006-4
Online ISBN: 978-3-030-88007-1
eBook Packages: Computer ScienceComputer Science (R0)