Abstract
Semi-supervised learning (SSL) is a successful paradigm that can use unlabelled data to alleviate the labelling cost problem in supervised learning. However, the excellent performance brought by SSL does not transfer well to the task of class imbalance. The reason is that the class bias of pseudo-labelling further misleads the decision boundary. To solve this problem, we propose a new plug-and-play approach to handle the class imbalance problem based on a theoretical extension and analysis of distribution alignment. The method, called Basis Transformation Based Distribution Alignment (BTDA), efficiently aligns class distributions while taking into account inter-class relationships.BTDA implements the basis transformation through a learnable transfer matrix, thereby reducing the performance loss caused by pseudo-labelling biases. Extensive experiments show that our proposed BTDA approach can significantly improve performance in class imbalance tasks in terms of both accuracy and recall metrics when integrated with advanced SSL algorithms. Although the idea of BTDA is not complex, it can show advanced performance on datasets such as CIFAR and SVHN.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Codes are available at https://github.com/211027128/BTDA.
- 2.
References
Berthelot, D., et al.: ReMixMatch: semi-supervised learning with distribution matching and augmentation anchoring. In: International Conference on Learning Representations (2020)
Berthelot, D., et al.: Mixmatch: a holistic approach to semi-supervised learning. Adv. Neural Inf. Process. Syst. 32 (2019)
Sohn, K., et al.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv. Neural Inf. Process. Syst. 33, 596–608 (2020)
Zhang, B., et al.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. Adv. Neural Inf. Process. Syst. 34, 18408–18419 (2021)
Hyun, M., et al.: Class-imbalanced semi-supervised learning. In: ICLR RobustML Workshop (2021)
He, J., et al.: Rethinking re-sampling in imbalanced semi-supervised learning. arXiv preprint arXiv:2106.00209 (2021)
Guo, L.-Z., Li, Y.F.: Class-imbalanced semi-supervised learning with adaptive thresholding. In: International Conference on Machine Learning. PMLR (2022)
Netzer, Y., et al.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning (2011)
Krizhevsky, A., et al.: Learning multiple layers of features from tiny images. Technical Report, University of Toronto (2009)
Zagoruyko, S., Komodakis, N.: Wide residual networks. In: British Machine Vision Conference (2016)
Sutskever, I., et al.: On the importance of initialization and momentum in deep learning. In: International Conference on Machine Learning. PMLR (2013)
Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1–17 (1964)
Abuduweili, A., et al.: Adaptive consistency regularization for semi-supervised transfer learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)
Fan, Y., Kukleva, A., Dai, D., et al.: Revisiting consistency regularization for semi-supervised learning. Int. J. Comput. Vision 131(3), 626–643 (2023)
Chen, B., et al.: Debiased self-training for semi-supervised learning. Adv. Neural Inf. Process. Syst. (2022)
Long, J., et al.: A novel self-training semi-supervised deep learning approach for machinery fault diagnosis. Int. J. Prod. Res. 61, 1–14 (2022)
Zhang, Y., et al.: Deep long-tailed learning: a survey. IEEE Trans. Pattern Anal. Mach. Intell. (2023)
Zhang, C., et al.: An empirical study on the joint impact of feature selection and data resampling on imbalance classification. Appl. Intell. 53(5), 5449–5461 (2023)
Wang, W., et al.: Imbalanced adversarial training with reweighting. In: 2022 IEEE International Conference on Data Mining (ICDM). IEEE (2022)
Li, J., Liu, Y., Li, Q.: Generative adversarial network and transfer-learning-based fault detection for rotating machinery with imbalanced data condition. Meas. Sci. Technol. 33(4), 045103 (2022)
Shi, Y., et al.: Improving imbalanced learning by pre-finetuning with data augmentation. In: Fourth International Workshop on Learning with Imbalanced Domains: Theory and Applications. PMLR (2022)
Bonner, S., et al.: Implications of topological imbalance for representation learning on biomedical knowledge graphs. Brief. Bioinf. 23(5), bbac279 (2022)
Gouabou, A.C.F., et al.: Rethinking decoupled training with bag of tricks for long-tailed recognition. In: 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA). IEEE (2022)
Acknowledgement
This research was supported by National Natural Science Foundation of China (61972187); Open Project of Key Laboratory of Medical Big Data Engineering in Fujian Province (KLKF202301); R &d Plan of Guangdong Province in key areas (2020B0101090005); the specific research fund of The Innovation Platform for Academician of Hainan Province (YSPTZX202145); Fujian Provincial Science and Technology Department Guided Project (2022H0012).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ye, J., Wu, J., Li, Z., Zheng, X. (2024). Rethinking Distribution Alignment for Inter-class Fairness. In: Vaidya, J., Gabbouj, M., Li, J. (eds) Artificial Intelligence Security and Privacy. AIS&P 2023. Lecture Notes in Computer Science, vol 14510. Springer, Singapore. https://doi.org/10.1007/978-981-99-9788-6_2
Download citation
DOI: https://doi.org/10.1007/978-981-99-9788-6_2
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9787-9
Online ISBN: 978-981-99-9788-6
eBook Packages: Computer ScienceComputer Science (R0)