DeCAB: Debiased Semi-supervised Learning for Imbalanced Open-Set Data

Abstract

Semi-supervised learning (SSL) has received significant attention due to its ability to use limited labeled data and various unlabeled data to train models with high generalization performance. However, the assumption of a balanced class distribution in traditional SSL approaches limits a wide range of real applications, where the training data exhibits long-tailed distributions. As a consequence, the model is biased towards head classes and disregards tail classes, thereby leading to severe class-aware bias. Additionally, since the unlabeled data may contain out-of-distribution (OOD) samples without manual filtering, the model will be inclined to assign OOD samples to non-tail classes with high confidence, which further overwhelms the tail classes. To alleviate this class-aware bias, we propose an end-to-end semi-supervised method Debias Class-Aware Bias (DeCAB). DeCAB introduces positive-pair scores for contrastive learning instead of positive-negative pairs based on unreliable pseudo-labels, avoiding false negative pairs negatively impacts the feature space. At the same time, DeCAB utilizes class-aware thresholds to select more tail samples and selective sample reweighting for feature learning, preventing OOD samples from being misclassified as head classes and accelerating the convergence speed of the model. Experimental results demonstrate that DeCAB is robust in various semi-supervised benchmarks and achieves state-of-the-art performance. Our code is temporarily available at https://github.com/xlhuang132/decab.

This work was supported in part by the National Natural Science Foundation of China under Grants 62002302, 62306181; in part by the FuXiaQuan National Independent Innovation Demonstration Zone Collaborative Innovation Platform under Grant 3502ZCQXT2022008; in part by the China Fundamental Research Funds for the Central Universities under Grants 20720230038.

Appendices

Appendix

A Analysis of the Effect of OOD Data to SSL Methods

Fig. 4.

Confusion matrices of SSL methods on the testing set of CIFAR-10 under two scenarios. The training set of CIFAR-10 is utilized as the labeled and unlabeled ID data with an imbalance factor of 100, and the testing set of Tiny ImageNet (TIN) is used as OOD data. (a) shows the case without OOD data, while (b) shows the case with OOD data.

To reveal the fact that the OOD samples exacerbate the long-tail problem in existing SSL methods, we conduct a quick experiment. In detail, we consider a scenario in which the labeled data is characterized by a long-tailed distribution, with a small number of classes containing a disproportionate number of samples, while the vast majority of classes have a limited number of samples. In addition, the unlabeled data comprises both ID and OOD samples. The unlabeled ID samples follow the same class distribution and a similar long-tailed distribution to the labeled data, while the OOD samples do not belong to any of the ID classes. We conduct a quick experiment to demonstrate that the performance of existing SSL methods deteriorates when confronted with OOD data. To evaluate the model performance under various scenarios, we manipulate the imbalanced factor as well as the inclusion of OOD samples to simulate different settings. Figure 4 compares the confusion matrices of SSL methods on imbalanced training data with and without OOD data. The ID data uses the training set of CIFAR-100 with an imbalance factor of 100, while OOD data uses the testing set of Tiny ImageNet (TIN). It can be seen from the figure that the long-tailed problem leads to performance degradation on the tail class because many tail class samples are misclassified as a head class. The presence of OOD samples exacerbates the long-tailes problem for existing SSL methods.

B Algorithm Flowchart

The algorithm of DeCAB is shown in Algorithm 1.

C Visualized Comparison

In order to evaluate the learning of the model on the feature space, we perform a visualized comparative analysis of the test set features extracted from the backbone of the model obtained by each method. Figure 5 shows the t-SNE visualization of feature space about the testing set of CIFAR-10, where the model is trained on CIFAR-10-LT (IF=100, TIN). In the figure, the black circle circles the space of the easily confused head and tail classes. The feature space that is learned by other methods typically shows an aggregation of similar head and tail classes, with a significant proportion of misclassified tail classes located in the middle of the black circles. In contrast, DeCAB exhibits a much clearer separation between head and tail classes in the feature space, resulting in fewer samples being misclassified in the middle. These results illustrate that our method can obtain a better feature space.

Fig. 5.

The t-SNE visualization of feature space of CIFAR-10-LT test set, trained on CIFAR-10-LT with IF=100 and Tiny ImageNet as OOD data. The black circle circles the easily confused head and tail samples.