Abstract
Objects are often organized in a hierarchy where coarse-grained categories are comprised of subordinate fine-grained classes. Comparing with the fine-grained labels, the coarse-grained labels are much affordable to obtain. The coarse-grained labels can boost the semi-supervised learning (SSL) by offering extra regularization on the feature space of finer-grained recognition. However, coarse-grained labels are ignored by most of works in SSL. An intuitive way to utilize the coarse labels for SSL is to impose an extra coarse-grained categorization constraint, which will cause the class confusion between fine-grained categories belonging to the same coarse-grained category thus is sub-optimal for SSL. In this paper, we present an instance-proxy loss (IPL) to boost the separability of the fine-grained classes within the same coarse-grained class, as well as keep the intra-class feature space of coarse-grained classes compact. Specifically, IPL includes instance-level loss and proxy-level loss to impose constraints on both instance-to-instance and instance-to-proxy relations. Our approach outperforms the state-of-the-art methods on three benchmark datasets, showing significant improvement with small proportion of fine-grained labels, e.g., it brings 10.14% accuracy improvement on CUB-200-2011 with 15% of labeled data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Berthelot, D., et al.: Remixmatch: semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785 (2019)
Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., Raffel, C.A.: Mixmatch: a holistic approach to semi-supervised learning. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Bukchin, G., et al.: Fine-grained angular contrastive learning with coarse labels. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8730–8740 (2021)
Chang, D., Pang, K., Zheng, Y., Ma, Z., Song, Y.Z., Guo, J.: Your “flamingo” is my “bird”: fine-grained, or not. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11476–11485 (2021)
Chen, T., Wu, W., Gao, Y., Dong, L., Luo, X., Lin, L.: Fine-grained representation learning and recognition by exploiting hierarchical semantic embedding. In: Proceedings of the 26th ACM International Conference on Multimedia, pp. 2023–2031 (2018)
Chen, T., Kornblith, S., Swersky, K., Norouzi, M., Hinton, G.: Big self-supervised models are strong semi-supervised learners. In: NeurIPS (2020)
Cui, J., Zhong, Z., Liu, S., Yu, B., Jia, J.: Parametric contrastive learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 715–724 (2021)
Garg, A., Bagga, S., Singh, Y., Anand, S.: Hiermatch: leveraging label hierarchies for improving semi-supervised learning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1015–1024 (2022)
Hu, Z., Yang, Z., Hu, X., Nevatia, R.: Simple: similar pseudo label exploitation for semi-supervised classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15099–15108 (2021)
Khosla, P., et al.: Supervised contrastive learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 18661–18673. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper/2020/file/d89a66c7c80a29b1bdbab0f2a1a94af8-Paper.pdf
Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3D object representations for fine-grained categorization. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 554–561 (2013). https://doi.org/10.1109/ICCVW.2013.77
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: ICLR (2017)
Lee, D.H., et al.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, vol. 3, p. 896 (2013)
Loh, C., et al.: On the importance of calibration in semi-supervised learning. arXiv abs/2210.04783 (2022)
Maji, S., Kannala, J., Rahtu, E., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. Technical report (2013)
Mugnai, D., Pernici, F., Turchini, F., Del Bimbo, A.: Fine-grained adversarial semi-supervised learning. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 18(1s), 1–19 (2022)
Sohn, K., et al.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. In: Advances in Neural Information Processing Systems, vol. 33, pp. 596–608 (2020)
Su, J.C., Cheng, Z., Maji, S.: A realistic evaluation of semi-supervised learning for fine-grained classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12966–12975 (2021)
Su, J.C., Maji, S.: The semi-supervised inaturalist challenge at the fgvc8 workshop (2021)
Su, J.C., Maji, S.: Semi-supervised learning with taxonomic labels. In: British Machine Vision Conference (2021)
Tarvainen, A., Valpola, H.: Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: NeurIPS (2017)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD birds-200-2011 dataset (2011)
Wang, W., Lin, L., Fan, Z., Liu, J.: Semi-supervised learning for mars imagery classification and segmentation. ACM Trans. Multimed. Comput. Commun. Appl. 19(4), 1–23 (2023)
Wang, X., Gao, J., Long, M., Wang, J.: Self-tuning for data-efficient deep learning. In: International Conference on Machine Learning, pp. 10738–10748. PMLR (2021)
Wu, H., Guo, H., Miao, Q., Huang, M., Wang, J.: Graph neural networks based multi-granularity feature representation learning for fine-grained visual categorization. In: Þór Jónsson, B., et al. (eds.) MMM 2022. LNCS, vol. 13142, pp. 230–242. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98355-0_20
Xie, Q., Dai, Z., Hovy, E., Luong, T., Le, Q.: Unsupervised data augmentation for consistency training. In: Advances in Neural Information Processing Systems, vol. 33, pp. 6256–6268 (2020)
Xu, Y., Qian, Q., Li, H., Jin, R., Hu, J.: Weakly supervised representation learning with coarse labels. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10593–10601 (2021)
Zagoruyko, S., Komodakis, N.: Wide residual networks. CoRR abs/1605.07146 (2016)
Zhang, B., et al.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. In: Advances in Neural Information Processing Systems, vol. 34, pp. 18408–18419 (2021)
Zhao, J., Liu, X., Zhao, W.: Balanced and accurate pseudo-labels for semi-supervised image classification. ACM Trans. Multimed. Comput. Commun. Appl. 18(3s), 1–18 (2022)
Acknowledgement
This work was supported by National Key R & D Program of China under Grant No.2021ZD0110400, National Natural Science Foundation of China (No.62276260, 62002356, 62271485, 62076235) and Zhejiang Lab (No.2021KH0AB07).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, H., Miao, Q., Guo, H., Huang, M., Wang, J. (2024). Instance-Proxy Loss for Semi-supervised Learning with Coarse Labels. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14436. Springer, Singapore. https://doi.org/10.1007/978-981-99-8555-5_19
Download citation
DOI: https://doi.org/10.1007/978-981-99-8555-5_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8554-8
Online ISBN: 978-981-99-8555-5
eBook Packages: Computer ScienceComputer Science (R0)