Abstract
Recent works in few-shot learning (FSL) have explored the incorporation of supplementary self-supervised auxiliary tasks to facilitate inductive knowledge transfer, yielding promising outcomes. Nevertheless, these approaches only optimize the shared parameters of the FSL model by minimizing a linear combination of two or more task losses, along with manually selecting the combination coefficients. Moreover, due to the unknown and intricate relationships between different tasks, such a simplistic linear combination operation is prone to inducing task conflicts, leading to adverse knowledge transfer. To tackle these challenges, we argue that in few-shot learning (FSL) augmented with auxiliary tasks, the emphasis should be laid on enhancing the performance of the primary FSL task. Specifically, to mitigate the aforementioned task conflicts, we introduce a new Gradient-biAsed Multi-task lEarning (GAME) method, which “makes the primary task primary” by considering both gradient direction and loss magnitude. Extensive experiments demonstrate that the proposed GAME method obtains substantial performance improvements over state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An, Y., Xue, H., Zhao, X., Zhang, L.: Conditional self-supervised learning for few-shot classification. In: IJCAI (2021)
Bertinetto, L., Henriques, J.F., Torr, P.H.S., Vedaldi, A.: Meta-learning with differentiable closed-form solvers. In: ICLR (2019)
Chen, Z., Badrinarayanan, V., Lee, C., Rabinovich, A.: Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In: ICML (2018)
Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML (2017)
Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting few-shot visual learning with self-supervision. In: ICCV (2019)
Gidaris, S., Komodakis, N.: Dynamic few-shot visual learning without forgetting. In: CVPR (2018)
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. In: ICLR (2018)
Guo, Y., Cheung, N.M.: Attentive weights generation for few shot learning via information maximization. In: CVPR (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Hu, S.X., Li, D., Stühmer, J., Kim, M., Hospedales, T.M.: Pushing the limits of simple pipelines for few-shot learning: External data and fine-tuning make a difference. In: CVPR (2022)
Javaloy, A., Valera, I.: Rotograd: Dynamic gradient homogenization for multi-task learning. In: ICLR (2022)
Kang, D., Cho, M.: Integrative few-shot learning for classification and segmentation. In: CVPR (2022)
Kim, J., Kim, H., Kim, G.: Model-agnostic boundary-adversarial sampling for test-time generalization in few-shot learning. In: ECCV (2020)
Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML (2015)
Laenen, S., Bertinetto, L.: On episodes, prototypical networks, and few-shot learning. In: NeurIPS (2021)
Larsson, G., Maire, M., Shakhnarovich, G.: Learning representations for automatic colorization. In: ECCV (2016)
Lee, H., Hwang, S.J., Shin, J.: Rethinking data augmentation: Self-supervision and self-distillation. In: CVPR (2019)
Lee, K., Maji, S., Ravichandran, A., Soatto, S.: Meta-learning with differentiable convex optimization. In: CVPR (2019)
Li, H., Eigen, D., Dodge, S., Zeiler, M., Wang, X.: Finding task-relevant features for few-shot learning by category traversal. In: CVPR (2019)
Li, W., Wang, L., Xu, J., Huo, J., Gao, Y., Luo, J.: Revisiting local descriptor based image-to-class measure for few-shot learning. In: CVPR (2019)
Lin, H., Han, G., Ma, J., Huang, S., Lin, X., Chang, S.F.: Supervised masked knowledge distillation for few-shot transformers. In: CVPR (2023)
Liu, C., Fu, Y., Xu, C., Yang, S., Li, J., Wang, C., Zhang, L.: Learning a few-shot embedding model with contrastive learning. In: AAAI (2021)
Liu, L., Li, Y., Kuang, Z., Xue, J., Chen, Y., Yang, W., Liao, Q., Zhang, W.: Towards impartial multi-task learning. In: ICLR (2021)
Mangla, P., Singh, M., Sinha, A., Kumari, N., Balasubramanian, V.N., Krishnamurthy, B.: Charting the right manifold: Manifold mixup for few-shot learning. In: WACV (2020)
Noroozi, M., Favaro, P.: Unsupervised learning of visual representations by solving jigsaw puzzles. In: ECCV (2016)
Oreshkin, B.N., López, P.R., Lacoste, A.: TADAM: task dependent adaptive metric for improved few-shot learning. In: NeurIPS (2018)
Rajasegaran, J., Khan, S.H., Hayat, M., Khan, F.S., Shah, M.: Self-supervised knowledge distillation for few-shot learning. In: BMVC (2020)
Ravi, S., Larochelle, H.: Optimization as a model for few-shot learning. In: ICLR (2017)
Ren, M., Triantafillou, E., Ravi, S., Snell, J., Swersky, K., Tenenbaum, J.B., Larochelle, H., Zemel, R.S.: Meta-learning for semi-supervised few-shot classification. In: ICLR (2018)
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Fei-Fei, L.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. (2014)
Rusu, A.A., Rao, D., Sygnowski, J., Vinyals, O., Pascanu, R., Osindero, S., Hadsell, R.: Meta-learning with latent embedding optimization. In: ICLR (2019)
Sener, O., Koltun, V.: Multi-task learning as multi-objective optimization. In: NeurIPS (2018)
Snell, J., Swersky, K., Zemel, R.S.: Prototypical networks for few-shot learning. In: NeurIPS (2017)
Su, J.C., Maji, S., Hariharan, B.: When does self-supervision improve few-shot learning? In: ECCV (2020)
Sun, Q., Liu, Y., Chua, T.S., Schiele, B.: Meta-transfer learning for few-shot learning. In: CVPR (2019)
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P.H., Hospedales, T.M.: Learning to compare: Relation network for few-shot learning. In: CVPR (2018)
Tian, Y., Wang, Y., Krishnan, D., Tenenbaum, J.B., Isola, P.: Rethinking few-shot image classification: a good embedding is all you need? In: ECCV (2020)
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., Wierstra, D.: Matching networks for one shot learning. In: NeurIPS (2016)
Wang, Y., Chao, W., Weinberger, K.Q., van der Maaten, L.: Simpleshot: Revisiting nearest-neighbor classification for few-shot learning. CoRR (2019)
Wertheimer, D., Tang, L., Hariharan, B.: Few-shot classification with feature map reconstruction networks. In: CVPR (2021)
Ye, H., Hu, H., Zhan, D., Sha, F.: Few-shot learning via embedding adaptation with set-to-set functions. In: CVPR (2020)
Ye, H.J., Ming, L., chuan Zhan, D., Chao, W.L.: Few-shot learning with a strong teacher. IEEE Trans. Pattern Anal. Mach. Intell. (2021)
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. In: NeurIPS (2020)
Zagoruyko, S., Komodakis, N.: Wide residual networks. In: BMVC (2016)
Zhang, C., Cai, Y., Lin, G., Shen, C.: Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. In: CVPR (2020)
Zhang, M., Zhang, J., Lu, Z., Xiang, T., Ding, M., Huang, S.: Iept: Instance-level and episode-level pretext tasks for few-shot learning. In: ICLR (2021)
Acknowledgement
This work is supported in part by the National Natural Science Foundation of China (62106100, 62276128, 62192783), the Jiangsu Natural Science Foundation (BK20221441), Young Elite Scientists Sponsorship Program by CAST (2023QNRC001), and Guangdong Basic and Applied Basic Research Foundation (2024A1515011340).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, Y. et al. (2025). Making the Primary Task Primary: Boosting Few-Shot Classification by Gradient-Biased Multi-task Learning. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol 15031. Springer, Singapore. https://doi.org/10.1007/978-981-97-8487-5_24
Download citation
DOI: https://doi.org/10.1007/978-981-97-8487-5_24
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-8486-8
Online ISBN: 978-981-97-8487-5
eBook Packages: Computer ScienceComputer Science (R0)