Structural Subspace Learning for Few-shot Fine-grained Recognition
Authors: 
Linjia Li, Jin Deng, Ying Huang, Yanyan Chen, Wei LuoAuthors Info & Claims
Linjia Li, Jin Deng, Ying Huang, Yanyan Chen, Wei LuoAuthors Info & ClaimsPages 693 - 699
Abstract
Classifying fine-grained objects with few-shot reference samples is a big challenge due to the intrinsic large intra-class and small inter-class variances in fine-grained tasks and the additional overfitting risk brought by the few-shot setting. Previous work resorts to models pretrained on tasks sampled from base classes with sufficient training data. Although much progress has been achieved, the performance still lags far behind satisfaction. In this study, inspired by that our human vision recognizes objects in a compositional way and the fine-grained objects share morphology structures, we study a weakly-supervised structural subspace learning (W3SL) method for few-shot fine-grained recognition (FSFGR). To this end, a group of subspace features from linear projections of the CNN feature are achieved. Specifically, a classification loss in each subspace and a similarity regularization between subspace projection matrices are applied to guide the subspaces to have discriminative structural geometry. Moreover, KL-divergences between the outputs of the CNN and subspace features are implemented to distill knowledge into these subspaces. As a result, the low-dimensional subspace features are with strong capacity to represent data from different classes. Extensive experiments on five fine-grained benchmarks verify that our method can effectively generalize to novel few-shot tasks without hurting the performance on base and whole-class few-shot tasks.
References
[1]
Antreas Antoniou, Harrison Edwards, and Amos Storkey. 2019. How to train your MAML. ICLR (2019). https://doi.org/10.48550/arXiv.1810.09502
[2]
Kaidi Cao, Maria Brbić, and Jure Leskovec. 2021. Concept Learners for Few-Shot Learning. In ICLR. https://doi.org/10.48550/arXiv.2007.07375
[3]
L. C. Chen, G. Papandreou, F. Schroff, and H. Adam. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. In CVPR.
[4]
A. Gupta Doersch, Carl and A. Zisserman. 2020. CrossTransformers: spatially-aware few-shot transfer. NeurIPS (2020).
[5]
Chuanqi Dong, Wenbin Li, Jing Huo, Zheng Gu, and Yang Gao. 2020. Learning Task-aware Local Representations for Few-shot Learning. In IJCAI-PRICAI-20. https://doi.org/10.24963/ijcai.2020/100
[6]
Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya, and Jacob Andreas. 2022. Subspace Regularizers for Few-Shot Class Incremental Learning. In ICLR.
[7]
Chelsea Finn, P. Abbeel, and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. In ICML. https://doi.org/arXiv:1703.03400
[8]
Spyros Gidaris and Nikos Komodakis. 2018. Dynamic Few-Shot Visual Learning Without Forgetting. In CVPR. 4367–4375. https://doi.org/arXiv:1804.09458
[9]
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Networks. In arXiv e-prints.
[10]
Mehrtash Tafazzoli Harandi, Richard I. Hartley, Chunhua Shen, Brian C. Lovell, and Conrad Sanderson. 2014. Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds. In International Journal of Computer Vision, Vol. 114. 113–136.
[11]
B. Hariharan and R. Girshick. 2017. Low-shot Visual Recognition by Shrinking and Hallucinating Features. In ICCV. https://doi.org/10.1109/ICCV.2017.328
[12]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask R-CNN. In IEEE Transactions on Pattern Analysis and Machine Intelligence.
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. https://doi.org/10.1109/CVPR.2016.90
[14]
Xiangteng He and Yuxin Peng. 2017. Weakly supervised learning of part selection model with spatial constraints for fine-grained image classification. In National Conference on Artificial Intelligence.
[15]
G. Huang, Z. Liu, Lvd Maaten, and K. Q. Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR. https://doi.org/10.1109/CVPR.2017.243
[16]
Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Fei-Fei Li. 2011. Novel dataset for Fine-Grained Image Categorization. In CVPR.
[17]
Valentin Khrulkov, Leyla Mirvakhabova, E. Ustinova, I. Oseledets, and Victor S. Lempitsky. 2019. Hyperbolic Image Embeddings. In CVPR. https://doi.org/10.13140/RG.2.2.27815.39849
[18]
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR, Vol. abs/1412.6980.
[19]
Jonathan Krause, Michael Stark, Jia Deng, and Fei-Fei Li. 2013. 3D Object Representations for Fine-Grained Categorization. In ICCV.
[20]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient based learning applied to document recognition. In Proceedings of the IEEE, Vol. 86. 2278–2324.
[21]
Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao, and Jiebo Luo. 2019. Revisiting Local Descriptor Based Image-To-Class Measure for Few-Shot Learning. In CVPR. 7253–7260.
[22]
Yann Lifchitz, Yannis Avrithis, Sylvaine Picard, and Andrei Bursuc. 2020. Dense Classification and Implanting for Few-Shot Learning. In CVPR.
[23]
Dragomir;Erhan Dumitru;Szegedy Christian;Reed Scott;Fu Cheng-Yang;Berg Alexander C. Liu, Wei;Anguelov. 2016. SSD: Single Shot MultiBox Detector. In ECCV. 21–37.
[24]
L.;Qin H.;Shi J.;Jia J. Liu, S.;Qi. 2018. Path Aggregation Network for Instance Segmentation(Conference Paper). In CVPR. 8759–8768.
[25]
Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. 2013. Fine-grained visual classification of aircraft. In arXiv preprint arXiv:1306.5151.
[26]
Mehdi Mirza and Simon Osindero. 2014. Conditional Generative Adversarial Nets. In arXiv e-prints.
[27]
Hang Qi, Matthew Brown, and David G. Lowe. 2017. Low-Shot Learning with Imprinted Weights. In CVPR. https://doi.org/10.48550/arXiv.1712.07136
[28]
Yan Qi, Han Sun, Ningzhong Liu, and Huiyu Zhou. 2022. A Task-aware Dual Similarity Network for Fine-grained Few-shot Learning. In PRICAI.
[29]
J. Redmon and A. Farhadi. 2017. YOLO9000: Better, Faster, Stronger. In IEEE Conference on Computer Vision and Pattern Recognition. 6517–6525.
[30]
X. Ruan, H. Liu, W. Pang, and S. Lu. 2019. Fine-grained Classification Algorithm based on Meta-learning. In 2019 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS).
[31]
Chi Zhang;Yujun Cai;Guosheng Lin;Chunhua Shen. 2020. DeepEMD: Few-Shot Image Classification With Differentiable Earth Mover’s Distance and Structured Classifiers. In CVPR.
[32]
C. Simon, P. Koniusz, R. Nock, and M. Harandi. 2020. Adaptive Subspaces for Few-Shot Learning. In CVPR.
[33]
K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. Computer Science.
[34]
Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical Networks for Few-shot Learning. In NIPS.
[35]
Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. 2018. Learning to Compare: Relation Network for Few-Shot Learning. In CVPR. 1199–1208.
[36]
Hung-Yu Tseng, Hsin-Ying Lee, Jia-Bin Huang, and Ming-Hsuan Yang. 2020. Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation. In ICLR.
[37]
Oriol Vinyals, Charles Blundell, Timothy P. Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. 2016. Matching Networks for One Shot Learning. In NIPS.
[38]
Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. 2011. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report. California Institute of Technology.
[39]
Yu-Xiong Wang, Ross B. Girshick, Martial Hebert, and Bharath Hariharan. 2018. Low-Shot Learning from Imaginary Data. In CVPR. 7278–7286.
[40]
Davis Wertheimer and Bharath Hariharan. 2019. Few-Shot Learning With Localization in Realistic Settings. In CVPR. 6551–6560.
[41]
Davis Wertheimer, Luming Tang, and Bharath Hariharan. 2020. Few-Shot Classification with Feature Map Reconstruction Networks. In CVPR. 8008–8017.
[42]
Han-Jia Ye, Hexiang Hu, De chuan Zhan, and Fei Sha. 2018. Few-Shot Learning via Embedding Adaptation With Set-to-Set Functions. In CVPR. 8805–8814.
[43]
Fisher Yu, Vladlen Koltun, and Thomas A. Funkhouser. 2017. Dilated Residual Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 636–644.
[44]
Xiaopeng Zhang, Hongkai Xiong, Wengang Zhou, Weiyao Lin, and Qi Tian. 2016. Picking Deep Filter Responses for Fine-Grained Image Recognition. In CVPR. 1134–1142.
Index Terms
- Structural Subspace Learning for Few-shot Fine-grained Recognition
Recommendations
Learning attention-guided pyramidal features for few-shot fine-grained recognition
Highlights- We propose a two-stage meta-learning framework to learn attention-guided pyramidal features for few-shot fine-grained recognition.
AbstractFew-shot fine-grained recognition (FS-FGR) aims to distinguish several highly similar objects from different sub-categories with limited supervision. However, traditional few-shot learning solutions typically exploit image-level ...
Discriminant subspace learning constrained by locally statistical uncorrelation for face recognition
High-dimensionality of data and the small sample size problem are two significant limitations for applying subspace methods which are favored by face recognition. In this paper, a new linear dimension reduction method called locally uncorrelated ...
Supervised orthogonal discriminant subspace projects learning for face recognition
In this paper, a new linear dimension reduction method called supervised orthogonal discriminant subspace projection (SODSP) is proposed, which addresses high-dimensionality of data and the small sample size problem. More specifically, given a set of ...
Comments
Information & Contributors
Information
Published In
February 2024
757 pages
ISBN:9798400709234
DOI:10.1145/3651671
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- the Scientific and Technological Planning Project of Guangzhou City
- the Young Scholar Project of Pazhou Lab
- National Natural Science Foundation of China
- Natural Science Foundation of Guangdong Province
- the Key Platform and Major Scientific Research Projects for Guangdong Universities
Conference
ICMLC 2024
ICMLC 2024: 2024 16th International Conference on Machine Learning and Computing
February 2 - 5, 2024
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 30Total Downloads
- Downloads (Last 12 months)30
- Downloads (Last 6 weeks)6
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format