Fine-Grained Recognition with Incremental Classes
Pages 672 - 679
Abstract
This work focuses on dealing with fine-grained recognition problems when incremental classes emerge. The task is desirable to not only distinguish subordinate visual classes based on discriminative but subtle object parts, but also recognize new coming sub-classes without suffering from catastrophic forgetting. In this paper, we first propose to localize both object- and part-level image regions for capturing powerful fine-grained patterns. Then, these fine-grained regions are fed into a bilateral network consisting of a stable branch and a flexible branch for supporting observed and incremental sub-classes recognition respectively. Moreover, a cumulative adaptation strategy is further equipped to adjust the network training during the incremental sessions. Meanwhile, to better retain the modeling capability of observed classes, we also replay samples from previous classes by a hallucination approach. Experiments are conducted on three popular fine-grained recognition datasets and results of the proposed method can reveal its superiority over state-of-the-arts.
Supplementary Material
Video and slides for presentation during the conference (aipr2022-136.zip)
- Download
- 337.94 MB
References
[1]
J. Bang, H. Kim, Y. Yoo, J.W. Ha, and J. Choi. 2021. Rainbow memory: Continual learning with a memory of diverse samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, Tennessee, USA, 8218–8227.
[2]
A. Douillard, M. Cord, C. Ollion, T. Robert, and E. Valle. 2020. PODNet: Pooled outputs distillation for small-tasks incremental learning. In Proceedings of the 16th European Conference on Computer Vision. Springer, Glasgow, United Kingdom, 86–102.
[3]
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. 2014. Generative Adversarial Nets. In Twenty-eighth Conference on Neural Information Processing Systems. Montreal, Quebec, Canada, 2672–2680.
[4]
K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Las Vegas, Nevada, USA, 770–778.
[5]
G. Hinton, O. Vinyals, and J. Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531(2015).
[6]
S. Hou, X. Pan, C.C. Loy, Z. Wang, and D. Lin. 2019. Learning a unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, California, USA, 831–839.
[7]
A. Khosla, N. Jayadevaprakash, B. Yao, and F.F. Li. 2011. Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine-grained visual categorization (FGVC), Vol. 2. Citeseer.
[8]
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A.A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114, 13(2017), 3521–3526.
[9]
M. Korkmaz and N. Yilmaz. 2016. Face recognition by using back propagation artificial neural network and windowing method. Journal of Image and Graphics 4, 1 (2016), 15–19.
[10]
Z. Li and D. Hoiem. 2017. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 12(2017), 2935–2947.
[11]
T.Y. Lin, A. RoyChowdhury, and S. Maji. 2015. Bilinear CNN models for fine-grained visual recognition. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, Santiago, Chile, 1449–1457.
[12]
Y. Liu, B. Schiele, and Q. Sun. 2021. Adaptive aggregation networks for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, Tennessee, USA, 2544–2553.
[13]
Y. Liu, Y. Su, A.A. Liu, B. Schiele, and Q. Sun. 2020. Mnemonics training: Multi-class incremental learning without forgetting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, USA, 12245–12254.
[14]
S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi. 2013. Fine-Grained visual classification of aircraft. arXiv preprint arXiv:1306.5151(2013).
[15]
M. Masana, X. Liu, B. Twardowski, M. Menta, A.D. Bagdanov, and J. van de Weijer. 2020. Class-incremental learning: Survey and performance evaluation on image classification. arXiv preprint arXiv:2010.15277(2020).
[16]
K. Pearson. 1901. LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science 2, 11 (1901), 559–572.
[17]
Y. Peng, X. He, and J. Zhao. 2017. Object-part attention model for fine-grained image classification. IEEE Transactions on Image Processing 27, 3 (2017), 1487–1500.
[18]
A. Radford, L. Metz, and S. Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434(2015).
[19]
S.A. Rebuffi, A. Kolesnikov, G. Sperl, and C.H. Lampert. 2017. ICaRL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Honolulu, Hawaii, 2001–2010.
[20]
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, 2015. ImageNet large scale visual recognition challenge. International journal of computer vision 115, 3 (2015), 211–252.
[21]
C. Simon, P. Koniusz, and M. Harandi. 2021. On learning the geodesic path for incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, Tennessee, USA, 1591–1600.
[22]
K. Simonyan and A. Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556(2014).
[23]
X. Tao, X. Hong, X. Chang, S. Dong, X. Wei, and Y. Gong. 2020. Few-shot class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Seattle, USA, 12183–12192.
[24]
L. Van der Maaten and G. Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579–2605. http://jmlr.org/papers/v9/vandermaaten08a.html
[25]
C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. 2011. The caltech-ucsd birds-200-2011 dataset. (2011).
[26]
Y. Wang, V.I. Morariu, and L.S. Davis. 2018. Learning a discriminative filter bank within a CNN for fine-grained recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Salt Lake City, USA, 4148–4157.
[27]
X.-S. Wei, C.W. Xie, J. Wu, and C. Shen. 2018. Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorization. Pattern Recognition 76(2018), 704–714.
[28]
X.-S. Wei, C.L. Zhang, Y. Li, C.W. Xie, J. Wu, C. Shen, and Z.H. Zhou. 2017. Deep descriptor transforming for image co-localization. In Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia, 3048–3054.
[29]
Y. Wu, Y. Chen, L. Wang, Y. Ye, Z. Liu, Y. Guo, and Y. Fu. 2019. Large scale incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, California, USA, 374–382.
[30]
S. Yan, J. Xie, and X. He. 2021. DER: Dynamically expandable representation for class incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, Tennessee, USA, 3014–3023.
[31]
Q. Yang, Y. Gu, and D. Wu. 2019. Survey of incremental learning. In Proceedings of the 2019 Chinese Control and Decision Conference (CCDC). IEEE, Nanchang, China, 399–404.
[32]
C.B. Zhang, J.W. Xiao, X. Liu, Y.C. Chen, and M.M. Cheng. 2022. Representation compensation networks for continual semantic segmentation. arXiv preprint arXiv:2203.05402(2022).
[33]
N. Zhang, J. Donahue, R. Girshick, and T. Darrell. 2014. Part-based R-CNNs for fine-grained category detection. In Proceedings of the 13th European Conference on Computer Vision. Springer, Zurich, Switzerland, 834–849.
Index Terms
- Fine-Grained Recognition with Incremental Classes
Recommendations
Incremental Learning for Fine-Grained Image Recognition
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia RetrievalThis paper considers the problem of fine-grained image recognition with a growing vocabulary. Since in many real world applications we often have to add a new object category or visual concept with just a few images to learn from, it is crucial to ...
Zero-Shot Attribute Attacks on Fine-Grained Recognition Models
Computer Vision – ECCV 2022AbstractZero-shot fine-grained recognition is an important classification task, whose goal is to recognize visually very similar classes, including the ones without training images. Despite recent advances on the development of zero-shot fine-grained ...
Fine-grained classes and how to find them
ICML'24: Proceedings of the 41st International Conference on Machine LearningIn many practical applications, coarse-grained labels are readily available compared to fine-grained labels that reflect subtle differences between classes. However, existing methods cannot leverage coarse labels to infer fine-grained labels in an ...
Comments
Information & Contributors
Information
Published In
September 2022
1221 pages
ISBN:9781450396899
DOI:10.1145/3573942
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
AIPR 2022
AIPR 2022: 2022 5th International Conference on Artificial Intelligence and Pattern Recognition
September 23 - 25, 2022
Xiamen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 24Total Downloads
- Downloads (Last 12 months)4
- Downloads (Last 6 weeks)2
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format