Abstract
The attention mechanism is one of the most vital branches to solve fine-grained image classification (FGIC) tasks, while most existing attention-based methods only focus on inter-class variance and barely model the intra-class similarity. They perform the classification tasks by enhancing inter-class variance, which narrows down the intra-class similarity indirectly. In this paper, we intend to utilize the intra-class similarity as assistance to improve the classification performance of the obtained attention feature maps. To obtain and utilize the intra-class information, a novel attention mechanism, named category-shared and category-specific feature extraction module (CSS-FEM) is proposed in this paper. CSS-FEM firstly extracts the category-shared features based on the intra-class semantic relationship, then focuses on the discriminative parts. CSS-FEM is assembled by two parts: 1) The category-shared feature extraction module extracts category-shared features that contain high intra-class semantic similarity, to reduce the large intra-class variances. 2) The category-specific feature extraction module performs spatial-attention mechanism in category-shared features to find the discriminative information as category-specific features to decrease the high inter-class similarity. Compared with the state-of-the-art methods, the experimental results on three commonly used FGIC datasets show that the effectiveness and competitiveness of the proposed CSS-FEM. Ablation experiments and visualizations are also provided for further demonstrations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: CVPR (2015)
Angelova, A., Zhu, S.: Efficient object detection and segmentation for fine-grained recognition. In: CVPR (2013)
Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: icoseg: Interactive co-segmentation with intelligent scribble guidance. In: CVPR (2010)
Cai, S., Zuo, W., Zhang, L.: Higher-order integration of hierarchical convolutional activations for fine-grained visual categorization. In: ICCV (2017)
Chang, D., et al.: The devil is in the channels: Mutual-channel loss for fine-grained image classification. IEEE Trans. Image Process. (2020)
Chen, L., et al.: Sca-cnn: spatial and channel-wise attention in convolutional networks for image captioning. In: CVPR (2017)
Chen, Y., Bai, Y., Zhang, W., Mei, T.: Destruction and construction learning for fine-grained image recognition. In: CVPR (2019)
Cong, R., et al.: An iterative co-saliency framework for rgbd images. IEEE Transactions on Cybernetics (2017)
Cui, Y., Zhou, F., Wang, J., Liu, X., Lin, Y., Belongie, S.: Kernel pooling for convolutional neural networks. In: CVPR (2017)
Dubey, A., Gupta, O., Guo, P., Raskar, R., Farrell, R., Naik, N.: Pairwise confusion for fine-grained visual classification. In: ECCV (2018)
Gao, Y., Han, X., Wang, X., Huang, W., Scott, M.: Channel interaction networks for fine-grained image categorization. In: AAAI (2020)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: CVPR (2018)
Ji, R., Wen, L., Zhang, L., Du, D., Wu, Y., Zhao, C., Liu, X., Huang, F.: Attention convolutional binary neural tree for fine-grained visual categorization. In: CVPR (2020)
Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: ICCV workshops (2013)
Luo, W., et al.: Cross-x learning for fine-grained visual categorization. In: ICCV (2019)
Ma, Z., Lai, Y., Kleijn, W.B., Song, Y.Z., Wang, L., Guo, J.: Variational bayesian learning for dirichlet process mixture of inverted dirichlet distributions in non-gaussian image feature modeling. IEEE Trans. Neural Networks Learn. Syst. (2018)
Ma, Z., Xie, J., Lai, Y., Taghia, J., Xue, J.H., Guo, J.: Insights into multiple/single lower bound approximation for extended variational inference in non-gaussian structured data modeling. zIEEE Trans. Neural Networks Learn. Syst. (2019)
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. HAL-INRIA (2013)
Pang, Y., Sun, M., Jiang, X., Li, X.: Convolution in convolution for network in network. IEEE Trans. Neural Networks Learn. Syst. (2017)
Rother, C., Minka, T., Blake, A., Kolmogorov, V.: Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs. In: CVPR (2006)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset. California Institute of Technology (2011)
Wang, Y., Morariu, V.I., Davis, L.S.: Learning a discriminative filter bank within a cnn for fine-grained recognition. In: CVPR (2018)
Zhao, B., Feng, J., Wu, X., Yan, S.: A survey on deep learning-based fine-grained object classification and semantic segmentation. Int. J. Autom. Comput. 14(2), 119–135 (2017). https://doi.org/10.1007/s11633-017-1053-3
Zheng, H., Fu, J., Zha, Z.J., Luo, J.: Looking for the devil in the details: Learning trilinear attention sampling network for fine-grained image recognition. In: CVPR (2019)
Zhuang, P., Wang, Y., Qiao, Y.: Learning attentive pairwise interaction for fine-grained classification. In: AAAI (2020)
Acknowledgement
This work was supported in part by the National Key R&D Program of China under Grant 2019YFF0303300 and under Subject II No. 2019YFF0303302, in part by National Natural Science Foundation of China (NSF) No. 62076031, 61922015, 61773071, U19B2036, in part by Beijing Natural Science Foundation Project No. Z200002, in part by the Beijing Academy of Artificial Intelligence (BAAI) under Grant BAAI2020ZJ0204, in part by the Beijing Nova Programme Interdisciplinary Cooperation Project under Grant Z191100001119140, and in part by BUPT Excellent Ph.D. Students Foundation No. CX2020105.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, H. et al. (2021). Exploring Category-Shared and Category-Specific Features for Fine-Grained Image Classification. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13019. Springer, Cham. https://doi.org/10.1007/978-3-030-88004-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-88004-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88003-3
Online ISBN: 978-3-030-88004-0
eBook Packages: Computer ScienceComputer Science (R0)