Abstract
Recognizing fine-grained categories (e.g., dog species) relies on part localization and fine-grained feature learning. However, these classification methods use fine labels and ignore the structural information between different classes. In contrast, we take into account the structural information and use it to improve fine-grained visual classification performance. In this paper, we propose a novel coarse label representation and the corresponding cost function. The new coarse label representation idea comes from the category representation in the multi-label classification. This kind of coarse label representation can well express the structural information embedded in the class hierarchy, and the coarse labels are only obtained from suffix names of different category names, or given in advance like CIFAR100 dataset. A new cost function is proposed to guide the fine label convergence with the constraint of coarse labels, so we can make full use of this kind of coarse label supervised information to improve fine-grained visual classification. Our method can be generalized to any fine-tuning task; it does not increase the size of the original model; and adds no overhead to the training time. We conduct comprehensive experiments and show that using coarse label constraint improves major fine-grained classification datasets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds 200-2011 Dataset. California Institute of Technology (2011)
Horn, G.V., et al.: Building a bird recognition app and large scale dataset with citizen scientists: the fine print in fine-grained dataset collection. In: Computer Vision and Pattern Recognition, pp. 595–604 (2015)
Khosla, A., Jayadevaprakash, N., Yao, B., Li, F.: Novel dataset for fine-grained image categorization (2013)
Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Sixth Indian Conference on Computer Vision, Graphics and Image Processing, ICVGIP 2008, pp. 722–729 (2009)
Yang, L., Luo, P., Chen, C.L., Tang, X.: A large-scale car dataset for fine-grained categorization and verification. In: Computer Vision and Pattern Recognition, pp. 3973–3981 (2015)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Li, F.F.: ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 248–255 (2009)
Yin, C., Yang, S., Chen, S., Howard, A., Belongie, S.: Large scale fine-grained categorization and domain-specific transfer learning (2018)
Wei, X.S., Xie, C.W., Wu, J.: Mask-CNN: localizing parts and selecting descriptors for fine-grained image recognition (2016)
Huang, S., Xu, Z., Tao, D., Zhang, Y.: Part-stacked CNN for fine-grained visual categorization. In: Computer Vision and Pattern Recognition, pp. 1173–1182 (2016)
Sun, M., Yuan, Y., Zhou, F., Ding, E.: Multi-attention multi-class constraint for fine-grained image recognition (2018)
Zheng, H., Fu, J., Mei, T., Luo, J.: Learning multi-attention convolutional neural network for fine-grained image recognition. In: IEEE International Conference on Computer Vision, pp. 5219–5227 (2017)
Fu, J., Zheng, H., Mei, T.: Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Computer Vision and Pattern Recognition, pp. 4476–4484 (2017)
Krause, J., et al.: The unreasonable effectiveness of noisy data for fine-grained recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 301–320. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_19
Zhang, M.L., Zhou, Z.H.: A review on multi-label learning algorithms. IEEE Trans. Knowl. Data Eng. 26, 1819–1837 (2014)
Gibaja, E., Ventura, S.: A tutorial on multilabel learning. ACM Comput. Surv. 47, 1–38 (2015)
Zhang, N., Donahue, J., Girshick, R., Darrell, T.: Part-based R-CNNs for fine-grained category detection. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 834–849. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_54
Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: Computer Vision and Pattern Recognition, pp. 842–850 (2015)
Lin, T.Y., Roychowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition, pp. 1449–1457 (2015)
Cui, Y., Zhou, F., Wang, J., Liu, X., Lin, Y., Belongie, S.: Kernel pooling for convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3049–3058 (2017)
Moghimi, M., Belongie, S., Saberian, M., Yang, J., Vasconcelos, N., Li, L.J.: Boosted convolutional neural networks. In: British Machine Vision Conference, pp. 24.21–24.13 (2016)
Horn, G.V., et al.: The iNaturalist species classification and detection dataset (2018)
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous distributed systems (2016)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Computer Science (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition, pp. 770–778 (2015)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision, pp. 2818–2826 (2015)
Zagoruyko, S., Komodakis, N.: Wide residual networks (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: International Conference on Neural Information Processing Systems, pp. 1097–1105 (2012)
Lin, M., Chen, Q., Yan, S.: Network in Network. Computer Science (2014)
Acknowledgment
This paper was partially supported by the Shenzhen Science & Technology Fundamental Research Program (No: JCYJ20160330095814461) & Shenzhen Key Laboratory for Intelligent Multimedia and Virtual Reality (ZDSYS201703031405467).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, C., Zou, Y. (2019). Using Coarse Label Constraint for Fine-Grained Visual Classification. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11296. Springer, Cham. https://doi.org/10.1007/978-3-030-05716-9_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-05716-9_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05715-2
Online ISBN: 978-3-030-05716-9
eBook Packages: Computer ScienceComputer Science (R0)