Abstract
Keypoint detection can help fine-grained bird recognition by aligning the bird with the detected keypoints. Most of the existing bird keypoint detection methods have poor performance on symmetric keypoints, because they mainly use texture features only, which usually can not distinguish between symmetric keypoints, such as the keypoints on left and right legs. Besides, these methods cannot deal well with the complex image background. Therefore, we propose a two-branch keypoint detection network that combines both 2D texture and 3D geometric features to tackle these problems. In the 2D branch, we use anchor loss to distinguish between foreground and background to alleviate the influence of complex background on keypoint detection. In the 3D branch, we introduce a 3D deformable mesh model to provide geometric information of symmetric keypoints. The prediction results of the two branches are fused to obtain the final keypoint detection results. We demonstrate the effectiveness of our proposed method on the widely-used CUB200-2011 [23] dataset. The experimental results show that our method can achieve superior accuracy in comparison with the state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Branson, S., Perona, P., Belongie, S.: Strong supervision from weak annotation: interactive training of deformable part models. In: Proceedings of the International Conference on Computer Vision, pp. 1832–1839. IEEE (2011)
Farrell, R., Oza, O., Zhang, N., Morariu, V.I., Darrell, T., Davis, L.S.: Birdlets: subordinate categorization using volumetric primitives and pose-normalized appearance. In: Proceedings of the International Conference on Computer Vision, pp. 161–168. IEEE (2011)
Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2009)
Guo, P., Farrell, R.: Aligned to the object, not to the image: a unified pose-aligned representation for fine-grained recognition. In: Proceedings of the Winter Conference on Applications of Computer Vision (WACV), pp. 1876–1885. IEEE (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hill, G.E., Hill, G.E., McGraw, K.J., Kevin, J., et al.: Bird Coloration: Function and Evolution, vol. 2. Harvard University Press (2006)
Huang, S., Gong, M., Tao, D.: A coarse-fine network for keypoint localization. In: Proceedings of the International Conference on Computer Vision, pp. 3028–3037 (2017)
Huang, S., Tao, D.: Real time fine-grained categorization with accuracy and interpretability. arXiv preprint arXiv:1610.00824 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105 (2012)
Law, H., Deng, J.: CornerNet: detecting objects as paired keypoints. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 765–781. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_45
Liu, J., Belhumeur, P.N.: Bird part localization using exemplar-based models with enforced pose and subcategory consistency. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2520–2527 (2013)
Liu, J., Kanazawa, A., Jacobs, D., Belhumeur, P.: Dog breed classification using part localization. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 172–185. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33718-5_13
Liu, J., Li, Y., Belhumeur, P.N.: Part-pair representation for part localization. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 456–471. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_30
Liu, X., Wang, J., Wen, S., Ding, E., Lin, Y.: Localizing by describing: attribute-guided attention localization for fine-grained recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Meena, S.D., Agilandeeswari, L.: An efficient framework for animal breeds classification using semi-supervised learning and multi-part convolutional neural network (MP-CNN). IEEE Access 7, 151783–151802 (2019)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Ryou, S., Jeong, S.G., Perona, P.: Anchor loss: modulating loss scale based on prediction difficulty. In: Proceedings of the International Conference on Computer Vision, pp. 5992–6001 (2019)
Shih, K.J., Mallya, A., Singh, S., Hoiem, D.: Part localization using multi-proposal consensus for fine-grained categorization. In: Proceedings of the British Machine Vision Conference, pp. 128.1–128.12 (2015)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of the International Conference on Learning Representations (2015)
Sun, S., Zhu, Z., Dai, X., Zhao, Q., Li, J.: Weakly-supervised reconstruction of 3D objects with large shape variation from single in-the-wild images. In: Proceedings of the Asian Conference on Computer Vision (2020)
Tian, Z., Shen, C., Chen, H., He, T.: FCOS: fully convolutional one-stage object detection. In: Proceedings of the International Conference on Computer Vision, pp. 9627–9636 (2019)
Van De Sande, K., Gevers, T., Snoek, C.: Evaluating color descriptors for object and scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1582–1596 (2009)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-UCSD birds-200-2011 dataset. Technical report (2011)
Yang, Y., Ramanan, D.: Articulated pose estimation with flexible mixtures-of-parts. In: CVPR 2011, pp. 1385–1392. IEEE (2011)
Yu, X., Zhou, F., Chandraker, M.: Deep deformation network for object landmark localization. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 52–70. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46454-1_4
Zhang, H., et al.: SPDA-CNN: unifying semantic part detection and abstraction for fine-grained recognition. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 1143–1152 (2016)
Zhang, N., Farrell, R., Iandola, F., Darrell, T.: Deformable part descriptors for fine-grained recognition and attribute prediction. In: Proceedings of the International Conference on Computer Vision, pp. 729–736 (2013)
Zhang, N., Shelhamer, E., Gao, Y., Darrell, T.: Fine-grained pose prediction, normalization, and recognition. arXiv preprint arXiv:1511.07063 (2015)
Zhu, P., Wang, H., Saligrama, V.: Generalized zero-shot recognition based on visually semantic embedding. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 2995–3003 (2019)
Acknowledgement
This work is supported by the National Natural Science Foundation of China (61773270, 61971005).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, T., Zhao, Q., Danzeng, P. (2021). Bird Keypoint Detection via Exploiting 2D Texture and 3D Geometric Features. In: Peng, Y., Hu, SM., Gabbouj, M., Zhou, K., Elad, M., Xu, K. (eds) Image and Graphics. ICIG 2021. Lecture Notes in Computer Science(), vol 12890. Springer, Cham. https://doi.org/10.1007/978-3-030-87361-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-87361-5_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87360-8
Online ISBN: 978-3-030-87361-5
eBook Packages: Computer ScienceComputer Science (R0)