Skip to main content
SpringerLink
Log in

Bilinear pyramid network for flower species categorization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

It is a challenging task to distinguish between numerous species of flowers due to their visually similarities and variations of the pose and structure. Thanks to properly modeling of the local feature interactions, bilinear CNN has succeeded in classifying of many non-rigid fine-grained species including flowers. However, bilinear CNN only computes the feature in a straightforward way without exploring the interactions between features from multiple layers in the network. In this paper, we present a novel Bilinear Pyramid Network (BPN) for flower categorization. Instead of passing through the network and directly feeding the final classifier, features from a convolutional layer are resized and multiplied with that from the former layer, which alternates multiple times to generates prediction vectors using the features from distinct layers. These features encoded from the feature pyramid spontaneously carry multi-level semantic cues, which yields stronger discriminative powers than single-layer features. Experiments show that the proposed network obtains superior classification results on the challenging dataset of flowers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Angelova A, Zhu S (2013) Efficient object detection and segmentation for fine-grained recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 811–818

  2. Angelova A, Zhu S, Lin Y, Wong J, Shpecht C (2012) Development and deployment of a large-scale flower recognition mobile app. NEC Labs America Technical Report

  3. Chai Y, Lempitsky V, Zisserman A (2011) Bicos: a bi-level co-segmentation method for image classification. In: 2011 International conference on computer vision. IEEE, pp 2579–2586

  4. Cui Y, Song Y, Sun C, Howard A, Belongie S (2018) Large scale fine-grained categorization and domain-specific transfer learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4109–4118

  5. Dat H, Ehsan E (2020) Fine-grained generalized zero-shot learning via dense attribute-based attention. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  6. Dubey A, Gupta O, Guo P, Raskar R, Farrell R, Naik N (2018) Pairwise confusion for fine-grained visual classification. In: Proceedings of the European conference on computer vision (ECCV), pp 70–86

  7. Fu J, Zheng H, Mei T (2017) Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4438–4446

  8. Fu S, Liu W, Zhou Y, Nie L (2019) Hplapgcn: Hypergraph p-laplacian graph convolutional networks. Neurocomputing 362:166–174

    Article  Google Scholar 

  9. Fu S, Liu W, Tao D, Zhou Y, Nie L (2020) Hesgcn: Hessian graph convolutional networks for semi-supervised classification. Inf Sci 514:484–498

    Article  MathSciNet  Google Scholar 

  10. Ge W, Lin X, Yu Y (2019) Weakly supervised complementary parts models for fine-grained image classification from the bottom up. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3034–3043

  11. Gogul I, Kumar VS (2017) Flower species recognition system using convolution neural networks and transfer learning. In: 2017 fourth international conference on signal processing, communication and networking (ICSCN)

  12. Huang Z, Li Y (2020) Interpretable and accurate fine-grained recognition via region grouping. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  13. Kanan C, Cottrell G (2010) Robust classification of objects, faces, and flowers using natural image statistics. In: 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 2472–2479

  14. Khan N, Chaudhuri U, Banerjee B, Chaudhuri S (2019) Graph convolutional network for multi-label vhr remote sensing scene recognition. Neurocomputing 357:36–46

    Article  Google Scholar 

  15. Lam M, Mahasseni B, Todorovic S (2017) Fine-grained recognition as hsnet search for informative image parts. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2520–2529

  16. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  17. Lin TY, RoyChowdhury A, Maji S (2017) Bilinear convolutional neural networks for fine-grained visual recognition. IEEE Trans Pattern Anal Mach Intell 40(6):1309–1322

    Article  Google Scholar 

  18. Mottos AB, Feris RS (2014) Fusing well-crafted feature descriptors for efficient fine-grained classification. In: 2014 IEEE international conference on image processing (ICIP). IEEE, pp 5197–5201

  19. Munro J, Damen D (2020) Multi-modal domain adaptation for fine-grained action recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  20. Nilsback ME, Zisserman A (2007) Delving into the whorl of flower segmentation. In: British machine vision conference, vol 2007, pp 1–10

  21. Nilsback ME, Zisserman A (2008) Automated flower classification over a large number of classes. In: 2008 sixth Indian conference on computer vision, graphics & image processing. IEEE, pp 722–729

  22. Pang C, Yao H, Sun X, Zhao S, Yu W (2018) Rediscover flowers structurally. Multimed Tools Appl 77(7):7851–7863

    Article  Google Scholar 

  23. Rendle S (2010) Factorization machines. In: 2010 IEEE international conference on data mining. IEEE, pp 995–1000

  24. Rosch E, Mervis CB, Gray WD, Johnson DM, Boyes-Braem P (1976) Basic objects in natural categories. Cogn Psychol 8(3):382–439

    Article  Google Scholar 

  25. Sichao F, Weifeng L, Shuying L, Yicong Z (2019) Two-order graph convolutional networks for semi-supervised classification. IET Image Process 13(14):2763–2771

    Article  Google Scholar 

  26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  27. Varma M, Ray D (2007) Learning the discriminative power-invariance trade-off. In: 2007 IEEE 11th international conference on computer vision. IEEE, pp 1–8

  28. Xia X, Cui X, Bing N (2017) Inception-v3 for flower classification. In: International conference on image

  29. Xie L, Tian Q, Hong R, Yan S, Zhang B (2013) Hierarchical part matching for fine-grained visual categorization. In: Proceedings of the IEEE international conference on computer vision, pp 1641–1648

  30. Yu J, Kuang Z, Zhang B, Zhang W, Lin D, Fan J (2018) Leveraging content sensitiveness and user trustworthiness to recommend fine-grained privacy settings for social image sharing. IEEE Transactions on Information Forensics and Security 13(5):1317–1332

    Article  Google Scholar 

  31. Yu J, Zhu C, Zhang J, Huang Q, Tao D (2019) Spatial pyramid-enhanced netvlad with weighted triplet loss for place recognition. IEEE Transactions on Neural Networks and Learning Systems 31:661–674

    Article  Google Scholar 

  32. Zhang Y, Wei XS, Wu J, Cai J, Lu J, Nguyen VA, Do MN (2016) Weakly supervised fine-grained categorization with part-based image representation. IEEE Trans Image Process 25(4):1713–1725

    Article  MathSciNet  Google Scholar 

  33. Zheng H, Fu J, Mei T, Luo J (2017) Learning multi-attention convolutional neural network for fine-grained image recognition. In: Proceedings of the IEEE international conference on computer vision, pp 5209–5217

  34. Zou J, Nagy G (2004) Evaluation of model-based interactive flower recognition. In: Proceedings of the international conference on pattern recognition, vol 2. IEEE, pp 311–314

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 61962014, Grant 61702129, Grant 61772149, and Grant U1701267, in part by the China Postdoctoral Science Foundation under Grant 2018M633047, and in part by the Guangxi Science and Technology Project under Grant AB17195057 and Grant AA18118039.

Author information

Authors and Affiliations

  1. School of Computer Science and Information Security, Guilin University of Electronic Technology, Guilin, 541004, China

    Cheng Pang, Wenhao Wang, Rushi Lan, Zhuo Shi & Xiaonan Luo

Authors
  1. Cheng Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rushi Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhuo Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaonan Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rushi Lan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pang, C., Wang, W., Lan, R. et al. Bilinear pyramid network for flower species categorization. Multimed Tools Appl 80, 215–225 (2021). https://doi.org/10.1007/s11042-020-09679-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09679-8

Keywords

Search

Navigation