Skip to main content
SpringerLink
Log in

GRVT: Toward Effective Grocery Recognition via Vision Transformer

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13443))

Included in the following conference series:

  • 1114 Accesses

Abstract

Grocery recognition aims to classify items by visual features of the image. The intention is to improve retailing experience, manage inventory and help visually impaired people. It is an important task in computer vision. Most previous works utilize global image features with a unique decision rule to recognize groceries and products via convolutional neural network (CNN) models. Such methods work on different CNN architectures to explore more accurate and representative features. However, fine-grained characteristics are not considered in feature extraction. Recently, vision transformer (ViT) models achieve success in multiple computer vision tasks. And fine-grained visual categorization is leveraging self-attention mechanism of ViT to learn discriminative regions and features. In this paper, we propose a novel ViT based framework named grocery recognition vision transformer (GRVT). It integrates multiple granularity scales of patches by multi-scale patch embedding to introduce robust image representation without incurring excessive computation cost. The mixed attention selection module guides the network to choose these discriminative patches and crucial regions for fine-grained feature extraction. Our GRVT achieves the state-of-the-art performance on Freiburg Groceries Dataset and Grocery Store Dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Wei, X.S., Cui, Q., Yang, L., Wang, P., Liu, L.: RPC: a large-scale retail product checkout dataset. arXiv preprint arXiv:1901.07249 (2019)

  2. Leo, M., Furnari, A., Medioni, G.G., Trivedi, M., Farinella, G.M.: Deep learning for assistive computer vision. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11134, pp. 3–14. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11024-6_1

    Chapter  Google Scholar 

  3. Wei, Y., Tran, S., Xu, S., Kang, B., Springer, M.: Deep learning for retail product recognition: challenges and techniques. Comput. Intell. Neurosci. 2020, 23 (2020). https://doi.org/10.1155/2020/8875910. Article ID: 8875910

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  5. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 28. Curran Associates, Inc. (2015)

    Google Scholar 

  6. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: 2017 IEEE International Conference on Computer Vision, pp. 2980–2988 (2017)

    Google Scholar 

  7. Dosovitskiy, A., et al.: An image is worth \(16\times 16\) words: transformers for image recognition at scale. In: International Conference on Learning Representations (2021)

    Google Scholar 

  8. Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable DETR: deformable transformers for end-to-end object detection. In: International Conference on Learning Representations (2021)

    Google Scholar 

  9. Zheng, S., et al.: Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6881–6890 (2021)

    Google Scholar 

  10. Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12873–12883 (2021)

    Google Scholar 

  11. Jund, P., Abdo, N., Eitel, A., Burgard, W.: The freiburg groceries dataset. arXiv preprint arXiv:1611.05799 (2016)

  12. Klasson, M., Zhang, C., Kjellström, H.: A hierarchical grocery store image dataset with visual and semantic labels. In: 2019 IEEE Winter Conference on Applications of Computer Vision, pp. 491–500. IEEE (2019)

    Google Scholar 

  13. Hu, T., Qi, H., Huang, Q., Lu, Y.: See better before looking closer: weakly supervised data augmentation network for fine-grained visual classification. arXiv preprint arXiv:1901.09891 (2019)

  14. Chen, Y., Bai, Y., Zhang, W., Mei, T.: Destruction and construction learning for fine-grained image recognition. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5157–5166 (2019)

    Google Scholar 

  15. Srivastava, M.M.: Bag of tricks for retail product image classification. In: Campilho, A., Karray, F., Wang, Z. (eds.) ICIAR 2020. LNCS, vol. 12131, pp. 71–82. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50347-5_8

    Chapter  Google Scholar 

  16. Ji, R., et al.: Attention convolutional binary neural tree for fine-grained visual categorization. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10468–10477 (2020)

    Google Scholar 

  17. He, J., et al.: TransFG: a transformer architecture for fine-grained recognition. arXiv preprint arXiv:2103.07976 (2021)

  18. Ciocca, G., Napoletano, P., Locatelli, S.G.: Multi-task learning for supervised and unsupervised classification of grocery images. In: Del Bimbo, A., et al. (eds.) ICPR 2021. LNCS, vol. 12662, pp. 325–338. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68790-8_26

    Chapter  Google Scholar 

  19. Noy, A., et al.: ASAP: architecture search, anneal and prune. In: International Conference on Artificial Intelligence and Statistics, pp. 493–503. PMLR (2020)

    Google Scholar 

  20. Nayman, N., Noy, A., Ridnik, T., Friedman, I., Jin, R., Zelnik, L.: XNAS: neural architecture search with expert advice. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

    Google Scholar 

  21. Wang, W., Cui, Y., Li, G., Jiang, C., Deng, S.: A self-attention-based destruction and construction learning fine-grained image classification method for retail product recognition. Neural Comput. Appl. 32(18), 14613–14622 (2020). https://doi.org/10.1007/s00521-020-05148-3

    Article  Google Scholar 

  22. Leo, M., Carcagnì, P., Distante, C.: A systematic investigation on end-to-end deep recognition of grocery products in the wild. In: 2020 25th International Conference on Pattern Recognition, pp. 7234–7241. IEEE (2021)

    Google Scholar 

  23. Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)

    Google Scholar 

  24. Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning, pp. 10347–10357. PMLR (2021)

    Google Scholar 

  25. Wang, W., et al.: Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 568–578 (2021)

    Google Scholar 

  26. Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)

    Google Scholar 

  27. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

  28. Chen, C.F.R., Fan, Q., Panda, R.: CrossViT: cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 357–366 (2021)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61902435, in part by the International Science and Technology Innovation Joint Base of Machine Vision and Medical Image Processing in Hunan Province under Grant 2021CB1013, and in part by the Fundamental Research Funds for the Central Universities of Central South University. We are grateful for resources from the High Performance Computing Center of Central South University.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Central South University, Changsha, 410083, China

    Shu Liu, Xiaoyu Wang & Beiji Zou

  2. School of Literature and Journalism, Central South University, Changsha, 410083, China

    Chengzhang Zhu

  3. Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Changsha, 410083, China

    Shu Liu, Xiaoyu Wang, Chengzhang Zhu & Beiji Zou

Authors
  1. Shu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengzhang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Beiji Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengzhang Zhu .

Editor information

Editors and Affiliations

  1. University of Geneva, Geneva, Switzerland

    Nadia Magnenat-Thalmann

  2. Bournemouth University, Poole, UK

    Jian Zhang

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. University of Crete, Heraklion, Greece

    George Papagiannakis

  5. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  6. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

  7. University of Calgary, Calgary, AB, Canada

    Marina Gavrilova

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, S., Wang, X., Zhu, C., Zou, B. (2022). GRVT: Toward Effective Grocery Recognition via Vision Transformer. In: Magnenat-Thalmann, N., et al. Advances in Computer Graphics. CGI 2022. Lecture Notes in Computer Science, vol 13443. Springer, Cham. https://doi.org/10.1007/978-3-031-23473-6_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-23473-6_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-23472-9

  • Online ISBN: 978-3-031-23473-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation