Skip to main content
SpringerLink
Log in

A Deep Learning-Based System for Product Recognition in Intelligent Retail Environment

  • Conference paper
  • First Online:
Image Analysis and Processing – ICIAP 2022 (ICIAP 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13232))

Included in the following conference series:

  • 1673 Accesses

Abstract

This work proposes a pipeline that aims to recognize the products in a shelf, at the level of the single SKU (Stock Keeping Unit), starting from a photo of that shelf. It is composed of a first neural network that detects the individual products on the shelf and has been trained with the SKU110K dataset and a second network, designed and built within this work that associates to the single image created by the first network, an embedding vector, which describes its distinctive features. By obtaining this vector of the input image, it is possible to measure the similarity, by means of the cosine similarity, between this vector and all the embedding vectors in the comparison dataset. The vector with the highest cosine similarity is associated to an image labeled with the EAN (European Article Number) code and, therefore, this EAN will be that of the input image. Given the particular task, there are not currently any dataset able to meet our requirements as they have not such a granular level of detail (EAN labeled), so a new properly designed dataset is created to solve this task.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/eg4000/SKU110K_CVPR19.

  2. 2.

    https://retailvisionworkshop.github.io/detection_challenge_2020/.

  3. 3.

    https://docs.python.org/library/tkinter.html.

  4. 4.

    https://gs1it.org/migliorare-processi/relazione-industria-distribuzione-best-practice-ecr/albero-categorie-classificazione-condivisa-prodotti/.

References

  1. Chollet, F.: Xception: deep learning with depthwise separable convolutions (2017)

    Google Scholar 

  2. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). https://doi.org/10.1109/CVPR.2009.5206848

  3. Frontoni, E., Paolanti, M., Pietrini, R.: People counting in crowded environment and re-identification. In: Rosin, P.L., Lai, Y.-K., Shao, L., Liu, Y. (eds.) RGB-D Image Analysis and Processing. ACVPR, pp. 397–425. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-28603-3_18

    Chapter  Google Scholar 

  4. George, M., Floerkemeier, C.: Recognizing products: a per-exemplar multi-label image classification approach. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8690, pp. 440–455. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10605-2_29

    Chapter  Google Scholar 

  5. Goldman, E., Herzig, R., Eisenschtat, A., Goldberger, J., Hassner, T.: Precise detection in densely packed scenes. In: Proceeding Conference Computer Vision Pattern Recognition, CVPR (2019)

    Google Scholar 

  6. Hermans, A., Beyer, L., Leibe, B.: In defense of the triplet loss for person re-identification (2017)

    Google Scholar 

  7. Howard, A., et al.: Searching for mobilenetv3 (2019)

    Google Scholar 

  8. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2017)

    Google Scholar 

  9. Lowe, D.: Object recognition from local scale-invariant features. In: Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999). https://doi.org/10.1109/ICCV.1999.790410

  10. Paolanti, M., Pietrini, R., Mancini, A., Frontoni, E., Zingaretti, P.: Deep understanding of shopper behaviours and interactions using RGB-D vision. Mach. Vis. Appl. 31(7), 1–21 (2020)

    Google Scholar 

  11. Pietrini, R., Manco, D., Paolanti, M., Placidi, V., Frontoni, E., Zingaretti, P.: An IOT edge-fog-cloud architecture for vision based planogram integrity. In: Proceedings of the 13th International Conference on Distributed Smart Cameras, pp. 1–5 (2019)

    Google Scholar 

  12. Puccinelli, N.M., Goodstein, R.C., Grewal, D., Price, R., Raghubir, P., Stewart, D.: Customer experience management in retailing: understanding the buying process. J. Retail. 85(1), 15–30 (2009)

    Article  Google Scholar 

  13. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks (2018)

    Google Scholar 

  14. Santra, B., Mukherjee, D.P.: A comprehensive survey on computer vision based approaches for automatic identification of products in retail store. Image Vis. Comput. 86, 45–63 (2019)

    Google Scholar 

  15. Schroff, F., Kalenichenko, D., Philbin, J.: Facenet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition, CVPR. IEEE (2015). https://doi.org/10.1109/cvpr.2015.7298682

  16. Shapiro, M.: Executing the Best Planogram, Norwalk, CT (2009)

    Google Scholar 

  17. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  18. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2015)

    Google Scholar 

  19. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  20. Timothy Chong, I.B., Wee, M.: Deep learning approach to planogram compliance in retail stores, Stanford, CA, USA (2016)

    Google Scholar 

  21. Tonioni, A., Serra, E., Di Stefano, L.: A deep learning pipeline for product recognition on store shelves. In: 2018 IEEE International Conference on Image Processing, Applications and Systems, IPAS, pp. 25–31. IEEE (2018)

    Google Scholar 

  22. Topps, J., Taylor, G.: Managing the Retail Supply Chain: Merchandising Strategies that Increase Sales and Improve Profitability. Kogan Page Publishers (2018)

    Google Scholar 

  23. Vaira, R., Pietrini, R., Pierdicca, R., Zingaretti, P., Mancini, A., Frontoni, E.: An IOT edge-fog-cloud architecture for vision based pallet integrity. In: Cristani, M., Prati, A., Lanz, O., Messelodi, S., Sebe, N. (eds.) ICIAP 2019. LNCS, vol. 11808, pp. 296–306. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30754-7_30

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. VRAI Lab, Dipartimento di Ingegneria dell’Informazione (DII), Università Politecnica delle Marche, 63100, Ancona, Italy

    Rocco Pietrini, Luca Rossi, Adriano Mancini, Primo Zingaretti, Emanuele Frontoni & Marina Paolanti

  2. Department of Political Sciences, Communication and International Relations, University of Macerata, Via Don Minzoni 22/A, 62100, Macerata, Italy

    Emanuele Frontoni & Marina Paolanti

Authors
  1. Rocco Pietrini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adriano Mancini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Primo Zingaretti
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emanuele Frontoni
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marina Paolanti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rocco Pietrini .

Editor information

Editors and Affiliations

  1. Boston University, Boston, MA, USA

    Stan Sclaroff

  2. National Research Council, Lecce, Italy

    Cosimo Distante

  3. National Research Council, Lecce, Italy

    Marco Leo

  4. University of Catania, Catania, Italy

    Giovanni M. Farinella

  5. Technische Universität München, Garching, Germany

    Federico Tombari

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pietrini, R., Rossi, L., Mancini, A., Zingaretti, P., Frontoni, E., Paolanti, M. (2022). A Deep Learning-Based System for Product Recognition in Intelligent Retail Environment. In: Sclaroff, S., Distante, C., Leo, M., Farinella, G.M., Tombari, F. (eds) Image Analysis and Processing – ICIAP 2022. ICIAP 2022. Lecture Notes in Computer Science, vol 13232. Springer, Cham. https://doi.org/10.1007/978-3-031-06430-2_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-06430-2_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-06429-6

  • Online ISBN: 978-3-031-06430-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation