Abstract
Contrastive learning has shown great results in image classification. It is primarily applied for semi-supervised or unsupervised representation learning. Contrastive learning has recently shown high accuracy results in supervised environments where fully labeled images were utilized for image classification. In the e-commerce field, product image datasets tend not to have a large number of instances which make the classification of new products more difficult. Thus, a model that can classify product images needs to utilize all the existing labeled images in a detailed manner. This paper adopts supervised contrastive learning to explicitly classify products based on their images. This is done by using Optical Character Recognition (OCR) as an auxiliary input and by fine-tuning pre-trained models. OCR takes advantage of the fact that product images have distinct features over standard images. Labels or texts that appear on product images are captured and fed into the projection network along with their corresponding images. This simplifies the classification of the images for the classifier. The use of the auxiliary input with the fine-tuning has shown a noticeable and promising increase in classification accuracy. Experiments have demonstrated that our proposed framework has achieved over a 15% top-1 accuracy increase over the existing methods on three different datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aloysius, N., Geetha, M.: A review on deep convolutional neural networks. In: 2017 International Conference on Communication and Signal Processing (ICCSP), pp. 0588–0592. IEEE (2017)
Chavaltada, C., Pasupa, K., Hardoon, D.R.: A comparative study of machine learning techniques for automatic product categorisation. In: Cong, F., Leung, A., Wei, Q. (eds.) ISNN 2017. LNCS, vol. 10261, pp. 10–17. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59072-1_2
Chen, L., Yang, F., Yang, H.: Image-based product recommendation system with convolutional neural networks (2017)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)
Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)
Dumais, S., Chen, H.: Hierarchical classification of web content. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 256–263 (2000)
Falcon, W., Cho, K.: A framework for contrastive self-supervised learning and designing a new approach. arXiv preprint arXiv:2009.00104 (2020)
Fengzi, L., Kant, S., Araki, S., Bangera, S., Shukla, S.S.: Neural networks for fashion image classification and visual search. SSRN 3602664 (2020)
Georgieva, P., Zhang, P.: Optical character recognition for autonomous stores. In: 2020 IEEE 10th International Conference on Intelligent Systems (IS), pp. 69–75. IEEE (2020)
Gunel, B., Du, J., Conneau, A., Stoyanov, V.: Supervised contrastive learning for pre-trained language model fine-tuning. arXiv preprint arXiv:2011.01403 (2020)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Jund, P., Abdo, N., Eitel, A., Burgard, W.: The freiburg groceries dataset. arXiv preprint arXiv:1611.05799 (2016)
Kaur, T., Gandhi, T.K.: Deep convolutional neural networks with transfer learning for automated brain image classification. Mach. Vis. Appl. 31(3), 1–16 (2020). https://doi.org/10.1007/s00138-020-01069-2
Khosla, P., et al.: Supervised contrastive learning. arXiv preprint arXiv:2004.11362 (2020)
Klasson, M., Zhang, C., Kjellström, H.: A hierarchical grocery store image dataset with visual and semantic labels. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 491–500. IEEE (2019)
Kozareva, Z.: Everyone likes shopping! Multi-class product categorization for e-commerce. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1329–1333 (2015)
Mahdianpari, M., Salehi, B., Rezaee, M., Mohammadimanesh, F., Zhang, Y.: Very deep convolutional neural networks for complex land cover mapping using multispectral remote sensing imagery. Remote Sens. 10(7), 1119 (2018)
Memon, J., Sami, M., Khan, R.A., Uddin, M.: Handwritten optical character recognition (OCR): a comprehensive systematic literature review (SLR). IEEE Access 8, 142642–142668 (2020)
Mori, S., Nishida, H., Yamada, H.: Optical Character Recognition. Wiley, Hoboken (1999)
Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2009)
Peng, J., Xiao, C., Li, Y.: RP2K: a large-scale retail product dataset for fine-grained image classification. arXiv preprint arXiv:2006.12634 (2020)
Schmarje, L., Santarossa, M., Schröder, S.M., Koch, R.: A survey on semi-, self-and unsupervised learning for image classification. arXiv preprint arXiv:2002.08721 (2020)
Shi, B., Bai, X., Yao, C.: An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2298–2304 (2016)
Sosnovshchenko, O., Baiev, O.: Machine Learning with Swift: Artificial Intelligence for IOS. Packt Publishing Ltd., Birmingham (2018)
Srivastava, M.M.: Bag of tricks for retail product image classification. In: Campilho, A., Karray, F., Wang, Z. (eds.) ICIAR 2020. LNCS, vol. 12131, pp. 71–82. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50347-5_8
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. CoRR arXiv:1512.00567 (2015)
Umaashankar, V., Prakash, A., et al.: Atlas: a dataset and benchmark for e-commerce clothing product categorization. arXiv preprint arXiv:1908.08984 (2019)
Wagh, D., Mahajan, S.: Product image classification techniques. Int. J. Innov. Technol. Explor. Eng. (IJITEE) 8, 389–393 (2019)
Wang, P., Han, K., Wei, X.S., Zhang, L., Wang, L.: Contrastive learning based hybrid networks for long-tailed image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 943–952 (2021)
Wang, Y., Wang, Z.: A survey of recent work on fine-grained image classification techniques. J. Vis. Commun. Image Represent. 59, 210–214 (2019)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487. PMLR (2016)
Zhou, X., et al.: EAST: an efficient and accurate scene text detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5551–5560 (2017)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Azizi, S., Fang, U., Adibi, S., Li, J. (2022). Supervised Contrastive Learning for Product Classification. In: Li, B., et al. Advanced Data Mining and Applications. ADMA 2022. Lecture Notes in Computer Science(), vol 13088. Springer, Cham. https://doi.org/10.1007/978-3-030-95408-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-95408-6_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95407-9
Online ISBN: 978-3-030-95408-6
eBook Packages: Computer ScienceComputer Science (R0)