Abstract
Product image matching is essential on e-commerce platforms since the target customers have to distinguish the same products organized in varied manners by different companies. Machine learning methods can be applied for such kind of task, which treat it as a classification problem. However, it is intrinsically intractable to enforce the model to expand to new products due to its invisibility to unknown categories. Metric learning has emerged as a promising approach to address this issue, which is designed to measure the distances between data points. In this paper, a metric learning based vision transformer (ML-VIT) is proposed to learn the embeddings of the product image data and match the images with small Euclidean distances to the same products. The proposed ML-VIT adopts Arcface loss to achieve intra-class compactness and inter-class dispersion. Compared with Siamese neural network and other pre-trained models in terms of F1 score and accuracy, ML-VIT is proved to yield modest embeddings for product image matching.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Akritidis, L., Bozanis, P.: Effective unsupervised matching of product titles with k-combinations and permutations. In: 2018 Innovations in Intelligent Systems and Applications (INISTA), pp. 1–10 (2018). https://doi.org/10.1109/INISTA.2018.8466294
Chen, X., He, K.: Exploring simple Siamese representation learning. arXiv preprint arXiv:2011.10566 (2020)
Cheng, G., Yang, C., Yao, X., Guo, L., Han, J.: When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Trans. Geosci. Remote Sens. 56(5), 2811–2821 (2018)
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4690–4699 (2019)
Dosovitskiy, A., et al.: An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? Metric learning approaches for face identification. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 498–505 (2009). https://doi.org/10.1109/ICCV.2009.5459197
Guy, I., Radinsky, K.: Structuring the unstructured: from startup to making sense of eBay’s huge eCommerce inventory. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2017, p. 1351. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3077136.3096469
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kaya, M., Bilge, H.Ş: Deep metric learning: a survey. Symmetry 11(9), 1066 (2019)
Kim, W., Goyal, B., Chawla, K., Lee, J., Kwon, K.: Attention-based ensemble for deep metric learning. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 760–777. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01246-5_45
Kulis, B., et al.: Metric learning: a survey. Found. Trends Mach. Learn. 5(4), 287–364 (2012)
Li, J., Dou, Z., Zhu, Y., Zuo, X., Wen, J.-R.: Deep cross-platform product matching in e-commerce. Inf. Retr. J. 23(2), 136–158 (2019). https://doi.org/10.1007/s10791-019-09360-1
Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., Song, L.: SphereFace: deep hypersphere embedding for face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 212–220 (2017)
Majumder, B.P., Subramanian, A., Krishnan, A., Gandhi, S., More, A.: Deep recurrent neural networks for product attribute extraction in ecommerce. arXiv preprint arXiv:1803.11284 (2018)
Mauge, K., Rohanimanesh, K., Ruvini, J.D.: Structuring e-commerce inventory. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 805–814 (2012)
Peeters, R., Bizer, C., Glavaš, G.: Intermediate training of Bert for product matching. Small 745(722), 2–112 (2020)
Putthividhya, D., Hu, J.: Bootstrapped named entity recognition for product attribute extraction. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 1557–1567 (2011)
Ristoski, P., Petrovski, P., Mika, P., Paulheim, H.: A machine learning approach for product matching and categorization. Semant. Web 9(5), 707–728 (2018)
Rivas-Sánchez, M., De La Paz Guerrero-Lebrero, M., Guerrero, E., Bárcena-Gonzalez, G., Martel, J., Galindo, P.L.: Using deep learning for image similarity in product matching. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2017. LNCS, vol. 10305, pp. 281–290. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-59153-7_25
Shah, K., Kopru, S., Ruvini, J.D.: Neural network based extreme classification and similarity models for product matching. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 3 (Industry Papers), pp. 8–15 (2018)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Sun, L.: Research on product attribute extraction and classification method for online review. In: 2017 International Conference on Industrial Informatics-Computing Technology, Intelligent Technology, Industrial Information Integration (ICIICII), pp. 117–121. IEEE (2017)
Tan, M., Le, Q.V.: EfficientNet: rethinking model scaling for convolutional neural networks (2019). https://arxiv.org/abs/1905.11946
Wang, H., et al.: CosFace: large margin cosine loss for deep face recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5265–5274 (2018)
Acknowledgements
The work described in this paper was substantially supported by the grant from the Research Grants Council of the Hong Kong Special Administrative Region [CityU 11200218], one grant from the Health and Medical Research Fund, the Food and Health Bureau, The Government of the Hong Kong Special Administrative Region [07181426], and the funding from Hong Kong Institute for Data Science (HKIDS) at City University of Hong Kong. The work described in this paper was partially supported by two grants from City University of Hong Kong (CityU 11202219, CityU 11203520). This research is also supported by the National Natural Science Foundation of China under Grant No. 32000464.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, L., Shao, W., Wang, F., Xie, W., Wong, KC. (2021). Metric Learning Based Vision Transformer for Product Matching. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13108. Springer, Cham. https://doi.org/10.1007/978-3-030-92185-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-92185-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92184-2
Online ISBN: 978-3-030-92185-9
eBook Packages: Computer ScienceComputer Science (R0)