Abstract
In this paper, we present a prototype web application of a product search engine of a fashion e-shop. Today, e-shop product metadata consist of text description, simple attributes (price, size, color, fabric, etc.) and visual information (product photo). Search engines used in e-shops mostly provide text and attribute/category interface for product filtering. In our model, we focus on the visual information applied in an interactive query-by-example scenario. The global visual descriptors may be often ambiguous and may not correspond well with the intended mental query of the user. Therefore, we proposed and evaluated model and GUI allowing user to guide the query process by selecting image regions (patches) of interest within the query. In the demo evaluation, we show that allowing user to specify relevant image patches led to a significant improvement of the results’ relevance in the vast majority of tested queries.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Demo available at http://herkules.ms.mff.cuni.cz/vadet-merged.
References
Hsiao, J.H., Li, L.J.: On visual similarity based interactive product recommendation for online shopping. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 3038–3041, October 2014
Skopal, T., Peška, L., Kovalčík, G., Grosup, T., Lokoč, J.: Product exploration based on latent visual attributes. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM 2017, pp. 2531–2534. ACM (2017)
Fischer, P., Dosovitskiy, A., Brox, T.: Descriptor matching with convolutional neural networks: a comparison to sift. CoRR, vol. abs/1405.5769 (2014)
Long, J., Zhang, N., Darrell, T.: Do convnets learn correspondence? CoRR, vol. abs/1411.1091 (2014)
Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: unifying feature and metric learning for patch-based matching. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3279–3286, June 2015
Zagoruyko, S., Komodakis, N.: Learning to compare image patches via convolutional neural networks. CoRR, vol. abs/1504.03641 (2015)
Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 1096–1104 (2016)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates Inc. (2012)
Čech, P., Maroušek, J., Lokoč, J., Silva, Y.N., Starks, J.: Comparing MapReduce-based k-NN similarity joins on Hadoop for high-dimensional data. In: Cong, G., Peng, W.-C., Zhang, W.E., Li, C., Sun, A. (eds.) ADMA 2017. LNCS (LNAI), vol. 10604, pp. 63–75. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69179-4_5
Peska, L., Trojanova, H.: Towards recommender systems for police photo lineup. In: Proceedings of the 2nd Workshop on Deep Learning for Recommender Systems, DLRS 2017, pp. 19–23. ACM (2017)
Lokoc, J., Bailer, W., Schoeffmann, K., Muenzer, B., Awad, G.: On influential trends in interactive video retrieval: video browser showdown 2015–2017. IEEE Trans. Multimed. (2018). https://doi.org/10.1109/TMM.2018.2830110
Acknowledgments
This research has been supported by Czech Science Foundation (GAČR) project Nr. 17-22224S.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Skopal, T., Peška, L., Grošup, T. (2018). Interactive Product Search Based on Global and Local Visual-Semantic Features. In: Marchand-Maillet, S., Silva, Y., Chávez, E. (eds) Similarity Search and Applications. SISAP 2018. Lecture Notes in Computer Science(), vol 11223. Springer, Cham. https://doi.org/10.1007/978-3-030-02224-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-02224-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02223-5
Online ISBN: 978-3-030-02224-2
eBook Packages: Computer ScienceComputer Science (R0)