Skip to main content
SpringerLink
Log in

UbiShop: Commercial item recommendation using visual part-based object representation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the popularity of online shopping, people have used to shop commercial items on the online shopping websites for convenience. However, based on traditional text search methods, people usually can not find the interesting commercial item they want if they do not know its detailed information, e.g., the name and the seller. Therefore, a more convenient method to help people find the commercial item they want is desired. In this work, we develop a practical system, UbiShop, on mobile phones, whereby users can timely get the related information of interesting commercial items by taking pictures of them. Users can also obtain recommendations on visually similar commercial items to help their buying selections. With the observation that people’s preferences on commercial items usually simply depend on their partial visual styles, we propose a novel representation, Visual Part-based Object Representation (VPOR), for commercial item images. The concept of VPOR is to decompose an item image into a set of disjointed partitions, with each of them represents a meaningful semantic parts. User can thus assign non-uniform preferences on the different parts of the commercial item to obtain a personalized recommended results. The experimental results verify our observation and show that the proposed VPOR based commercial item recommendation can achieve better performance than existing text-based and visual-based methods according to the user study.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. http://www.taotaosou.com/

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings CVPR, pp 1597–1604

  2. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Proceedings VLDB, pp 487–499

  3. Arbeláez P, Hariharan B, Gu C, Gupta S, Bourdev L, Malik J (2012) Semantic segmentation using regions and parts. In: Proceedings CVPR, pp 3378–3385

  4. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(4):509–522

    Article  Google Scholar 

  5. Berg TL, Berg AC, Shih J (2010) Automatic attribute discovery and characterization from noisy web data. In: Proceedings ECCV, pp 663–676

  6. Chi H-Y, Cheng W-H, Chen M-S, Tsui AW (2014) MOSRO: Enabling mobile sensing for real-scene objects with grid based structured output learning. In: Proceedings MMM, pp 207–218

  7. Creusen MEH, Schoormans JPL (2005) The different roles of product appearance in consumer choice. J Prod Innov Manag 22(1):63–81

    Article  Google Scholar 

  8. Chen Y, Sampathkumar H, Luo B, Chen X-W (2013) iLike: Bridging the semantic gap in vertical image search by integrating text and visual features, vol 25, pp 2257–2270

  9. Cao L, Fei-Fei L (2007) Spatially coherent latent topic model for concurrent segmentation and classification of objects and scenes. In: Proceedings ICCV, pp 1–8

  10. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–698

    Article  Google Scholar 

  11. De Borobia R, Gonalves P (2008) Consumer behavior: Product characteristics and quality perception. Mpra paper, University Library of Munich

  12. Eslami S, Williams C (2012) A generative model for parts-based object segmentation. In: Advances in Neural Information Processing Systems, pp 100–107

  13. Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315:972–976

    Article  MathSciNet  MATH  Google Scholar 

  14. Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627–1645

    Article  Google Scholar 

  15. Gorelick L, Galun M, Sharon E, Basri R, Brandt A (2006) Shape representation and classification using the poisson equation. IEEE Trans Pattern Anal Mach Intell 28:1991–2005

    Article  Google Scholar 

  16. Gong B, Grauman K, Sha F (2013) Reshaping visual datasets for domain adaptation. In: Advances in Neural Information Processing Systems, pp 1286–1294

  17. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings CVPR, pp 580–587

  18. Herv J, Douze M, Schmid C (2010) Improving bag-of-features for large scale image search. Int J Comput Vision 87:316–336

    Article  Google Scholar 

  19. Hu Y, Xie X, Ma WY, Chia LT, Rajan D (2004) Salient region detection using weighted feature maps based on the human visual attention model. In: Pacifc-Rim Conference on Multimedia, pp 993–1000

  20. Huang J-W, Hung Y-H, Chen M-S (2011) Groupbuyer: A personalized group buying event recommender system using social information filtering. In: Proceedings of the 15th Pacific-Asia Conference on Knowledge Discovery and Data Mining

  21. Huang J-W, Tseng C-Y, Chen M-C, Chen M-S (2011) Pisar: Progressive image search and recommendation system by auto-interpretation and user behavior. In: IEEE International Conference on Systems, Man, and Cybernetics, pp 1442–1447

  22. Kraft DH, Meadow CT, Boyce BR (2007) Text information retrieval systems. Emerald Group Publishing Limited

  23. Kuo Y-H, Cheng W-H, Lin H-T, Hsu WH (2012) Unsupervised semantic feature discovery for image object retrieval and tag refinement. IEEE Trans Multimed 14(4):1079–1090

    Article  Google Scholar 

  24. Kim KJ, Ahn H (2008) A recommender system using GA k-means clustering in an online shopping market. Expert Syst Appl 34(2):1200–1209

    Article  Google Scholar 

  25. Kim YS, Yum B-J, Song J, Myeon Kim S (2005) Development of a recommender system based on navigational and behavioral patterns of customers in e-commerce sites. Expert Syst Appl 28(2):381–393

    Article  Google Scholar 

  26. Kuo Y-H, Lin H-T, Cheng W-H, Yang Y-H, Hsu WH (2011) Unsupervised auxiliary visual words discovery for large-scale image object retrieval. In: Proceedings CVPR, pp 905–912

  27. Li H, Wang X, Tang J, Yi L, Xiao L (2011) iSearch: towards precise retrieval of item image. In: Proceedings of the Third International Conference on Internet Multimedia Computing and Service, pp 5–8

  28. Ma Y-F, Zhang H-J (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings 11th ACM International Conference Multimedia, pp 374–381

  29. Ma H, Zhou D, Liu C, Lyu MR, King I (2011) Recommender systems with social regularization. In: Proceedings of the 4th ACM International Conference on Web Search and Data Mining, pp 287– 296

  30. Manjunath BS, Ohm J-R, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Trans Circ Syst Video Technol 11(6):703–715

    Article  Google Scholar 

  31. Norman DA (2003) Emotional Design: Why We Love (or Hate) Everyday Things. Basic Books

  32. Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. In: Proceedings CVPR, pp 2161–2168

  33. Opelt A, Pinz A, Zisserman A (2006) A boundary-fragment-model for object detection. In: Proceedings ECCV, pp 575–588

  34. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: Proceedings CVPR, pp 1–8

  35. Park JS, Chen M-S, Philip SY (1995) An effective hash-based algorithm for mining association rules

  36. Schafer JB, Konstan J, Riedi J (1999) Recommender systems in e-commerce. In: Proceedings of the 1st ACM Conference on Electronic Commerce, pp 158–166

  37. Sivic J, Zisserman A (2009) Efficient visual search of videos cast as text retrieval. IEEE Trans Pattern Anal Mach Intell 31(4):591–606

    Article  Google Scholar 

  38. Soh P-H, Lin Y-C, Chen M-S (2013) Recommendation for online social feeds by exploiting user response behavior. In: Proceedings of the 22Nd International Conference on World Wide Web Companion, pp 197–198

  39. Soille P (2013) Morphological Image Analysis: Principles and Applications. Springer Science & Business Media

  40. Tsai T-H, Cheng W-H, You C-W, Hu M-C, Tsui AW, Chi H-Y (2014) Learning and recognition of on-premise signs from weakly labeled street view images. IEEE Trans Image Process 23(3):1047–1059

    Article  MathSciNet  Google Scholar 

  41. Tang X, Ke L, Cui J, Wen F, Wang X (2012) Intentsearch: Capturing user intention for one-click internet image search. IEEE Trans Pattern Anal Mach Intell 34(7):1342–1353

    Article  Google Scholar 

  42. Wang X, Zhang T (2011) Clothes search in consumer photos via color matching and attribute learning. In: Proceedings of the 19th ACM international conference on Multimedia, pp 1353–1356

  43. Woods RE, Gonzalez RC, Eddins SL (2009) Digital Image Processing Using MATLAB. Gatesmark Publishing

  44. Xiao W, Deng X-P, Liang L-L, Peng Q (2012) Interactive product image search with complex scenes

  45. Yu A, Grauman K (2014) Fine-grained visual comparisons with local learning. In: Proceedings CVPR, pp 192–199

  46. Yager RR (1988) On ordered weighted averaging aggregation operators in multicriteria decisionmaking. IEEE Trans Syst, Man Cybern 18(1):183–190

    Article  MathSciNet  MATH  Google Scholar 

  47. Zheng Z, Ma H, Lyu MR, King I (2009) Wsrec: A collaborative filtering based web service recommender system. In: IEEE International Conference on Web Services, pp 437–444

  48. Zhao B, Fei-Fei L, Xing EP (2010) Image segmentation with topic random field. In: Proceedings ECCV, pp 785–798

Download references

Author information

Authors and Affiliations

  1. Research Center for Information Technology Innovation (CITI), Academia Sinica, No.128, Section 2, Academia Road, Nankang District, Taipei, 11529, Taiwan

    Heng-Yu Chi, Wen-Huang Cheng & Ming-Syan Chen

  2. Department of Electrical Engineering, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei, 10617, Taiwan

    Heng-Yu Chi & Ming-Syan Chen

  3. Graduate Institute of Networking and Multimedia, National Taiwan University, No. 1, Sec. 4, Roosevelt Rd., Taipei, 10617, Taiwan

    Chun-Chieh Chen

Authors
  1. Heng-Yu Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chun-Chieh Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-Huang Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming-Syan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Huang Cheng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chi, HY., Chen, CC., Cheng, WH. et al. UbiShop: Commercial item recommendation using visual part-based object representation. Multimed Tools Appl 75, 16093–16115 (2016). https://doi.org/10.1007/s11042-015-2916-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2916-7

Keywords

Search

Navigation