Skip to main content
SpringerLink
Log in

How to Wear Beautifully? Clothing Pair Recommendation

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

In this paper, we present a practical system to automatically suggest the most pairing clothing items, given the reference clothing (upper-body or low-body). This has been a challenge, due to clothes having a variety of categories. Clothing is one of the most informative cues for human appearance. In our daily life, people need to wear properly and beautifully to show their confidence, politeness and social status in various occasions. However, it is not easy to decide what to wear and how to coordinate their own clothes. To address this problem, we propose a recommendation approach that includes clothing region detection, clothing pair recommendation and distance fusion. Clothing region detection based on Faster R-CNN is used to detect clothing region. Clothing pair recommendation consists of a quadruple network architecture, where one dual network of the architecture adopts Siamese convolution neural network architecture. Training examples are pairs of upper-body and low-body clothing items that are either compatible or incompatible. The other dual network is used to learn clothing style features of the input image. This framework is designed to learn a feature transformation from the images of clothing items into two latent spaces, which we call them compatible space and style space respectively. After training the two dual networks, we use a distance fusion method to fuse the features extracted from the compatible and style dual networks. To acquire an optimized model and verify our proposed method, we expand an existing large clothing dataset WoG (Weather-to-Garment), and the resulted dataset is called “How to Wear Beautifully” (H2WB). Experiments on the H2WB dataset demonstrate that our approach is effective with clothing region detection and clothing pair recommendation as well as distance fusion.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kiapour M H, Han X, Lazebnik S, Berg A C. Where to buy it: Matching street clothing photos in online shops. In Proc. IEEE International Conference on Computer Vision, Dec. 2015, pp.3343-3351.

  2. Li Z, Li Y, Gao Y, Liu Y. Fast cross-scenario clothing retrieval based on indexing deep features. In Proc. Pacific-Rim Conference on Advances in Multimedia Information Processing, Sept. 2016, pp.107-118.

  3. Lu H, Xu C, Liu G, Song Z, Liu S, Yan S. Street-to-shop: Cross-scenario clothing retrieval via parts alignment and auxiliary set. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2012, pp.1335-1336.

  4. Chen Q, Huang J, Feris R, Brown L M, Dong J, Yan S. Deep domain adaptation for describing people based on fine-grained clothing attributes. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.5315-5324.

  5. Liu Z, Luo P, Qiu S, Wang X, Tang X. DeepFashion: Powering robust clothes recognition and retrieval with rich annotations. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2016, pp.1096-1104.

  6. Zhang X, Jia J, Gao K, Zhang Y, Zhang D, Li J, Tian Q. Trip outfits advisor: Location-oriented clothing recommendation. IEEE Transactions on Multimedia, 2017, 19(11): 2533-2544.

    Article  Google Scholar 

  7. Liu S, Nguyen T V, Feng J, Wang M, Yan S. Hi, magic closet, tell me what to wear! In Proc. ACM International Conference on Multimedia, October 2012, pp.1333-1334.

  8. Liu Y, Gao Y, Feng S, Li Z. Weather-to-garment: Weather-oriented clothing recommendation. In Proc. IEEE International Conference on Multimedia and Expo, June 2017, pp.181-186.

  9. Sha D, Wang D, Zhou X, Feng S, Zhang Y, Yu G. An Approach for Clothing Recommendation Based on Multiple Image Attributes. Springer International Publishing, 2016.

  10. Cheng M M, Hou Q B, Zhang S H et al. Intelligent visual media processing: When graphics meets vision. Journal of Computer Science and Technology, 2017, 32(1): 110-121.

    Article  Google Scholar 

  11. Simo-Serra E, Fidler S, Moreno-Noguer F, Urtasun R. Neuroaesthetics in fashion: Modeling the perception of fashionability. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.869-877.

  12. Jagadeesh V, Piramuthu R, Bhardwaj A, DiW, Sundaresan N. Large scale visual recommendations from street fashion images. In Proc. ACM SIGKDD international conference on Knowledge discovery and data mining, August 2014, pp.1925-1934.

  13. Yamaguchi K, Berg T L, Ortiz L E. Chic or social: Visual popularity analysis in online fashion networks. In Proc. ACM International Conference on Multimedia, Nov. 2014, pp.773-776.

  14. Zheng Z H, Zhang H T, Zhang F L, Mu T J. Image-based clothes changing system. Computational Visual Media, 2017, 3(4): 337-347.

    Article  Google Scholar 

  15. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.

    Article  Google Scholar 

  16. Di W, Wah C, Bhardwaj A, Piramuthu R, Sundaresan N. Style finder: Fine-grained clothing style detection and retrieval. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition Workshops, June 2013, pp.8-13.

  17. Chen K T, Chen K, Cong P, Hsu W H, Luo J. Who are the devils wearing prada in New York city? In Proc. ACM International Conference on Multimedia, August 2015, pp.177-180.

  18. Yamaguchi K, Kiapour M H, Ortiz L E, Berg T L. Parsing clothing in fashion photographs. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2012, pp.3570-3577.

  19. Garcia N, Vogiatzis G. Dress like a star: Retrieving fashion products from videos. In Proc. IEEE International Conference on Computer Vision, October 2017.

  20. Cheng Z Q, Wu X, Liu Y, Hua X S. Video2Shop: Exact matching clothes in videos to online shopping images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2017, pp.4169-4177.

  21. Kalantidis Y, Kennedy L, Li L J. Getting the look: Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proc. International Conference on Multimedia Retrieval, April 2013, pp.105-112.

  22. Shen E, Lieberman H, Lam F. What am I gonna wear?: Scenario-oriented recommendation. In Proc. International Conference on Intelligent User Interfaces, January 2007, pp.365-368.

  23. Zhao Y, Araki K. A recommendation system for a fashion coordination based on user’s information. Ieice Technical Report Life Intelligence and Office Information Systems, 2012, 111: 13-18.

  24. Hu Y, Yi X, Davis L S. Collaborative fashion recommendation: A functional tensor factorization approach. In Proc. ACM International Conference on Multimedia, October 2015, pp.129-138.

  25. Veit A, Kovacs B, Bell S, Mcauley J. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proc. IEEE International Conference on Computer Vision, Dec. 2015, pp.4642-4650.

  26. Song X, Feng F, Liu J, Li Z, Nie L, Ma J. Neurostylist: Neural compatibility modeling for clothing matching. In Proc. ACM International Conference on Multimedia, October 2017, pp.753-761.

  27. Melekhov I, Kannala J, Rahtu E. Siamese network features for image matching. In Proc. International Conference on Pattern Recognition, April 2017, pp.378-383.

  28. Hidayati S C, You C W, Cheng W H, Hua K L. Learning and recognition of clothing genres from full-body images. IEEE Transactions on Cybernetics, 2017, PP(99): 1-13.

  29. Liu Q, Wu S, Wang L. DeepStyle: Learning user preferences for visual recommendation. In Proc. the International ACM SIGIR Conference, June 2017, pp.841-844.

  30. Jia Y, Shelhamer E, Donahue J et al. Caffe: Convolutional architecture for fast feature embedding. In Proc. ACM International Conference on Multimedia, June 2014, pp.675-678.

  31. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural Networks. In Proc. International Conference on Neural Information Processing Systems, January 2012, pp.1097-1105.

  32. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211-252.

    Article  MathSciNet  Google Scholar 

  33. Yang Y, Ramanan D. Articulated pose estimation with exible mixtures-of-parts. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2011, pp.1385-1392.

  34. Liang X, Lin L, Yang W, Luo P, Huang J, Yan S. Clothes coparsing via joint image segmentation and labeling with application to clothing retrieval. IEEE Transactions on Multimedia, 2016, 18(6): 1175-1186.

  35. Hao P Y, Xia Y, Li X X, Kamata S I, Chen S Y. Discriminative histogram intersection metric learning and its applications. Journal of Computer Science and Technology, 2017, 32(3): 507-519.

    Article  MathSciNet  Google Scholar 

  36. Rubner Y, Tomasi C, Guibas L J. The earth mover’s distance as a metric for image retrieval. International Journal of Computer Vision, 2000, 40(2): 99-121.

    Article  MATH  Google Scholar 

  37. Siddiquie B, Feris R S, Davis L S. Image ranking and retrieval based on multi-attribute queries. In Proc. IEEE International Conference on Computer Vision and Pattern Recognition, June 2011, pp.801-808.

Download references

Author information

Authors and Affiliations

  1. College of Computer and Communication Engineering, China University of Petroleum (East China), Qingdao, 266580, China

    Yu-Jie Liu, Yong-Biao Gao, Ling-Yan Bian, Wen-Ya Wang & Zong-Min Li

Authors
  1. Yu-Jie Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong-Biao Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling-Yan Bian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wen-Ya Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zong-Min Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu-Jie Liu.

Electronic supplementary material

Below is the link to the electronic supplementary material.

ESM 1

(PDF 297 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, YJ., Gao, YB., Bian, LY. et al. How to Wear Beautifully? Clothing Pair Recommendation. J. Comput. Sci. Technol. 33, 522–530 (2018). https://doi.org/10.1007/s11390-018-1836-1

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-018-1836-1

Keywords

Search

Navigation