Skip to main content
SpringerLink
Log in

Clothing fashion style recognition with design issue graph

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Fashion style recognition of clothing images facilitates the clothing retrieval and recommendation in E-commerce. It is still a challenging task because the clothing images of same style may have diverse visual appearances. Existing fashion style recognition methods utilize deep neural networks to classify clothing images based on pixel-level or region-level features. However, these features of local regions lack the semantics of fashion issues and make the style recognition sensitive to clothing appearance changing. To tackle this problem, we construct Design Issue Graphs (DIGs) with clothing attributes to form global and semantic representations of fashion styles, and propose a joint fashion style recognition model which consists of two convolutional neural networks based on clothing images and DIGs. The experiments on DeepFashion data sets validate that the proposed model is effective to recognize the clothing fashion styles of diverse appearances. The integration of DIGs into Deep Convolutional Neural Networks (DCNNs) achieves 1.75%, 0.99%, 1.03%, 1.53% improvements for multi-style recognition and 1.22%, 2.06%, 1.58%, 2.20% improvements for certain style recognition in the evaluations of accuracy, precision, recall and F1-score on average respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Akm KE, Kassimm AA, Limm JH, Thamm JY (2018) Learning attribute representations with localization for flexible fashion search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7708–7717

  2. Al-Halah Z, Stiefelhagen R, Grauman K (2017) Fashion forward: forecasting visual style in fashion. In: Proceedings of the IEEE international conference on computer vision, pp 388–397

  3. Bossard L, Dantone M, Leistner C, Wengert C, Quack T, Van Gool L (2012) Apparel classification with style. In: Asian conference on computer vision. Springer, New York, pp 321–335

  4. Chen H, Gallagher A, Girod B (2012) Describing clothing by semantic attributes. In: European conference on computer vision. Springer, New York, pp 609–623

  5. Chen L, Wang R, Yang J, Xue L, Hu M (2019a) Multi-label image classification with recurrently learning semantic dependencies. Vis Comput 35(10):1361–1371

    Article  Google Scholar 

  6. Chen Q, Huang J, Feris R, Brown LM, Dong J, Yan S (2015) Deep domain adaptation for describing people based on fine-grained clothing attributes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5315–5324

  7. Chen Z, Fu Y, Zhang Y, Jiang YG, Xue X, Sigal L (2019b) Multi-level semantic feature augmentation for one-shot learning. IEEE Trans Image Process 28(9):4594–4605

    Article  MathSciNet  Google Scholar 

  8. Cheng CI, Liu DSM (2008) An intelligent clothes search system based on fashion styles. In: 2008 International conference on machine learning and cybernetics, IEEE, vol 3, pp 1592–1597

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), IEEE, vol 1, pp 886–893

  10. Di W, Wah C, Bhardwaj A, Piramuthu R, Sundaresan N (2013) Style finder: fine-grained clothing style detection and retrieval. In: Proceedings of the IEEE Conference on computer vision and pattern recognition workshops, pp 8–13

  11. Fu J, Wang J, Li Z, Xu M, Lu H (2012) Efficient clothing retrieval with semantic-preserving visual phrases. In: Asian conference on computer vision. Springer, New York, pp 420–431

  12. Fu J, Liu Y, Jia J, Ma Y, Meng F, Huang H (2017) A virtual personal fashion consultant: learning from the personal preference of fashion. In: AAAI, pp 5087–5088

  13. Fujita H (2013a) Knowledge-based cybernetics in decision support system based on subjective emotional intelligence. In: 2013 IEEE 9th international conference on computational cybernetics (ICCC), IEEE, pp 13–13

  14. Fujita H (2013b) Knowledge-based in medical decision support system based on subjective intelligence. J Med Inform Technol 22:331–361

    Google Scholar 

  15. Ge Y, Zhang R, Wang X, Tang X, Luo P (2019) Deepfashion2: a versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5337–5345

  16. Gu X, Wong Y, Shou L, Peng P, Chen G, Kankanhalli MS (2018) Multi-modal and multi-domain embedding learning for fashion retrieval and analysis. IEEE Trans Multimed 21(6):1524–1537

    Article  Google Scholar 

  17. Hadi Kiapour M, Han X, Lazebnik S, Berg AC, Berg TL (2015) Where to buy it: Matching street clothing photos in online shops. In: Proceedings of the IEEE international conference on computer vision, pp 3343–3351

  18. He T, Hu Y (2018) Fashionnet: personalized outfit recommendation with deep neural network. arXiv:181002443

  19. Hidayati SC, Hsu CC, Chang YT, Hua KL, Fu J, Cheng WH (2018) What dress fits me best?: fashion recommendation on the clothing style for personal body shape. In: 2018 ACM multimedia conference on multimedia conference, ACM, pp 438–446

  20. Huang J, Feris RS, Chen Q, Yan S (2015) Cross-domain image retrieval with a dual attribute-aware ranking network. In: Proceedings of the IEEE international conference on computer vision, pp 1062–1070

  21. Ji X, Wang W, Zhang M, Yang Y (2017) Cross-domain image retrieval with attention modeling. In: Proceedings of the 25th ACM international conference on multimedia, ACM, pp 1654–1662

  22. Jiang S, Fu Y (2017) Fashion style generator. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, IJCAI-17, pp 3721–3727

  23. Jiang S, Shao M, Jia C, Fu Y (2016) Consensus style centralizing auto-encoder for weak style classification. In: AAAI, pp 1223–1229

  24. Jiang YG, Wu Z, Wang J, Xue X, Chang SF (2017) Exploiting feature and class relationships in video categorization with regularized deep neural networks. IEEE Trans Pattern Anal Mach Intell 40 (2):352–364

    Article  Google Scholar 

  25. Li LJ, Su H, Fei-Fei L, Xing EP (2010) Object bank: a high-level image representation for scene classification & semantic feature sparsification. In: Advances in neural information processing systems, pp 1378–1386

  26. Liao L, He X, Zhao B, Ngo CW, Chua TS (2018) Interpretable multimodal retrieval for fashion products. In: 2018 ACM multimedia conference on multimedia conference, ACM, pp 1571–1579

  27. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125

  28. Lin Y, Ren P, Chen Z, Ren Z, Ma J, De Rijke M (2019) Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Trans Knowl Data Eng pp 1–1

  29. Liu KH, Chen TY, Chen CS (2016a) Mvc: A dataset for view-invariant clothing retrieval and attribute prediction. In: Proceedings of the 2016 ACM on international conference on multimedia retrieval, ACM, pp 313–316

  30. Liu Z, Luo P, Qiu S, Wang X, Tang X (2016b) Deepfashion: powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1096–1104

  31. Liu Z, Yan S, Luo P, Wang X, Tang X (2016c) Fashion landmark detection in the wild. In: European conference on computer vision. Springer, New York, pp 229–245

  32. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110

    Article  Google Scholar 

  33. Ma Y, Jia J, Zhou S, Fu J, Liu Y, Tong Z (2017) Towards better understanding the clothing fashion styles: a multimodal deep learning approach. In: AAAI, pp 38–44

  34. Niepert M, Ahmed M, Kutzkov K (2016) Learning convolutional neural networks for graphs. In: International conference on machine learning, pp 2014–2023

  35. Park J, Woo S, Lee JY, Kweon IS (2020) A simple and light-weight attention module for convolutional neural networks. Int J Comput Vis pp 1–16

  36. Shen S, Han SX, Aberle DR, Bui AA, Hsu W (2019) An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification. Exp Syst Appl 128:84–95

    Article  Google Scholar 

  37. Simo-Serra E, Ishikawa H (2016) Fashion style in 128 floats: joint ranking and classification using weak data for feature extraction. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 298–307

  38. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:14091556

  39. Sun GL, Wu X, Chen HH, Peng Q (2015) Clothing style recognition using fashion attribute detection. In: Proceedings of the 8th international conference on mobile multimedia communications, pp 145–148

  40. Takagi M, Simo-Serra E, Iizuka S, Ishikawa H (2017) What makes a style: Experimental analysis of fashion prediction. In: Proceedings of the IEEE international conference on computer vision, pp 2247–2253

  41. Tangseng P, Yamaguchi K, Okatani T (2017) Recommending outfits from personal closet. In: Proceedings of the IEEE international conference on computer vision, pp 2275– 2279

  42. Wang D, Mao K (2019) Task-generic semantic convolutional neural network for web text-aided image classification. Neurocomputing 329:103–115

    Article  Google Scholar 

  43. Wang S, Liu T, Nam J, Tan L (2018a) Deep semantic feature learning for software defect prediction. IEEE Trans Softw Eng pp 1–1

  44. Wang W, Xu Y, Shen J, Zhu SC (2018b) Attentive fashion grammar network for fashion landmark detection and clothing category classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4271– 4280

  45. Wang X, Zhang T (2011) Clothes search in consumer photos via color matching and attribute learning. In: Proceedings of the 19th ACM international conference on multimedia, ACM, pp 1353–1356

  46. Yamaguchi K, Hadi Kiapour M, Berg TL (2013) Paper doll parsing: Retrieving similar styles to parse clothing items. In: Proceedings of the IEEE international conference on computer vision, pp 3519–3526

  47. Yan S, Liu Z, Luo P, Qiu S, Wang X, Tang X (2017) Unconstrained fashion landmark detection via hierarchical recurrent transformer networks. In: Proceedings of the 25th ACM international conference on multimedia, ACM, pp 172–180

  48. Yang W, Luo P, Lin L (2014) Clothing co-parsing by joint image segmentation and labeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3182–3189

  49. Yuan C, Wu Y, Qin X, Qiao S, Pan Y, Huang P, Liu D, Han N (2019) An effective image classification method for shallow densely connected convolution networks through squeezing and splitting techniques. Appl Intell 49(10):3570–3586

    Article  Google Scholar 

  50. Yue X, Miao D, Cao L, Wu Q, Chen Y (2014) An efficient color quantization based on generic roughness measure. Pattern Recogn 47:1777–1789

    Article  Google Scholar 

  51. Yue X, Cao L, Miao D, Chen Y, Xu B (2015) Multi-view attributes reduction model for traffic bottleneck analysis. Knowl-Based Syst 86:1–10

    Article  Google Scholar 

  52. Zhang X, Jia J, Gao K, Zhang Y, Zhang D, Li J, Tian Q (2017) Trip outfits advisor: location-oriented clothing recommendation. IEEE Trans Multimed 19(11):2533–2544

    Article  Google Scholar 

  53. Zhao N, Cao Y, Lau RW (2018) What characterizes personalities of graphic designs?. ACM Trans Graph (TOG) 37(4):1–15

    Google Scholar 

  54. Zheng S, Yang F, Kiapour MH, Piramuthu R (2018) Modanet: A large-scale street fashion dataset with polygon annotations. In: 2018 ACM multimedia conference on multimedia conference, ACM, pp 1670–1678

  55. Zhou G, Fan Y, Cui R, Bian W, Zhu X, Gai K (2018) Rocket launching: s universal and efficient framework for training well-performing light net. In: Thirty-second AAAI conference on artificial intelligence, pp 4580–4588

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Nos. 61976134, 61573235, 61991410, 61991415), and Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province (No. CICIP2018001).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai, China

    Xiaodong Yue, Cheng Zhang & Ying Lv

  2. Faculty of Software and Information Science, Iwate Prefectural University, Iwate, Japan

    Hamido Fujita

Authors
  1. Xiaodong Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hamido Fujita
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaodong Yue.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yue, X., Zhang, C., Fujita, H. et al. Clothing fashion style recognition with design issue graph. Appl Intell 51, 3548–3560 (2021). https://doi.org/10.1007/s10489-020-01950-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01950-7

Keywords

Search

Navigation