Skip to main content
SpringerLink
Log in

MCCP: multi-modal fashion compatibility and conditional preference model for personalized clothing recommendation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Personalized clothing recommendation remains challenging due to the richness of fashion item representations, the non-uniqueness of fashion compatibility relationship and the complicated conditions of user preference. To address these problems, a novel model combining Multi-modal Fashion Compatibility and Conditional Preference (MCCP) is proposed. Firstly, we extract and fuse the multi-modal features (visual and textual) to comprehensively represent fashion items which can learn item-to-item compatibility and items-to-item compatibility. Secondly, we define conditional preference by dividing user-item interaction data into preference conditions and constructing conditional weight branch to learn preference degrees. Finally, we jointly train all of them based on Bayesian Personalized Ranking (BPR) to offer personalized and fashionable recommendations for user. We create a dataset WEAR-U including user label information and fashion data. Extensive experiment results on WEAR-U verify the effectiveness of the proposed model MCCP.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Data Availability

Data available on request from the authors. The data that support the findings of this study are not openly available due to the demand of dataset expansion for further research and are available from the corresponding author upon reasonable request include information on the data’s location, e.g. in a controlled access repository where relevant.

Notes

  1. https://wear.net/

References

  1. Bettaney EM, Hardwick SR, Zisimopoulos O, Chamberlain BP (2020) Fashion outfit generation for e-commerce. In: Joint European conference on machine learning and knowledge Discovery in Databases, Springer, pp 339–354

  2. Chaidaroon S, Fang Y, Xie M, Magnani A (2019) Neural compatibility ranking for text-based fashion matching. In: Proceedings of the 42nd International ACM SIGIR conference on research and development in information retrieval, pp 1229–1232

  3. Chen K, Franko K, Sang R (2021) Structured model pruning of convolutional networks on tensor processing units. arXiv:2107.04191

  4. Chen L, He Y (2018) Dress fashionably: Learn fashion collocation with deep mixed-category metric learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  5. Chen W, Huang P, Xu J, Guo X, Guo C, Sun F, Li C, Pfadler A, Zhao H, Zhao B (2019) Pog: personalized outfit generation for fashion recommendation at alibaba ifashion. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2662–2670

  6. Cheng Z, Chang X, Zhu L, Kanjirathinkal RC, Kankanhalli M (2019) Mmalfm: Explainable recommendation by leveraging reviews and images. ACM Trans Inf Syst (TOIS) 37(2):1–28

    Article  Google Scholar 

  7. Cheng WH, Song S, Chen CY, Hidayati SC, Liu J (2021) Fashion meets computer vision: a survey. ACM Comput Surv (CSUR) 54(4):1–41

    Article  Google Scholar 

  8. Cui Z, Li Z, Wu S, Zhang XY, Wang L (2019) Dressing as a whole: Outfit compatibility learning based on node-wise graph neural networks. In: The World Wide Web conference, pp 307–317

  9. de Barros Costa E, Rocha HJB, Silva ET, Lima NC, Cavalcanti J (2017) Understanding and personalising clothing recommendation for women. In: World conference on information systems and technologies, Springer, pp 841–850

  10. Dong X, Song X, Feng F, Jing P, Xu XS, Nie L (2019) Personalized capsule wardrobe creation with garment and user modeling. In: Proceedings of the 27th ACM international conference on multimedia, pp 302–310

  11. Dong X, Wu J, Song X, Dai H, Nie L (2020) Fashion compatibility modeling through a multi-modal try-on-guided scheme. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 771–780

  12. Han X, Song X, Yin J, Wang Y, Nie L (2019) Prototype-guided attribute-wise interpretable scheme for clothing matching. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 785–794 (2019)

  13. Han X, Wu Z, Jiang YG, Davis LS (2017) Learning fashion compatibility with bidirectional lstms. In: Proceedings of the 25th ACM international conference on multimedia, pp 1078–1086

  14. Han X, Song X, Yao Y, Xu XS, Nie L (2019) Neural compatibility modeling with probabilistic knowledge distillation. IEEE Trans Image Process 29:871–882

    Article  MathSciNet  Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  16. He R, McAuley J (2016) Vbpr: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI conference on artificial intelligence, vol 30

  17. Hidayati SC, Hsu CC, Chang YT, Hua KL, Fu J, Cheng WH (2018) What dress fits me best? fashion recommendation on the clothing style for personal body shape. In: Proceedings of the 26th ACM international conference on multimedia, pp 438–446

  18. Hsieh CY, Li YM (2019) Fashion recommendation with social intelligence on personality and trends. In: 2019 8th International Congress on Advanced Applied Informatics (IIAI-AAI), IEEE, pp 85–90

  19. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia, pp 675–678

  20. Kang WC, Kim E, Leskovec J, Rosenberg C, McAuley J (2019) Complete the look: Scene-based complementary product recommendation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10532–10541

  21. Laenen K, Moens MF (2020) Attention-based fusion for outfit recommendation. In: Fashion Recommender Systems, Springer, pp 69–86

  22. Lai S, Xu L, Liu K, Zhao J (2015) Recurrent convolutional neural networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 29

  23. Li X, Wang X, He X, Chen L, Xiao J, Chua TS (2020) Hierarchical fashion graph network for personalized outfit recommendation. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 159–168 (2020)

  24. Li Z, Cui Z, Wu S, Zhang X, Wang L (2019) Semi-supervised compatibility learning across categories for clothing matching. In: 2019 IEEE international conference on multimedia and expo (ICME), IEEE, pp 484–489

  25. Lin YL, Tran S, Davis LS (2020) Fashion outfit complementary item retrieval. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3311–3319

  26. Song Lin J, X, Gan T, Yao Y, Liu W, Nie L, (2021) Paintnet: a shape-constrained generative framework for generating clothing from fashion model. Multimed Tools Appl 80:17183–17203

  27. Liu J, Song X, Chen Z, Ma J (2019) Neural fashion experts: i know how to make the complementary clothing matching. Neurocomputing 359:249–263

    Article  Google Scholar 

  28. Liu S, Feng J, Song Z, Zhang T, Lu H, Xu C, Yan S (2012) Hi, magic closet, tell me what to wear! In: Proceedings of the 20th ACM international conference on Multimedia, pp 619–628

  29. Lu Z, Hu Y, Jiang Y, Chen Y, Zeng B (2019) Learning binary code for personalized fashion recommendation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10562–10570

  30. McAuley J, Targett C, Shi Q Van Den Hengel A (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pp 43–52

  31. Meng L, Feng F, He X, Gao X, Chua TS (2020) Heterogeneous fusion of semantic and collaborative information for visually-aware food recommendation. In: Proceedings of the 28th ACM international conference on multimedia, pp 3460–3468

  32. Rendle S, Freudenthaler C, Gantner Z (2012) Schmidt-Thieme L. Bpr, Bayesian personalized ranking from implicit feedback arXiv:1205.2618

    Google Scholar 

  33. Sagar D, Garg J, Kansal P, Bhalla S, Shah RR, Yu Y (2020) Pai-bpr: personalized outfit recommendation scheme with attribute-wise interpretability. In: 2020 IEEE sixth international conference on multimedia big data (BigMM), IEEE, pp 221–230

  34. Sanchez-Riera J, Lin JM, Hua KL, Cheng WH, Tsui AW (2017) I-stylist: finding the right dress through your social networks. In: International conference on multimedia modeling, Springer, pp 662–673

  35. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Sci

  36. Song X, Feng F, Liu J, Li Z, Nie L, Ma J (2017) Neurostylist: Neural compatibility modeling for clothing matching. In: Proceedings of the 25th ACM international conference on Multimedia, pp 753–761

  37. Song X, Han X, Li Y, Chen J, Xu XS, Nie L (2019) Gp-bpr: Personalized compatibility modeling for clothing matching. In: Proceedings of the 27th ACM international conference on multimedia, pp 320–328

  38. Sun GL, Cheng ZQ, Wu X, Peng Q (2018) Personalized clothing recommendation combining user social circle and fashion style consistency. Multimed Tools Appl 77(14):17731–17754

    Article  Google Scholar 

  39. Sun GL, He JY, Wu X, Zhao B, Peng Q (2020) Learning fashion compatibility across categories with deep multimodal neural networks. Neurocomputing 395:237–246

    Article  Google Scholar 

  40. Vasileva MI, Plummer BA, Dusad K, Rajpal S, Kumar R, Forsyth D (2018) Learning type-aware embeddings for fashion compatibility. In: Proceedings of the European conference on computer vision (ECCV), pp 390–405

  41. Wang T, Xu X, Yang Y, Hanjalic A, Shen HT, Song J (2019) Matching images and text with multi-modal tensor fusion and re-ranking. In: Proceedings of the 27th ACM international conference on multimedia, pp 12–20

  42. Yang X, Song X, Feng F, Wen H, Duan LY, Nie L (2021) Attribute-wise explainable fashion compatibility modeling. ACM Trans Multimed Comput Commun Appl (TOMM) 17(1):1–21

    Article  Google Scholar 

  43. Yu LF, Yeung SK, Terzopoulos D, Chan TF (2012) Dressup!: outfit synthesis through automatic optimization. ACM Trans Graph 31(6):134–1

    Article  Google Scholar 

  44. Zhang H, Huang W, Liu L, Chow TW (2019) Learning to match clothing from textual feature-based compatible relationships. IEEE Transactions on Industrial Informatics 16(11):6750–6759

    Article  Google Scholar 

  45. Zhou X, Guo G, Sun Z, Liu Y (2020) Multi-facet user preference learning for fine-grained item recommendation. Neurocomputing 385:258–268

    Article  Google Scholar 

  46. Zhu N, Cao J, Liu Y, Yang Y, Ying H, Xiong H (2020) Sequential modeling of hierarchical user intention and preference for next-item recommendation. In: Proceedings of the 13th international conference on web search and data mining, pp 807–815

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 62262036, 61962030), Yunnan Provincial Foundation for Leaders of Disciplines in Science and Technology (No. 202005AC160036).

Author information

Authors and Affiliations

  1. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, China

    Yunzhu Wang, Li Liu, Xiaodong Fu & Lijun Liu

  2. Computer Technology Application Key Lab of Yunnan Province, Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, China

    Li Liu, Xiaodong Fu & Lijun Liu

Authors
  1. Yunzhu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Li Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaodong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lijun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Liu.

Ethics declarations

Conflicts of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Liu, L., Fu, X. et al. MCCP: multi-modal fashion compatibility and conditional preference model for personalized clothing recommendation. Multimed Tools Appl 83, 9621–9645 (2024). https://doi.org/10.1007/s11042-023-15659-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15659-5

Keywords

Search

Navigation