Skip to main content
SpringerLink
Log in
Book cover

International Conference on Computational Science

ICCS 2022: Computational Science – ICCS 2022 pp 164–177Cite as

Content-Aware Generative Model for Multi-item Outfit Recommendation

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13350))

Abstract

Recently, deep learning-based recommender systems have received increasing attention of researchers and demonstrate excellent results at solving various tasks in various areas. One of the last growing trends is learning the compatibility of items in a set and predicting the next item or several ones by input ones. Fashion compatibility modeling is one of the areas in which this task is being actively researched. Classical solutions are training on existing sets and are learning to recommend items that have been combined with each other before. This severely limits the number of possible combinations. GAN models proved to be the most effective for decreasing the impact of this problem and generating unseen combinations of items, but they also have several limitations. They use a fixed number of input and output items. However, real outfits contain a variable number of items. Also, they use unimodal or multimodal data to generate only visual features. However, this approach is not guaranteed to save content attributes of items during generation. We propose a multimodal transformer-based GAN with cross-modal attention to simultaneously explore visual features and textual attributes. We also propose to represent a set of items as a sequence of items to allow the model to decide how many items should be in the set. Experimenting on FOTOS dataset at the fill-in-the-blank task is showed that our method outperforms such strong baselines as Bi-LSTM-VSE, MGCM, HFGN, and others. Our model has reached 0.878 accuracy versus 0.724 of Bi-LSTM-VSE, 0.822 of MGCM, 0.826 of HFGN.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Han, X., Wu, Z., Jiang, Y.G., Davis, L.S.: Learning fashion compatibility with bidirectional LSTMs. In: MM 2017 - Proceedings of the 2017 ACM Multimedia Conference, July 2017, pp. 1078–1086 (2017). https://doi.org/10.1145/3123266.3123394

  2. Song, X., Feng, F., Liu, J., Li, Z., Nie, L., Ma, J.: NeuroStylist: neural compatibility modeling for clothing matching. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 753–761 (2017). https://doi.org/10.1145/3123266.3123314

  3. Yang, X., Ma, Y., Liao, L., Wang, M., Chua, T.S.: TransNFCM: translation-based neural fashion compatibility modeling. In: 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, December 2018, pp. 403–410 (2018). https://doi.org/10.1609/aaai.v33i01.3301403

  4. Tangseng, P., Yamaguchi, K., Okatani, T.: Recommending outfits from personal closet. In: 2017 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 2275–2279 (2017)

    Google Scholar 

  5. Lu, Z., Hu, Y., Chen, Y., Zeng, B.: Personalized outfit recommendation with learnable anchors. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12722–12731 (2021)

    Google Scholar 

  6. Li, X., Wang, X., He, X., Chen, L., Xiao, J., Chua, T.S.: Hierarchical fashion graph network for personalized outfit recommendation. In: SIGIR 2020 - Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, May 2020, pp. 159–168 (2020). https://doi.org/10.1145/3397271.3401080

  7. Lu, Z., Hu, Y., Jiang, Y., Chen, Y., Zeng, B.: Learning binary code for personalized fashion recommendation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2019, pp. 10554–10562 (2019). https://doi.org/10.1109/CVPR.2019.01081

  8. Cui, Z., Li, Z., Wu, S., Zhang, X., Wang, L.: Dressing as a whole: outfit compatibility learning based on node-wise graph neural networks. In: The Web Conference 2019 - Proceedings of the World Wide Web Conference, WWW 2019, February 2019, pp. 307–317 (2019). https://doi.org/10.1145/3308558.3313444

  9. Cardoso, A., Daolio, F., Vargas, S.: Product characterisation towards personalisation: learning attributes from unstructured data to recommend fashion products. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, March 2018, pp. 80–89 (2018). https://doi.org/10.1145/3219819.3219888

  10. Sagar, D., Garg, J., Kansal, P., Bhalla, S., Shah, R.R., Yu, Y.: PAI-BPR: personalized outfit recommendation scheme with attribute-wise interpretability. In: Proceedings - 2020 IEEE 6th International Conference on Multimedia Big Data, BigMM 2020, August 2020, pp. 221–230 (2020). https://doi.org/10.1109/BigMM50055.2020.00039

  11. Yuan, F., Karatzoglou, A., Arapakis, I., Jose, J.M., He, X.: A simple convolutional generative network for next item recommendation. In: WSDM 2019 - Proceedings of the 12th ACM International Conference on Web Search and Data Mining, vol. 19, pp. 582–590 (2018). https://doi.org/10.1145/3289600.3290975

  12. Kumar, S., das Gupta, M.: c+GAN: complementary fashion item recommendation. In: KDD 2019: Workshop on AI for Fashion, June 2019. https://arxiv.org/abs/1906.05596v1. Accessed 21 Jan 2022

  13. Kang, W.C., Fang, C., Wang, Z., McAuley, J.: Visually-aware fashion recommendation and design with generative image models. In: Proceedings - IEEE International Conference on Data Mining, ICDM, November 2017, pp. 207–216 (2017). https://doi.org/10.1109/ICDM.2017.30

  14. Liu, L., Zhang, H., Ji, Y., Jonathan Wu, Q.M.: Toward AI fashion design: an attribute-GAN model for clothing match. Neurocomputing 341, 156–167 (2019). https://doi.org/10.1016/J.NEUCOM.2019.03.011

    Article  Google Scholar 

  15. Liu, J., Song, X., Chen, Z., Ma, J.: MGCM: multi-modal generative compatibility modeling for clothing matching. Neurocomputing 414, 215–224 (2020). https://doi.org/10.1016/J.NEUCOM.2020.06.033

    Article  Google Scholar 

  16. Yu, C., Hu, Y., Chen, Y., Zeng, B.: Personalized fashion design. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). October 2019, pp. 9045–9054 (2019). https://doi.org/10.1109/ICCV.2019.00914

  17. Hsiao, W.L., Grauman, K.: Creating capsule wardrobes from fashion images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, December 2017, pp. 7161–7170 (2017). https://doi.org/10.1109/CVPR.2018.00748

  18. Dong, X., Jing, P., Song, X., Xu, X.S., Feng, F., Nie, L.: Personalized capsule wardrobe creation with garment and user modeling. In: MM 2019 - Proceedings of the 27th ACM International Conference on Multimedia, October 2019, pp. 302–310 (2019). https://doi.org/10.1145/3343031.3350905

  19. Zheng, N., Song, X., Niu, Q., Dong, X., Zhan, Y., Nie, L.: Collocation and try-on network: whether an outfit is compatible. In: MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, October 2021, pp. 309–317 (2021). https://doi.org/10.1145/3474085.3475691

  20. Dong, X., Wu, J., Song, X., Dai, H., Nie, L.: Fashion compatibility modeling through a multi-modal try-on-guided scheme. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, July 2020, pp. 771–780 (2020). https://doi.org/10.1145/3397271.3401047

  21. Vasileva, M.I., Plummer, B.A., Dusad, K., Rajpal, S., Kumar, R., Forsyth, D.: Learning type-aware embeddings for fashion compatibility. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 405–421. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_24

    Chapter  Google Scholar 

  22. Bettaney, E.M., Hardwick, S.R., Zisimopoulos, O., Chamberlain, B.P.: Fashion outfit generation for e-commerce. In: Dong, Y., Ifrim, G., Mladenić, D., Saunders, C., VanHoecke, S. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12461, pp. 339–354. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67670-4_21

    Chapter  Google Scholar 

  23. Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. Adv. Neural Inf. Process. Syst. 5999–6009 (2017). https://arxiv.org/abs/1706.03762v5. Accessed 21 Jan 2022

  24. Cheng, Y., Wang, R., Pan, Z., Feng, R., Zhang, Y.: Look, listen, and attend: co-attention network for self-supervised audio-visual representation learning. In: MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia, vol. 20, pp. 3884–3892 (2020). https://doi.org/10.1145/3394171.3413869

  25. Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: 7th International Conference on Learning Representations. ICLR 2019, July 2018. https://arxiv.org/abs/1807.00734v3. Accessed 21 Jan 2022

  26. Mao, X., Li, Q., Xie, H., Lau, R.Y.K., Wang, Z., Smolley, S.P.: Least squares generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, October 2016, pp. 2813–2821 (2016). https://doi.org/10.1109/ICCV.2017.304

Download references

Acknowledgements

This research is financially supported by The Russian Science Foundation, Agreement №17-71-30029 with co-financing of Bank Saint Petersburg.

Author information

Authors and Affiliations

  1. ITMO University, Kronverksky Pr. 49 bldg. A, 197101, St. Petersburg, Russia

    Valery Volokha & Klavdiya Bochenina

Authors
  1. Valery Volokha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klavdiya Bochenina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Valery Volokha .

Editor information

Editors and Affiliations

  1. Brunel University London, London, UK

    Derek Groen

  2. University of Amsterdam, Amsterdam, The Netherlands

    Clélia de Mulatier

  3. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  4. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  5. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  6. University of Amsterdam, Amsterdam, The Netherlands

    Peter M. A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Volokha, V., Bochenina, K. (2022). Content-Aware Generative Model for Multi-item Outfit Recommendation. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13350. Springer, Cham. https://doi.org/10.1007/978-3-031-08751-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08751-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08750-9

  • Online ISBN: 978-3-031-08751-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation