Skip to main content
Springer Nature Link
Log in

The Joy of Dressing Is an Art: Outfit Generation Using Self-attention Bi-LSTM

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track (ECML PKDD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12979))

  • 1381 Accesses

Abstract

Fashion represents one’s personality, what you wear is how you present yourself to the world. While in traditional brick & mortar stores, there is staff available to assist customers which results in increased sales, online stores rely on recommender systems. Proposing an outfit with-respect-to the desired product is one such type of recommendation. This paper describes an outfit generation framework that utilizes a deep-learning sequence classification based model. While most of the literature related to outfit generation is regarding model development, the segment describing training data generation is still not mature. We have proposed a novel approach to generate an accurate training dataset that uses the latent distance between positive and random outfits to classify negative outfits. Outfits are defined as a sequence of fashion items where each fashion item is represented by its respective embedding vector obtained from the Bayesian Personalised Ranking- Matrix Factorisation (BPR-MF) algorithm which takes user clickstream activity as an input. An outfit is classified as positive or negative depending on its Goodness Score predicted by a Bi-LSTM model. Further, we show that applying Self-Attention based Bi-LSTM model improved the performance (AUC), relevance (NDCG) by an average 13%, 16% respectively for all gender-categories. The proposed outfit generation framework is deployed on Myntra, a large-scale fashion e-commerce platform in India.

“The Joy of Dressing is an Art” is a famous quote by John Galliano

A. Chouragade—Work done while at Myntra.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.shopify.com/enterprise/ecommerce-fashion-industry.

  2. 2.

    https://www.forbes.com/sites/cognitiveworld/2019/07/16/the-fashion-industry-is-getting-more-intelligent-with-ai/.

References

  1. Iwata, T., Watanabe, S., Sawada, H.: Fashion coordinates recommender system using photographs from fashion magazines. In: IJCAI (2011)

    Google Scholar 

  2. Veit, A., Kovacs, B., Bell, S., McAuley, J., Bala, K., Belongie, S.: Learning visual clothing style with heterogeneous dyadic co-occurrences. In: International Conference on Computer Vision (ICCV), Santiago, Chile (2015). *Equal Contribution

    Google Scholar 

  3. Liu, S., et al.: Hi, magic closet, tell me what to wear! In: Proceedings of the 20th ACM International Conference on Multimedia, MM 2012, pp. 619–628. Association for Computing Machinery, New York (2012)

    Google Scholar 

  4. Han, X., Wu, Z., Jiang, Y.-G., Davis, L.S.: Learning fashion compatibility with bidirectional LSTMs. In: Proceedings of the 25th ACM International Conference on Multimedia, MM 2017, pp. 1078–1086. Association for Computing Machinery, New York (2017)

    Google Scholar 

  5. Hu, Y., Yi, X., Davis, L.S.: Collaborative fashion recommendation: a functional tensor factorization approach. In: Proceedings of the 23rd ACM International Conference on Multimedia, MM 2015, pp. 129–138. Association for Computing Machinery, New York (2015)

    Google Scholar 

  6. Wang, X., Wu, B., Zhong, Y.: Outfit compatibility prediction and diagnosis with multi-layered comparison network. In: Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, pp. 329–337. Association for Computing Machinery, New York (2019)

    Google Scholar 

  7. Lin, Y., Ren, P., Chen, Z., Ren, Z., Ma, J., de Rijke, M.: Explainable outfit recommendation with joint outfit matching and comment generation. IEEE Trans. Knowl. Data Eng. 32(8), 1502–1516 (2020)

    Article  Google Scholar 

  8. Li, Y., Cao, L., Zhu, J., Luo, J.: Mining fashion outfit composition using an end-to-end deep learning approach on set data. IEEE Trans. Multimedia 19, 1946–1955 (2017)

    Article  Google Scholar 

  9. Bettaney, E.M., Hardwick, S.R., Zisimopoulos, O., Chamberlain, B.P.: Fashion outfit generation for e-commerce. arXiv, abs/1904.00741 (2019)

    Google Scholar 

  10. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2009, pp. 452–461. AUAI Press, Arlington (2009)

    Google Scholar 

  11. Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. (TOIS) 20(4), 422–446 (2002)

    Article  Google Scholar 

  12. Kuang, Z., et al.: Fashion retrieval via graph reasoning networks on a similarity pyramid. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3066–3075 (2019)

    Google Scholar 

  13. Liao, L., He, X., Zhao, B., Ngo, C.-W., Chua, T.-S.: Interpretable multimodal retrieval for fashion products. In: Proceedings of the 26th ACM International Conference on Multimedia, MM 2018, pp. 1571–1579. Association for Computing Machinery, New York (2018)

    Google Scholar 

  14. Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: CVPR, pp. 1096–1104. IEEE Computer Society (2016)

    Google Scholar 

  15. Hadi Kiapour, M., Han, X., Lazebnik, S., Berg, A.C., Berg, T.L.: Where to buy it: matching street clothing photos in online shops. In: International Conference on Computer Vision (2015)

    Google Scholar 

  16. Kang, W.-C., McAuley, J.J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206 (2018)

    Google Scholar 

  17. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, November 2016, pp. 606–615. Association for Computational Linguistics (2016)

    Google Scholar 

  18. Laenen, K., Moens, M.-F.: Attention-based fusion for outfit recommendation. arXiv, abs/1908.10585 (2019)

    Google Scholar 

  19. Hoyer, P.O.: Non-negative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)

    MathSciNet  MATH  Google Scholar 

  20. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv, abs/1508.01991 (2015)

    Google Scholar 

  21. Graves, A., Mohamed, A., Hinton, G.: Speech recognition with deep recurrent neural networks. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 38, March 2013

    Google Scholar 

  22. Sharfuddin, A.A., Tihami, M.N., Islam, M.S.: A deep recurrent neural network with BilSTM model for sentiment classification. In: 2018 International Conference on Bangla Speech and Language Processing (ICBSLP), pp. 1–4 (2018)

    Google Scholar 

  23. Pan, Y., Mei, T., Yao, T., Li, H., Rui, Y.: Jointly modeling embedding and translation to bridge video and language. In: CVPR (2016)

    Google Scholar 

  24. Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 5998–6008. Curran Associates Inc (2017)

    Google Scholar 

  25. Lin, Z., et al.: A structured self-attentive sentence embedding. In: International Conference on Learning Representations 2017 (Conference Track) (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Myntra Designs, Bangalore, India

    Manchit Madan, Ankur Chouragade & Sreekanth Vempati

Authors
  1. Manchit Madan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ankur Chouragade
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sreekanth Vempati
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Manchit Madan or Sreekanth Vempati .

Editor information

Editors and Affiliations

  1. Facebook AI, Seattle, WA, USA

    Yuxiao Dong

  2. Torre Telefonica, Barcelona, Spain

    Nicolas Kourtellis

  3. Bielefeld University, CITEC, Bielefeld, Germany

    Barbara Hammer

  4. Basque Center for Applied Mathematics, Bilbao, Spain

    Jose A. Lozano

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Madan, M., Chouragade, A., Vempati, S. (2021). The Joy of Dressing Is an Art: Outfit Generation Using Self-attention Bi-LSTM. In: Dong, Y., Kourtellis, N., Hammer, B., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12979. Springer, Cham. https://doi.org/10.1007/978-3-030-86517-7_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-86517-7_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-86516-0

  • Online ISBN: 978-3-030-86517-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships