skip to main content
10.1145/3442381.3449833acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

FANCY: Human-centered, Deep Learning-based Framework for Fashion Style Analysis

Published: 03 June 2021 Publication History

Abstract

Fashion style analysis is of the utmost importance for fashion professionals. However, it has an issue of having different style classification criteria that rely heavily on professionals’ subjective experiences with no quantitative criteria. We present FANCY (Fashion Attributes detectioN for Clustering stYle), a human-centered, deep learning-based framework to support fashion professionals’ analytic tasks using a computational method integrated with their insights. We work closely with fashion professionals in the whole study process to reflect their domain knowledge and experience as much as possible. We redefine fashion attributes, demonstrate a strong association with fashion attributes and styles, and develop a deep learning model that detects attributes in a given fashion image and reflects fashion professionals’ insight. Based on attribute-annotated 302,772 runway fashion images, we developed 25 new fashion styles (FANCY dataset 1). We summarize quantitative standards of the fashion style groups and present fashion trends based on time, location, and brand.

References

[1]
Roland Barthes 1983. The fashion system. University of California Press Berkeley.
[2]
Lukas Bossard, Matthias Dantone, Christian Leistner, Christian Wengert, Till Quack, and Luc Van Gool. 2012. Apparel classification with style. In Asian conference on computer vision. Springer, 321–335.
[3]
Minsuk Chang, Léonore V Guillain, Hyeungshik Jung, Vivian M Hare, Juho Kim, and Maneesh Agrawala. 2018. Recipescape: An interactive tool for analyzing cooking instructions at scale. In Proceedings of the Conference on Human Factors in Computing Systems. 1–12.
[4]
Huizhong Chen, Andrew Gallagher, and Bernd Girod. 2012. Describing clothing by semantic attributes. In European conference on computer vision. Springer, 609–623.
[5]
KuanTing Chen, Kezhen Chen, Peizhong Cong, Winston H Hsu, and Jiebo Luo. 2015. Who are the devils wearing prada in new york city?. In Proceedings of the 23rd ACM international conference on Multimedia. ACM, 177–180.
[6]
Qiang Chen, Junshi Huang, Rogerio Feris, Lisa M Brown, Jian Dong, and Shuicheng Yan. 2015. Deep domain adaptation for describing people based on fine-grained clothing attributes. In Proceedings of the IEEE conference on computer vision and pattern recognition. 5315–5324.
[7]
Charles Corbiere, Hedi Ben-Younes, Alexandre Ramé, and Charles Ollion. 2017. Leveraging weakly annotated data for fashion image retrieval and label prediction. In Proceedings of the International Conference on Computer Vision Workshops. 2268–2274.
[8]
Jifeng Dai, Yi Li, Kaiming He, and Jian Sun. 2016. R-fcn: Object detection via region-based fully convolutional networks. In Advances in neural information processing systems. 379–387.
[9]
Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C Berg, 2014. Imagenet large scale visual recognition challenge. arXiv preprint arXiv:1409.0575(2014).
[10]
Fiona Dieffenbacher. 2013. Fashion thinking: Creative approaches to the design process. Vol. 14. A&C Black.
[11]
Qi Dong, Shaogang Gong, and Xiatian Zhu. 2017. Multi-task curriculum transfer deep learning of clothing attributes. In IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 520–529.
[12]
Xue Dong, Xuemeng Song, Fuli Feng, Peiguang Jing, Xin-Shun Xu, and Liqiang Nie. 2019. Personalized Capsule Wardrobe Creation with Garment and User Modeling. In Proceedings of the 27th ACM International Conference on Multimedia. 302–310.
[13]
Ann Marie Fiore and Hyun-Jeong Jin. 2003. Influence of image interactivity on approach responses towards an online retailer. Internet Research 13, 1 (2003), 38–48.
[14]
Ann Marie Fiore, Hyun-Jeong Jin, and Jihyun Kim. 2005. For fun and profit: Hedonic value from image interactivity and responses toward an online store. Psychology & Marketing 22, 8 (2005), 669–694.
[15]
Patrizia Gazzola, Enrica Pavione, Roberta Pezzetti, and Daniele Grechi. 2020. Trends in the Fashion Industry. The Perception of Sustainability and Circular Economy: A Gender/Generation Quantitative Approach. Sustainability 12, 7 (2020), 2809.
[16]
Yu-I Ha, Sejeong Kwon, Meeyoung Cha, and Jungseock Joo. 2017. Fashion conversation data on instagram. In Eleventh International AAAI Conference on Web and Social Media.
[17]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision. 2961–2969.
[18]
Junshi Huang, Rogerio S Feris, Qiang Chen, and Shuicheng Yan. 2015. Cross-domain image retrieval with a dual attribute-aware ranking network. In Proceedings of the IEEE international conference on computer vision. 1062–1070.
[19]
Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, 2017. Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7310–7311.
[20]
Tim Jackson. 2007. The process of trend development leading to a fashion season. In Fashion Marketing. Routledge, 192–211.
[21]
Menglin Jia, Mengyun Shi, Mikhail Sirotenko, Yin Cui, Bharath Hariharan, Claire Cardie, and Serge Belongie. 2019. The fashionpedia ontology and fashion segmentation dataset. Cornell University (2019).
[22]
Menglin Jia, Yichen Zhou, Mengyun Shi, and Bharath Hariharan. 2018. A Deep-Learning-Based Fashion Attributes Detection Model. arXiv preprint arXiv:1810.10148(2018).
[23]
M Hadi Kiapour, Kota Yamaguchi, Alexander C Berg, and Tamara L Berg. 2014. Hipster wars: Discovering elements of fashion styles. In European conference on computer vision. Springer, 472–488.
[24]
Adriana Kovashka, Devi Parikh, and Kristen Grauman. 2012. Whittlesearch: Image search with relative attribute feedback. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2973–2980.
[25]
Daniel D Lee and H Sebastian Seung. 1999. Learning the parts of objects by non-negative matrix factorization. Nature 401, 6755 (1999), 788.
[26]
Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980–2988.
[27]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740–755.
[28]
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. 2016. Ssd: Single shot multibox detector. In European conference on computer vision. Springer, 21–37.
[29]
Ziwei Liu, Ping Luo, Shi Qiu, Xiaogang Wang, and Xiaoou Tang. 2016. Deepfashion: Powering robust clothes recognition and retrieval with rich annotations. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1096–1104.
[30]
Devi Parikh and Kristen Grauman. 2011. Relative attributes. In 2011 International Conference on Computer Vision. IEEE, 503–510.
[31]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research 12, Oct (2011), 2825–2830.
[32]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems. 91–99.
[33]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 1135–1144.
[34]
Negar Rostamzadeh, Seyedarian Hosseini, Thomas Boquet, Wojciech Stokowiec, Ying Zhang, Christian Jauvin, and Chris Pal. 2018. Fashion-gen: The generative fashion dataset and challenge. arXiv preprint arXiv:1806.08317(2018).
[35]
Edgar Simo-Serra, Sanja Fidler, Francesc Moreno-Noguer, and Raquel Urtasun. 2015. Neuroaesthetics in fashion: Modeling the perception of fashionability. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 869–877.
[36]
Edgar Simo-Serra and Hiroshi Ishikawa. 2016. Fashion style in 128 floats: Joint ranking and classification using weak data for feature extraction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 298–307.
[37]
Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, and Liqiang Nie. 2019. GP-BPR: Personalized Compatibility Modeling for Clothing Matching. In Proceedings of the 27th ACM International Conference on Multimedia. 320–328.
[38]
George B Sproles. 1981. Analyzing fashion life cycles-Principles and perspectives. Journal of Marketing 45, 4 (1981), 116–124.
[39]
Moeko Takagi, Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2017. What makes a style: Experimental analysis of fashion prediction. In Proceedings of the IEEE International Conference on Computer Vision. 2247–2253.
[40]
Pongsate Tangseng, Zhipeng Wu, and Kota Yamaguchi. 2017. Looking at outfit to parse clothing. arXiv preprint arXiv:1703.01386(2017).
[41]
Andreas Veit, Balazs Kovacs, Sean Bell, Julian McAuley, Kavita Bala, and Serge Belongie. 2015. Learning visual clothing style with heterogeneous dyadic co-occurrences. In Proceedings of the IEEE International Conference on Computer Vision. 4642–4650.
[42]
Wenguan Wang, Yuanlu Xu, Jianbing Shen, and Song-Chun Zhu. 2018. Attentive fashion grammar network for fashion landmark detection and clothing category classification. In Proceedings of the Conference on Computer Vision and Pattern Recognition. 4271–4280.
[43]
Xin Wang, Bo Wu, and Yueqi Zhong. 2019. Outfit Compatibility Prediction and Diagnosis with Multi-Layered Comparison Network. In Proceedings of the 27th ACM International Conference on Multimedia. 329–337.
[44]
Wei Xu, Xin Liu, and Yihong Gong. 2003. Document clustering based on non-negative matrix factorization. In Proceedings of the international conference on Research and development in informaion retrieval. ACM, 267–273.
[45]
Aron Yu and Kristen Grauman. 2015. Just noticeable differences in visual attributes. In Proceedings of the IEEE International Conference on Computer Vision. 2416–2424.
[46]
Xiong Zhang, Jonathan Engel, Sara Evensen, Yuliang Li, Çağatay Demiralp, and Wang-Chiew Tan. 2020. Teddy: A System for Interactive Review Analysis. In Proceedings of the Conference on Human Factors in Computing Systems. 1–13.

Cited By

View all
  • (2025)Understanding the Differences in an AI-Based Creativity Support Tool Between Creativity Types in Fashion DesignInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2448484(1-14)Online publication date: 14-Jan-2025
  • (2024)CoCoStyle: Mixed initiative co-creative system to support creative process of fashion designSoftwareX10.1016/j.softx.2024.10185727(101857)Online publication date: Sep-2024
  • (2024)ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision TransformerInternational Journal of Computer Vision10.1007/s11263-024-02290-6Online publication date: 26-Nov-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WWW '21: Proceedings of the Web Conference 2021
April 2021
4054 pages
ISBN:9781450383127
DOI:10.1145/3442381
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Deep learning
  2. Fashion
  3. Human-centered AI
  4. Model application
  5. Quantitative fashion trend analysis

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

WWW '21
Sponsor:
WWW '21: The Web Conference 2021
April 19 - 23, 2021
Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)112
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Understanding the Differences in an AI-Based Creativity Support Tool Between Creativity Types in Fashion DesignInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2448484(1-14)Online publication date: 14-Jan-2025
  • (2024)CoCoStyle: Mixed initiative co-creative system to support creative process of fashion designSoftwareX10.1016/j.softx.2024.10185727(101857)Online publication date: Sep-2024
  • (2024)ICEv2: Interpretability, Comprehensiveness, and Explainability in Vision TransformerInternational Journal of Computer Vision10.1007/s11263-024-02290-6Online publication date: 26-Nov-2024
  • (2024)The Effects of Artificial Intelligence on the Fashion Industry—Opportunities and Challenges for Sustainable TransformationSustainable Development10.1002/sd.3312Online publication date: 26-Dec-2024
  • (2023)Conceptual framework of hybrid style in fashion image datasets for machine learningFashion and Textiles10.1186/s40691-023-00338-810:1Online publication date: 15-May-2023
  • (2023)Improving Workflow Integration with xPath: Design and Evaluation of a Human-AI Diagnosis System in PathologyACM Transactions on Computer-Human Interaction10.1145/357701130:2(1-37)Online publication date: 17-Mar-2023
  • (2023)A Study on User Perception and Experience Differences in Recommendation Results by Domain Expertise: The Case of Fashion DomainsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585641(1-7)Online publication date: 19-Apr-2023
  • (2023)What is Human-Centered about Human-Centered AI? A Map of the Research LandscapeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580959(1-23)Online publication date: 19-Apr-2023
  • (2023)Unsupervised Fashion Style Learning by Solving Fashion Jigsaw Puzzles2023 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME55011.2023.00317(1847-1852)Online publication date: Jul-2023
  • (2023)Adversarial Normalization: I Can visualize Everything (ICE)2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01166(12115-12124)Online publication date: Jun-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media