skip to main content
10.1145/3474085.3475210acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Shape Controllable Virtual Try-on for Underwear Models

Published: 17 October 2021 Publication History

Abstract

Image virtual try-on task has abundant applications and has become a hot research topic recently. Existing 2D image-based virtual try-on methods aim to transfer a target clothing image onto a reference person, which has two main disadvantages: cannot control the size and length precisely; unable to accurately estimate the user's figure in the case of users wearing thick clothing, resulting in inaccurate dressing effect. In this paper, we put forward an akin task that aims to dress clothing for underwear models. To solve the above drawbacks, we propose a Shape Controllable Virtual Try-On Network (SC-VTON), where a graph attention network integrates the information of model and clothing to generate the warped clothing image. In addition, the control points are incorporated into SC-VTON for the desired clothing shape. Furthermore, by adding a Splitting Network and a Synthesis Network, we can use in-shop clothing/model pair data to help optimize the deformation module and generalize the task to the typical virtual try-on task. Extensive experiments show that the proposed method can achieve accurate shape control. Meanwhile, compared with other methods, our method can generate high-resolution results with detailed textures, which can be applied in real applications.

Supplementary Material

ZIP File (mfp0275aux.zip)
We provide the Supplementary Material in pdf format with the following contents: Model architectures of Splitting Network and Synthesis Network, details of the Underwear Model Virtual try-on (UMV) dataset, and additional visual results.
MP4 File (mm21-mfp0275.mp4)
In this paper, we propose a Shape Controllable Virtual Try-On Network (SC-VTON), that aims to dress clothing for underwear models. A graph attention network integrates the information of model and clothing to generate the warped clothing image. In addition, the control points are incorporated into SC-VTON for the desired clothing shape. Furthermore, by adding a Splitting Network and a Synthesis Network, we can use in-shop clothing/model pair data to help optimize the deformation module and generalize the task to the typical virtual try-on task.

References

[1]
Thiemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian Theobalt, and Gerard Pons-Moll. 2019. Learning to reconstruct people in clothing from a single RGB camera. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1175--1186.
[2]
Natalia Neverova, and Iasonas Kokkinos. 2018. Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7297--7306.
[3]
Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. 2019. Multi-garment net: Learning to dress 3d people from images. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5420--5430.
[4]
Bindita Chaudhuri, Nikolaos Sarafianos, Linda Shapiro, and Tony Tung. 2021. Semi-supervised Synthesis of High-Resolution Editable Textures for 3D Humans. arXiv preprint arXiv:2103.17266 (2021).
[5]
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV). 801--818.
[6]
Haoqing Cheng, Heng Liu, Fei Gao, and Zhuo Chen. 2020. ADGAN: A Scalable GAN-based Architecture for Image Anomaly Detection. In 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), Vol. 1. IEEE, 987--993.
[7]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[8]
Haoye Dong, Xiaodan Liang, Xiaohui Shen, Bochao Wang, Hanjiang Lai, Jia Zhu, Zhiting Hu, and Jian Yin. 2019. Towards multi-pose guided virtual try-on network. In Proceedings of the IEEE International Conference on Computer Vision. 9026--9035.
[9]
Yuying Ge, Yibing Song, Ruimao Zhang, Chongjian Ge, Wei Liu, and Ping Luo. 2021. Parser-Free Virtual Try-on via Distilling Appearance Flows. arXiv preprint arXiv:2103.04559 (2021).
[10]
Riza Alp Guler and Iasonas Kokkinos. 2019. Holopose: Holistic 3d human reconstruction in-the-wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 10884--10894.
[11]
Natalia Neverova, and Iasonas Kokkinos. 2018. Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7297--7306.
[12]
Erhan Gundogdu, Victor Constantin, Amrollah Seifoddini, Minh Dang, Mathieu Salzmann, and Pascal Fua. 2019. GarNet: A two-stream network for fast and accurate 3D cloth draping. In Proceedings of the IEEE International Conference on Computer Vision. 8739--8748.
[13]
Marc Habermann, Weipeng Xu, Michael Zollhofer, Gerard Pons-Moll, and Christian Theobalt. 2020. Deepcap: Monocular human performance capture using weak supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5052--5063.
[14]
Xintong Han, Xiaojun Hu, Weilin Huang, and Matthew R Scott. 2019. Clothflow: A flow-based model for clothed person generation. In Proceedings of the IEEE International Conference on Computer Vision. 10471--10480.
[15]
Xintong Han, Zuxuan Wu, Zhe Wu, Ruichi Yu, and Larry S Davis. 2018. Viton: An image-based virtual try-on network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7543--7552.
[16]
Peter J Huber. 1992. Robust estimation of a location parameter. In Breakthroughs in statistics. Springer, 492--518.
[17]
Thibaut Issenhuth, Jérémie Mary, and Clément Calauzènes. 2019. End-to-End Learning of Geometric Deformations of Feature Maps for Virtual Try-On. arXiv preprint arXiv:1906.01347 (2019).
[18]
Thibaut Issenhuth, Jérémie Mary, and Clément Calauzènes. 2020. Do Not Mask What You Do Not Need to Mask: a Parser-Free Virtual Try-On. arXiv preprint arXiv:2007.02721 (2020).
[19]
Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. 2015. Spatial transformer networks. In Advances in neural information processing systems. 2017-- 2025.
[20]
Surgan Jandial, Ayush Chopra, Kumar Ayush, Mayur Hemani, Balaji Krishnamurthy, and Abhijeet Halwai. 2020. SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On. In The IEEE Winter Conference on Applications of Computer Vision. 2182--2190.
[21]
Boyi Jiang, Juyong Zhang, Yang Hong, Jinhao Luo, Ligang Liu, and Hujun Bao. 2020. BCNet: Learning Body and Cloth Shape from A Single Image. arXiv preprint arXiv:2004.00214 (2020).
[22]
Angjoo Kanazawa, Michael J Black, David W Jacobs, and Jitendra Malik. 2018. End-to-end recovery of human shape and pose. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7122--7131.
[23]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4401--4410.
[24]
Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016).
[25]
Nikos Kolotouros, Georgios Pavlakos, and Kostas Daniilidis. 2019. Convolutional mesh regression for single-image human shape reconstruction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4501--4510.
[26]
Der-Tsai Lee and Bruce J Schachter. 1980. Two algorithms for constructing a Delaunay triangulation. International Journal of Computer & Information Sciences 9, 3 (1980), 219--242.
[27]
Jituo Li, Juntao Ye, Yangsheng Wang, Li Bai, and Guodong Lu. 2010. Fitting 3D garment models onto individual human models. Computers & graphics 34, 6 (2010), 742--755.
[28]
Kedan Li, Min Jin Chong, Jingen Liu, and David Forsyth. 2020. Toward Accurate and Realistic Virtual Try-on Through Shape Matching and Multiple Warps. arXiv preprint arXiv:2003.10817 (2020).
[29]
Peike Li, Yunqiu Xu, Yunchao Wei, and Yi Yang. 2019. Self-Correction for Human Parsing. arXiv preprint arXiv:1910.09777 (2019).
[30]
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2015. SMPL: A skinned multi-person linear model. ACM transactions on graphics (TOG) 34, 6 (2015), 1--16.
[31]
Matiur Rahman Minar, Thai Thanh Tuan, Heejune Ahn, Paul Rosin, and Yu-Kun Lai. 2020. 3D Reconstruction of Clothes using a Human Body Model and its Application to Image-based Virtual Try-On. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[32]
Matiur Rahman Minar, Thai Thanh Tuan, Heejune Ahn, Paul Rosin, and Yu-Kun Lai. 2020. CP-VTON+: Clothing Shape and Texture Preserving Image-Based Virtual Try-On. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[33]
Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. Tailornet: Predicting clothing in 3d as a function of human pose, shape and garment style. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7365--7375.
[34]
Amir Hossein Raffiee and Michael Sollami. 2020. GarmentGAN: Photo-realistic Adversarial Fashion Transfer. arXiv preprint arXiv:2003.01894 (2020).
[35]
Yurui Ren, Xiaoming Yu, Junming Chen, Thomas H Li, and Ge Li. 2020. Deep image spatial transformation for person image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7690--7699.
[36]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.
[37]
Debapriya Roy, Sanchayan Santra, and Bhabatosh Chanda. 2020. LGVTON: A Landmark Guided Approach to Virtual Try-On. arXiv preprint arXiv:2004.00562 (2020).
[38]
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. 2019. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. In Proceedings of the IEEE International Conference on Computer Vision. 2304--2314.
[39]
Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. 2020. PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 84--93.
[40]
Igor Santesteban, Miguel A Otaduy, and Dan Casas. 2019. Learning-Based Animation of Clothing for Virtual Try-On. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 355--366.
[41]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).
[42]
Olga Sorkine and Marc Alexa. 2007. As-rigid-as-possible surface modeling. In Symposium on Geometry processing, Vol. 4. 109--116.
[43]
Ke Sun, Yang Zhao, Borui Jiang, Tianheng Cheng, Bin Xiao, Dong Liu, Yadong Mu, Xinggang Wang, Wenyu Liu, and Jingdong Wang. 2019. High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514 (2019).
[44]
Nobuyuki Umetani, Danny M Kaufman, Takeo Igarashi, and Eitan Grinspun. 2011. Sensitive couture for interactive garment modeling and editing. ACM Trans. Graph. 30, 4 (2011), 90.
[45]
Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017).
[46]
Raquel Vidaurre, Igor Santesteban, Elena Garces, and Dan Casas. 2020. Fully Convolutional Graph Neural Networks for Parametric Virtual Try-On. arXiv preprint arXiv:2009.04592 (2020).
[47]
Bochao Wang, Huabin Zheng, Xiaodan Liang, Yimin Chen, Liang Lin, and Meng Yang. 2018. Toward characteristic-preserving image-based virtual try-on network. In Proceedings of the European Conference on Computer Vision (ECCV). 589--604.
[48]
Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European Conference on Computer Vision (ECCV). 52--67.
[49]
Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. 2019. Pixel2mesh++: Multi-view 3d mesh generation via deformation. In Proceedings of the IEEE International Conference on Computer Vision. 1042--1051.
[50]
Simon N Wood. 2003. Thin plate regression splines. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65, 1 (2003), 95--114.
[51]
Zhonghua Wu, Guosheng Lin, Qingyi Tao, and Jianfei Cai. 2019. M2e-try on net: Fashion from model to everyone. In Proceedings of the 27th ACM International Conference on Multimedia. 293--301.
[52]
Yuanlu Xu, Song-Chun Zhu, and Tony Tung. 2019. Denserac: Joint 3d pose and shape estimation by dense render-and-compare. In Proceedings of the IEEE International Conference on Computer Vision. 7760--7770.
[53]
Han Yang, Ruimao Zhang, Xiaobao Guo, Wei Liu, Wangmeng Zuo, and Ping Luo. 2020. Towards Photo-Realistic Virtual Try-On by Adaptively Generating- Preserving Image Content. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7850--7859.
[54]
Gokhan Yildirim, Nikolay Jetchev, Roland Vollgraf, and Urs Bergmann. 2019. Generating high-resolution fashion model images wearing custom outfits. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 0--0.
[55]
Ruiyun Yu, Xiaoqi Wang, and Xiaohui Xie. 2019. Vtnfp: An image-based virtual try-on network with body and clothing feature preservation. In Proceedings of the IEEE International Conference on Computer Vision. 10511--10520.
[56]
Heming Zhu, Yu Cao, Hang Jin, Weikai Chen, Dong Du, Zhangye Wang, Shuguang Cui, and Xiaoguang Han. 2020. Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images. arXiv preprint arXiv:2003.12753 (2020).

Cited By

View all
  • (2024)Shape-Guided Clothing Warping for Virtual Try-OnProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680756(2593-2602)Online publication date: 28-Oct-2024
  • (2024)Controlling Virtual Try-on Pipeline Through Rendering Policies2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00576(5854-5836)Online publication date: 3-Jan-2024
  • (2024)Toward Low Artifact Virtual Try-On Via Pre-Warping Partitioned Clothing Alignment2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647304(2264-2270)Online publication date: 27-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '21: Proceedings of the 29th ACM International Conference on Multimedia
October 2021
5796 pages
ISBN:9781450386517
DOI:10.1145/3474085
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. graph attention networks
  2. image warping
  3. virtual try-on

Qualifiers

  • Research-article

Funding Sources

Conference

MM '21
Sponsor:
MM '21: ACM Multimedia Conference
October 20 - 24, 2021
Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)24
  • Downloads (Last 6 weeks)1
Reflects downloads up to 03 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Shape-Guided Clothing Warping for Virtual Try-OnProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680756(2593-2602)Online publication date: 28-Oct-2024
  • (2024)Controlling Virtual Try-on Pipeline Through Rendering Policies2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00576(5854-5836)Online publication date: 3-Jan-2024
  • (2024)Toward Low Artifact Virtual Try-On Via Pre-Warping Partitioned Clothing Alignment2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647304(2264-2270)Online publication date: 27-Oct-2024
  • (2024)Image-Based Virtual Try-On: A SurveyInternational Journal of Computer Vision10.1007/s11263-024-02305-2Online publication date: 10-Dec-2024
  • (2023)Self-Adaptive Clothing Mapping Based Virtual Try-onACM Transactions on Multimedia Computing, Communications, and Applications10.1145/361345320:3(1-26)Online publication date: 23-Oct-2023
  • (2023)Deep Person Generation: A Survey from the Perspective of Face, Pose, and Cloth SynthesisACM Computing Surveys10.1145/357565655:12(1-37)Online publication date: 28-Mar-2023
  • (2023)LC-VTON: Length Controllable Virtual Try-on NetworkIEEE Access10.1109/ACCESS.2023.330644911(88451-88461)Online publication date: 2023
  • (2022)SPG-VTON: Semantic Prediction Guidance for Multi-Pose Virtual Try-onIEEE Transactions on Multimedia10.1109/TMM.2022.314371224(1233-1246)Online publication date: 2022
  • (2022)Weakly Supervised High-Fidelity Clothing Model Generation2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00343(3430-3439)Online publication date: Jun-2022
  • (2021)Smart Fashion: A Review of AI Applications in Virtual Try-On & Fashion SynthesisJournal of Artificial Intelligence and Capsule Networks10.36548/jaicn.2021.4.0023:4(284-304)Online publication date: 26-Nov-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media