Skip to main content
SpringerLink
Log in

LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12307))

Included in the following conference series:

Abstract

Current leading algorithms of the image-based virtual try-on systems mainly model the deformation of clothes as a whole. However, the deformation of different clothes parts can change drastically. Thus the existing algorithms fail to transfer the clothes to the proper shape in cases, such as self-occlusion, complex pose, and sophisticated textures. Based on this observation, we propose a Landmark-Guided Virtual Try-On Network (LG-VTON), which explicitly divides the clothes into regions using estimated landmarks, and performs a part-wise transformation using the Thin Plate Spline (TPS) for each region independently. The part-wise TPS transformation can be calculated according to the estimated landmarks. Finally, a virtual try-on sub-network is introduced to estimate the composition mask to fuse the wrapped clothes and person image to synthesize the try-on result. Extensive experiments on the virtual try-on dataset demonstrate that LG-VTON can handle complicated clothes deformation and synthesize satisfactory virtual try-on images, achieving state-of-the-art performance both qualitatively and quantitatively.

Supported by the Key Field R & D Program of Guangdong Province (2019B010155003) and the National Natural Science Foundation of China (61876104).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: Viton: an image-based virtual try-on network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7543–7552 (2018)

    Google Scholar 

  2. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., Yang, M.: Toward characteristic-preserving image-based virtual try-on network. In: Proceedings the European Conference on Computer Vision (ECCV), pp. 589–604 (2018)

    Google Scholar 

  3. Han, X., Hu, X., Huang, W., Scott, M.R.: Clothflow: a flow-based model for clothed person generation. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 10471–10480 (2019)

    Google Scholar 

  4. Yu, R., Wang, X., Xie, X.: VTNFP: an image-based Virtual try-on network with body and clothing feature preservation. In: The IEEE International Conference on Computer Vision (ICCV), pp. 10511–10520 (2019)

    Google Scholar 

  5. Dong, H., Liang, X., Wang, B., Lai, H., Zhu, J., Yin, J.: Towards multi-pose guided virtual try-on network. In: The IEEE International Conference on Computer Vision (ICCV), pp. 9026–9035 (2019)

    Google Scholar 

  6. Zheng, N., Song, X., Chen, Z., Hu, L., Cao, D., Nie, L.: Virtually trying on new clothing with arbitrary poses. In: Proceedings of the 27th ACM International Conference on Multimedia (ACMMM), pp. 266–274 (2019)

    Google Scholar 

  7. Dong, H., Liang, X., Shen, X., Wu, B., Chen, B.-C., Yin, J.: FW-GAN: flow-navigated warping GAN for video virtual try-on. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 1161–1170 (2019)

    Google Scholar 

  8. Bookstein, F.L.: Principal warps: thin-Plate Splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)

    Article  Google Scholar 

  9. Sekine, M., Sugita, K., Perbet, F., Stenger, B., Nishiyama, M.: Virtual fitting by single-shot body shape estimation. In: 3D Body Scanning Technologies, pp. 406–413 (2014)

    Google Scholar 

  10. Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (TOG) 36(4), 73 (2017)

    Article  Google Scholar 

  11. Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: DRAPE: dressing any person. ACM Trans. Graph. 31(4), 35 (2012)

    Article  Google Scholar 

  12. Yang, S., et al.: Detailed garment recovery from a single-view image. arXiv preprint (2016). arXiv:1608.01250

  13. Johnson, S., Everingham, M.: Clustered pose and nonlinear appearance models for human pose estimation. In: Proceedings of the British Machine Vision Conference (2010). https://doi.org/10.5244/C.24.12201

  14. Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 3686–3693 (2014)

    Google Scholar 

  15. Lin, T.-Y.: Microsoft coco: common objects in context. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 3686–3693 (2014)

    Google Scholar 

  16. Li, W., et al.: Rethinking on multi-stage networks for human pose estimation. arXiv preprint (2019). arXiv:1901.00148

  17. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7103–7112 (2018)

    Google Scholar 

  18. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 386–397 (2020)

    Article  Google Scholar 

  19. Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1096–1104 (2016)

    Google Scholar 

  20. Ge, Y., Zhang, R., Wu, L., Wang, X., Tang, X., Luo, P.: DeepFashion2: a versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5332–5340 (2019)

    Google Scholar 

  21. Gong, K., Gao, Y., Liang, X., Shen, X., Wang, M., Lin, L.: Graphonomy: universal human parsing via graph transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7442–7451 (2019)

    Google Scholar 

  22. Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310 (2017)

    Google Scholar 

  23. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 694–711 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, China

    Zhenyu Xie, Jianhuang Lai & Xiaohua Xie

  2. School of Information Science, Xinhua College of Sun Yat-Sen University, Guangzhou, China

    Jianhuang Lai

Authors
  1. Zhenyu Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianhuang Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaohua Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhuang Lai .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  3. Dalian University of Technology, Dalian, China

    Huchuan Lu

  4. Chinese Academy of Sciences, Beijing, China

    Zhenan Sun

  5. Chinese Academy of Sciences, Beijing, China

    Chenglin Liu

  6. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Peking University, Beijing, China

    Hongbin Zha

  8. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, Z., Lai, J., Xie, X. (2020). LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12307. Springer, Cham. https://doi.org/10.1007/978-3-030-60636-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60636-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60635-0

  • Online ISBN: 978-3-030-60636-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation