Skip to main content

LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2020)

Abstract

Current leading algorithms of the image-based virtual try-on systems mainly model the deformation of clothes as a whole. However, the deformation of different clothes parts can change drastically. Thus the existing algorithms fail to transfer the clothes to the proper shape in cases, such as self-occlusion, complex pose, and sophisticated textures. Based on this observation, we propose a Landmark-Guided Virtual Try-On Network (LG-VTON), which explicitly divides the clothes into regions using estimated landmarks, and performs a part-wise transformation using the Thin Plate Spline (TPS) for each region independently. The part-wise TPS transformation can be calculated according to the estimated landmarks. Finally, a virtual try-on sub-network is introduced to estimate the composition mask to fuse the wrapped clothes and person image to synthesize the try-on result. Extensive experiments on the virtual try-on dataset demonstrate that LG-VTON can handle complicated clothes deformation and synthesize satisfactory virtual try-on images, achieving state-of-the-art performance both qualitatively and quantitatively.

Supported by the Key Field R & D Program of Guangdong Province (2019B010155003) and the National Natural Science Foundation of China (61876104).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: Viton: an image-based virtual try-on network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7543–7552 (2018)

    Google Scholar 

  2. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., Yang, M.: Toward characteristic-preserving image-based virtual try-on network. In: Proceedings the European Conference on Computer Vision (ECCV), pp. 589–604 (2018)

    Google Scholar 

  3. Han, X., Hu, X., Huang, W., Scott, M.R.: Clothflow: a flow-based model for clothed person generation. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 10471–10480 (2019)

    Google Scholar 

  4. Yu, R., Wang, X., Xie, X.: VTNFP: an image-based Virtual try-on network with body and clothing feature preservation. In: The IEEE International Conference on Computer Vision (ICCV), pp. 10511–10520 (2019)

    Google Scholar 

  5. Dong, H., Liang, X., Wang, B., Lai, H., Zhu, J., Yin, J.: Towards multi-pose guided virtual try-on network. In: The IEEE International Conference on Computer Vision (ICCV), pp. 9026–9035 (2019)

    Google Scholar 

  6. Zheng, N., Song, X., Chen, Z., Hu, L., Cao, D., Nie, L.: Virtually trying on new clothing with arbitrary poses. In: Proceedings of the 27th ACM International Conference on Multimedia (ACMMM), pp. 266–274 (2019)

    Google Scholar 

  7. Dong, H., Liang, X., Shen, X., Wu, B., Chen, B.-C., Yin, J.: FW-GAN: flow-navigated warping GAN for video virtual try-on. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 1161–1170 (2019)

    Google Scholar 

  8. Bookstein, F.L.: Principal warps: thin-Plate Splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)

    Article  Google Scholar 

  9. Sekine, M., Sugita, K., Perbet, F., Stenger, B., Nishiyama, M.: Virtual fitting by single-shot body shape estimation. In: 3D Body Scanning Technologies, pp. 406–413 (2014)

    Google Scholar 

  10. Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (TOG) 36(4), 73 (2017)

    Article  Google Scholar 

  11. Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: DRAPE: dressing any person. ACM Trans. Graph. 31(4), 35 (2012)

    Article  Google Scholar 

  12. Yang, S., et al.: Detailed garment recovery from a single-view image. arXiv preprint (2016). arXiv:1608.01250

  13. Johnson, S., Everingham, M.: Clustered pose and nonlinear appearance models for human pose estimation. In: Proceedings of the British Machine Vision Conference (2010). https://doi.org/10.5244/C.24.12201

  14. Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 3686–3693 (2014)

    Google Scholar 

  15. Lin, T.-Y.: Microsoft coco: common objects in context. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 3686–3693 (2014)

    Google Scholar 

  16. Li, W., et al.: Rethinking on multi-stage networks for human pose estimation. arXiv preprint (2019). arXiv:1901.00148

  17. Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7103–7112 (2018)

    Google Scholar 

  18. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 386–397 (2020)

    Article  Google Scholar 

  19. Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1096–1104 (2016)

    Google Scholar 

  20. Ge, Y., Zhang, R., Wu, L., Wang, X., Tang, X., Luo, P.: DeepFashion2: a versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5332–5340 (2019)

    Google Scholar 

  21. Gong, K., Gao, Y., Liang, X., Shen, X., Wang, M., Lin, L.: Graphonomy: universal human parsing via graph transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7442–7451 (2019)

    Google Scholar 

  22. Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310 (2017)

    Google Scholar 

  23. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 694–711 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jianhuang Lai .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, Z., Lai, J., Xie, X. (2020). LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12307. Springer, Cham. https://doi.org/10.1007/978-3-030-60636-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60636-7_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60635-0

  • Online ISBN: 978-3-030-60636-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics