Abstract
Current leading algorithms of the image-based virtual try-on systems mainly model the deformation of clothes as a whole. However, the deformation of different clothes parts can change drastically. Thus the existing algorithms fail to transfer the clothes to the proper shape in cases, such as self-occlusion, complex pose, and sophisticated textures. Based on this observation, we propose a Landmark-Guided Virtual Try-On Network (LG-VTON), which explicitly divides the clothes into regions using estimated landmarks, and performs a part-wise transformation using the Thin Plate Spline (TPS) for each region independently. The part-wise TPS transformation can be calculated according to the estimated landmarks. Finally, a virtual try-on sub-network is introduced to estimate the composition mask to fuse the wrapped clothes and person image to synthesize the try-on result. Extensive experiments on the virtual try-on dataset demonstrate that LG-VTON can handle complicated clothes deformation and synthesize satisfactory virtual try-on images, achieving state-of-the-art performance both qualitatively and quantitatively.
Supported by the Key Field R & D Program of Guangdong Province (2019B010155003) and the National Natural Science Foundation of China (61876104).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: Viton: an image-based virtual try-on network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7543–7552 (2018)
Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., Yang, M.: Toward characteristic-preserving image-based virtual try-on network. In: Proceedings the European Conference on Computer Vision (ECCV), pp. 589–604 (2018)
Han, X., Hu, X., Huang, W., Scott, M.R.: Clothflow: a flow-based model for clothed person generation. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 10471–10480 (2019)
Yu, R., Wang, X., Xie, X.: VTNFP: an image-based Virtual try-on network with body and clothing feature preservation. In: The IEEE International Conference on Computer Vision (ICCV), pp. 10511–10520 (2019)
Dong, H., Liang, X., Wang, B., Lai, H., Zhu, J., Yin, J.: Towards multi-pose guided virtual try-on network. In: The IEEE International Conference on Computer Vision (ICCV), pp. 9026–9035 (2019)
Zheng, N., Song, X., Chen, Z., Hu, L., Cao, D., Nie, L.: Virtually trying on new clothing with arbitrary poses. In: Proceedings of the 27th ACM International Conference on Multimedia (ACMMM), pp. 266–274 (2019)
Dong, H., Liang, X., Shen, X., Wu, B., Chen, B.-C., Yin, J.: FW-GAN: flow-navigated warping GAN for video virtual try-on. In: Proceedings the IEEE International Conference on Computer Vision (ICCV), pp. 1161–1170 (2019)
Bookstein, F.L.: Principal warps: thin-Plate Splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 11(6), 567–585 (1989)
Sekine, M., Sugita, K., Perbet, F., Stenger, B., Nishiyama, M.: Virtual fitting by single-shot body shape estimation. In: 3D Body Scanning Technologies, pp. 406–413 (2014)
Pons-Moll, G., Pujades, S., Hu, S., Black, M.J.: Clothcap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. (TOG) 36(4), 73 (2017)
Guan, P., Reiss, L., Hirshberg, D.A., Weiss, A., Black, M.J.: DRAPE: dressing any person. ACM Trans. Graph. 31(4), 35 (2012)
Yang, S., et al.: Detailed garment recovery from a single-view image. arXiv preprint (2016). arXiv:1608.01250
Johnson, S., Everingham, M.: Clustered pose and nonlinear appearance models for human pose estimation. In: Proceedings of the British Machine Vision Conference (2010). https://doi.org/10.5244/C.24.12201
Andriluka, M., Pishchulin, L., Gehler, P., Schiele, B.: 2D human pose estimation: new benchmark and state of the art analysis. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 3686–3693 (2014)
Lin, T.-Y.: Microsoft coco: common objects in context. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 3686–3693 (2014)
Li, W., et al.: Rethinking on multi-stage networks for human pose estimation. arXiv preprint (2019). arXiv:1901.00148
Chen, Y., Wang, Z., Peng, Y., Zhang, Z., Yu, G., Sun, J.: Cascaded pyramid network for multi-person pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7103–7112 (2018)
He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. IEEE Trans. Pattern Anal. Mach. Intell. 42(2), 386–397 (2020)
Liu, Z., Luo, P., Qiu, S., Wang, X., Tang, X.: DeepFashion: powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1096–1104 (2016)
Ge, Y., Zhang, R., Wu, L., Wang, X., Tang, X., Luo, P.: DeepFashion2: a versatile benchmark for detection, pose estimation, segmentation and re-identification of clothing images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5332–5340 (2019)
Gong, K., Gao, Y., Liang, X., Shen, X., Wang, M., Lin, L.: Graphonomy: universal human parsing via graph transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7442–7451 (2019)
Cao, Z., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310 (2017)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Proceedings the European Conference on Computer Vision (CVPR), pp. 694–711 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, Z., Lai, J., Xie, X. (2020). LG-VTON: Fashion Landmark Meets Image-Based Virtual Try-On. In: Peng, Y., et al. Pattern Recognition and Computer Vision. PRCV 2020. Lecture Notes in Computer Science(), vol 12307. Springer, Cham. https://doi.org/10.1007/978-3-030-60636-7_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-60636-7_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60635-0
Online ISBN: 978-3-030-60636-7
eBook Packages: Computer ScienceComputer Science (R0)