Abstract
Virtual fitting, in which a person’s image is changed to an arbitrary clothing image, is expected to be applied to shopping sites and videoconferencing. In real-time virtual fitting, image-based methods using a knowledge distillation technique can generate high-quality fitting images by inputting only the image of arbitrary clothing and a person without requiring the additional data like pose information. However, there are few studies that perform fast virtual fitting from arbitrary clothing images stably with real person images for situations such as videoconferencing considering temporal consistency. Therefore, the purpose of this demo is to perform robust virtual fitting with temporal consistency for videoconferencing. First, we created a virtual fitting system and verified how effective the existing fast image fitting method is for webcam video. The results showed that the existing methods do not adjust the dataset and do not consider temporal consistency, and thus are unstable for input images similar to videoconferencing. Therefore, we propose to train a model that adjusts the dataset to be similar to a videoconference and to add temporal consistency loss. Qualitative evaluation of the proposed model confirms that the model exhibits less flicker than the baseline. Figure 1 shows an example usage of our try-on system which is running on Zoom.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of European Conference on Computer Vision (2018)
Dong, H., Liang, X., Shen, X., Wu, B., Chen, B.C., Yin, J.: FW-GAN: Flow-navigated warping GAN for video virtual try-on. In: Proceedings of IEEE International Conference on Computer Vision (2019)
Ge, Y., Song, Y., Zhang, R., Ge, C., Liu, W., Luo, P.: Parser-free virtual try-on via distilling appearance flows. In: Proceedings of IEEE Computer Vision and Pattern Recognition, pp. 8485–8493 (2021)
Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: VITON: an image-based virtual try-on network. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2018)
Issenhuth, T., Mary, J., Calauzènes, C.: End-to-end learning of geometric deformations of feature maps for virtual try-on. arXiv:1906.01347 (2019)
Kuppa, G., Jong, A., Liu, V., Liu, Z., Moh, T.: ShineOn: illuminating design choices for practical video-based virtual clothing try-on. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision Workshops, pp. 191–200 (2021)
Minar, M.R., Tuan, T.T., Ahn, H., Rosin, P., Lai, Y.K.: CP-VTON+: clothing shape and texture preserving image-based virtual try-on. In: Proceedings of IEEE Computer Vision and Pattern Recognition (2020)
Wang, T.C., Liu, M.Y., Tao, A., Liu, G., Kautz, J., Catanzaro, B.: Few-shot video-to-video synthesis. In: Proceedings of Advances in Neural Information Processing Systems (2019)
Wei, D., Xu, X., Shen, H., Huang, K.: C2F-FWN: coarse-to-fine flow warping network for spatial-temporal consistent motion transfer. Proc. AAAI Conf. Artif. Intell. 35(4), 2852–2860 (2021)
Zhong, X., Wu, Z., Tan, T., Lin, G., Wu, Q.: MV-TON: memory-based video virtual try-on network. In: Proceedings of ACM International Conference Multimedia, pp. 908–916 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Shimizu, D., Yanai, K. (2023). Virtual Try-On Considering Temporal Consistency for Videoconferencing. In: Dang-Nguyen, DT., et al. MultiMedia Modeling. MMM 2023. Lecture Notes in Computer Science, vol 13834. Springer, Cham. https://doi.org/10.1007/978-3-031-27818-1_64
Download citation
DOI: https://doi.org/10.1007/978-3-031-27818-1_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-27817-4
Online ISBN: 978-3-031-27818-1
eBook Packages: Computer ScienceComputer Science (R0)