Skip to main content
SpringerLink
Log in

MagicMirror: A 3-D Real-Time Virtual Try-On System Through Cloth Simulation

  • Conference paper
  • First Online:
Advances in Computer Graphics (CGI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14496))

Included in the following conference series:

  • 400 Accesses

Abstract

Nowadays, with the increasing development of online shopping, there exists a huge latent benefit area in clothing e-commerce. It has been leading the application of emerging technologies to this field. However, online shopping can not intuitively feel the material of clothes fabric and the dynamic effect of trying on clothes. Methodologies based on cloth simulation and human-computer interaction can be used to solve this challenge. In this paper, we proposed a virtual try-on system based cloth simulation technique to tackle the realism of cloth, using physical law in garment to strengthen the realism of virtual try-on and integrated markless motion capture technique realized by common RGB-D camera to synchronize movement of models and people. We also adopt a GPU acceleration solution to ensure real-time simulation. We realized the system based Unity3D using TaiChi Programming Language to control and stimulate the garment. And we verify the significance of GPU acceleration and conduct several experiments to prove the real-time performance of the simulation-based virtual try-on system. We compared the simulation time on CPU and GPU and validated the accuracy of motion capture satisfying virtual try-on task. In the end we conducted a user study to find out if the average consumer was satisfied with our proposed virtual try-on system.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. An, S., et al.: ARShoe: real-time augmented reality shoe try-on system on smartphones. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 1111–1119 (2021)

    Google Scholar 

  2. Baraff, D.: Large steps in cloth simulation. In: Proc. SIGGRAPH ’98, pp. 43–54 (1998)

    Google Scholar 

  3. Bhatnagar, B., Tiwari, G., Theobalt, C., Pons-Moll, G.: Multi-garment net: Learning to dress 3D people from images. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5419–5429 (2019). https://doi.org/10.1109/ICCV.2019.00552

  4. Black, M.J., Matthew, L., Naureen, M., Gerard, P.M., Javier, R.: Skinned multi-person linear model. ACM (2015)

    Google Scholar 

  5. Bridson, R.E., Marino, S., Fedkiw, R.P.: Simulation of clothing with folds and wrinkles (2005)

    Google Scholar 

  6. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. 43(1), 172–186 (2021)

    Article  Google Scholar 

  7. Chen, X., Zhou, B., Lu, F., Wang, L., Bi, L., Tan, P.: Garment modeling with a depth camera. ACM Trans. Graph. (TOG) 34(6), 1–12 (2015)

    Google Scholar 

  8. Choi, K.J., Ko, H.S.: Stable but responsive cloth. In: ACM SIGGRAPH 2005 Courses, pp. 1-es (2005)

    Google Scholar 

  9. Cirio, G., Lopez-Moreno, J., Miraut, D., Otaduy, M.A.: Yarn-level simulation of woven cloth. ACM Trans. Graph. (TOG) 33(6), 1–11 (2014)

    Article  Google Scholar 

  10. Du, C., et al.: VTON-SCFA: a virtual try-on network based on the semantic constraints and flow alignment. IEEE Trans. Multimedia 25, 777–791 (2022)

    Article  Google Scholar 

  11. Du, C., Yu, F., Jiang, M., Wei, X., Peng, T., Hu, X.: Multi-pose virtual try-on via self-adaptive feature filtering. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2544–2548 (2022). https://doi.org/10.1109/ICASSP43922.2022.9747847

  12. Fratarcangeli, M., Pellacini, F.: Scalable partitioning for parallel position based dynamics. In: Computer Graphics Forum. vol. 34, pp. 405–413. Wiley Online Library (2015)

    Google Scholar 

  13. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  14. Han, X., Wu, Z., Wu, Z., Yu, R., Davis, L.S.: VITON: an image-based virtual try-on network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7543–7552 (2018). https://doi.org/10.1109/CVPR.2018.00787

  15. Hauswiesner, S., Straka, M., Reitmayr, G.: Virtual try-on through image-based rendering. IEEE Trans. Visual Comput. Graph. 19(9), 1552–1565 (2013)

    Article  Google Scholar 

  16. Kaldor, J.M., James, D.L., Marschner, S.: Efficient yarn-based cloth with adaptive contact linearization. In: ACM SIGGRAPH 2010 Papers, pp. 1–10 (2010)

    Google Scholar 

  17. Kim, D., Koh, W., Narain, R., Fatahalian, K., Treuille, A., O’Brien, J.F.: Near-exhaustive precomputation of secondary cloth effects. ACM Trans. Graph. (TOG) 32(4), 1–8 (2013)

    Google Scholar 

  18. Kocabas, M., Athanasiou, N., Black, M.J.: Vibe: video inference for human body pose and shape estimation. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5252–5262 (2020). https://doi.org/10.1109/CVPR42600.2020.00530

  19. Liu, T., Bargteil, A.W., O’Brien, J.F., Kavan, L.: Fast simulation of mass-spring systems. ACM Trans. Graph. (TOG) 32(6), 1–7 (2013)

    Google Scholar 

  20. Manfredi, G., Capece, N., Erra, U., Gilio, G., Baldi, V., Di Domenico, S.G.: TryItOn: a virtual dressing room with motion tracking and physically based garment simulation. In: De Paolis, L.T., Arpaia, P., Sacco, M. (eds.) XR Salento 2022. LNCS, vol. 13445, pp. 63–76. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-15546-8_5

    Chapter  Google Scholar 

  21. Mir, A., Alldieck, T., Pons-Moll, G.: Learning to transfer texture from clothing images to 3D humans. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7021–7032 (2020). https://doi.org/10.1109/CVPR42600.2020.00705

  22. Müller, M.: Hierarchical position based dynamics. DBLP (2008)

    Google Scholar 

  23. Patel, C., Liao, Z., Pons-Moll, G.: TailorNet: predicting clothing in 3D as a function of human pose, shape and garment style. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7363–7373 (2020). https://doi.org/10.1109/CVPR42600.2020.00739

  24. Pons-Moll, G., Pujades, S., Sonny, H.U., Black, M.J.: ClothCap: seamless 4D clothing capture and retargeting. ACM Trans. Graph. 36(4CD), 1–15 (2017)

    Google Scholar 

  25. Provot, X.: Deformation constraints in a mass-spring model to describe rigid cloth behavior (2001)

    Google Scholar 

  26. Selle, A., Su, J., Irving, G., Fedkiw, R.: Robust high-resolution cloth using parallelism, history-based collisions, and accurate friction. IEEE Trans. Visual Comput. Graph. 15(2), 339–350 (2009). https://doi.org/10.1109/TVCG.2008.79

    Article  Google Scholar 

  27. Stoll, C., Gall, J., de Aguiar, E., Thrun, S., Theobalt, C.: Video-based reconstruction of animatable human characters. ACM Trans. Graph. 29(6), 1–10 (2010). https://doi.org/10.1145/1882261.1866161

  28. Terzopoulos, D., Platt, J., Barr, A., Fleischer, K.: Elastically deformable models. In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, pp. 205–214 (1987)

    Google Scholar 

  29. Volino, P., Magnenat-Thalmann, N., Faure, F.: A simple approach to nonlinear tensile stiffness for accurate cloth simulation. ACM Trans. Graph. 28(4), Article-No (2009)

    Google Scholar 

  30. Wang, B., Zheng, H., Liang, X., Chen, Y., Lin, L., Yang, M.: Toward characteristic-preserving image-based virtual try-on network. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  31. Wang, Z., Wu, L., Fratarcangeli, M., Tang, M., Wang, H.: Parallel multigrid for nonlinear cloth simulation. In: Computer Graphics Forum, vol. 37, pp. 131-141 (2018). https://doi.org/10.1111/cgf.13554

  32. Wu, Y.L.: One pose fits all: a novel kinematic approach to 3D human pose estimation (2021)

    Google Scholar 

  33. Yamamoto, H., Moriya, T., Takahashi, T.: Development of a virtual try-on system for eyeglasses considering lens distortion. In: International Workshop on Advanced Imaging Technology (IWAIT) 2021, vol. 11766, pp. 409–414. SPIE (2021)

    Google Scholar 

  34. Zhao, F., et al.: M3D-VTON: a monocular-to-3D virtual try-on network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13239–13249 (2021)

    Google Scholar 

  35. Zhou, B., Chen, X., Fu, Q., Guo, K., Tan, P.: Garment modeling from a single image. In: Computer Graphics Forum, vol. 32, pp. 85–91. Wiley Online Library (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Wuhan Textile University, Wuhan, China

    Zhanyi Huang, Wenqing Zhao, Tangsheng Guo & Jin Huang

  2. Hong Kong Polytechnic University, Kowloon, Hong Kong

    Ping Li

  3. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

Authors
  1. Zhanyi Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenqing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tangsheng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ping Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bin Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jin Huang .

Editor information

Editors and Affiliations

  1. Shanghai Jiao Tong University, Shanghai, China

    Bin Sheng

  2. Shanghai Jiao Tong University, Shanghai, China

    Lei Bi

  3. University of Sydney, Sydney, NSW, Australia

    Jinman Kim

  4. MIRALab-CUI, University of Geneve, Carouge, Geneve, Switzerland

    Nadia Magnenat-Thalmann

  5. Swiss Federal Institute of Technology, Lausanne, Switzerland

    Daniel Thalmann

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Huang, Z., Zhao, W., Guo, T., Huang, J., Li, P., Sheng, B. (2024). MagicMirror: A 3-D Real-Time Virtual Try-On System Through Cloth Simulation. In: Sheng, B., Bi, L., Kim, J., Magnenat-Thalmann, N., Thalmann, D. (eds) Advances in Computer Graphics. CGI 2023. Lecture Notes in Computer Science, vol 14496. Springer, Cham. https://doi.org/10.1007/978-3-031-50072-5_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50072-5_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50071-8

  • Online ISBN: 978-3-031-50072-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation