Zhen Luo
ABSTRACT
Recent progress in learning-based methods of garment mesh generation is resulting in increased efficiency and maintenance of reality during the generation process. However, none of the previous works so far have focused on variations in material types based on a parameterized material parameter under static poses. In this work, we propose a learning-based method, GarMatNet, for predicting garment deformation based on the functions of human poses and garment materials while maintaining detailed garment wrinkles. GarMatNet consists of two components: a generally-fitting network for predicting smoothed garment mesh and a locally-detailed network for adding detailed wrinkles based on smoothed garment mesh. We hypothesize that material properties play an essential role in the deformation of garments. Since the influences of material type are relatively smaller than pose or body shape, we employ linear interpolation among different factors to control deformation. More specifically, we apply a parameterized material space based on the mass-spring model to express the difference between materials and construct a suitable network structure with weight adjustment between material properties and poses. The experimental results demonstrate that GarMatNet is comparable to the physically-based simulation (PBS) prediction and offers advantages regarding generalization ability, model size, and training time over the baseline model.
Supplemental Material
- Thiemo Alldieck, Marcus Magnor, Bharat Lal Bhatnagar, Christian Theobalt, and Gerard Pons-Moll. 2019. Learning to Reconstruct People in Clothing From a Single RGB Camera. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1175–1186. https://doi.org/10.1109/CVPR.2019.00127Google Scholar
Cross Ref
- David Baraff and Andrew Witkin. 1998. Large Steps in Cloth Simulation. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’98). Association for Computing Machinery, New York, NY, USA, 43–54. https://doi.org/10.1145/280814.280821Google ScholarDigital Library
- Bharat Lal Bhatnagar, Garvita Tiwari, Christian Theobalt, and Gerard Pons-Moll. 2019. Multi-Garment Net: Learning to Dress 3D People From Images. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 5419–5429. https://doi.org/10.1109/ICCV.2019.00552Google ScholarCross Ref
- David E. Breen, Donald H. House, and Michael J. Wozny. 1994. Predicting the Drape of Woven Cloth Using Interacting Particles. In Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’94). Association for Computing Machinery, New York, NY, USA, 365–372. https://doi.org/10.1145/192161.192259Google ScholarDigital Library
- Dan Casas and Miguel A. Otaduy. 2018. Learning Nonlinear Soft-Tissue Dynamics for Interactive Avatars. Proc. ACM Comput. Graph. Interact. Tech. 1, 1, Article 10 (July 2018), 15 pages. https://doi.org/10.1145/3203187Google ScholarDigital Library
- CLO Virtual Fashion Inc.2021. Marvelous Designer. https://www.marvelousdesigner.com/ Accessed June, 2021.Google Scholar
- David Clyde, Joseph Teran, and Rasmus Tamstorf. 2017. Modeling and Data-Driven Parameter Estimation for Woven Fabrics. In Proceedings of the ACM SIGGRAPH / Eurographics Symposium on Computer Animation (Los Angeles, California) (SCA ’17). Association for Computing Machinery, New York, NY, USA, Article 17, 11 pages. https://doi.org/10.1145/3099564.3099577Google ScholarDigital Library
- Edilson de Aguiar, Leonid Sigal, Adrien Treuille, and Jessica K. Hodgins. 2010. Stable Spaces for Real-Time Clothing. ACM Trans. Graph. 29, 4, Article 106 (July 2010), 9 pages. https://doi.org/10.1145/1778765.1778843Google ScholarDigital Library
- Marco Fratarcangeli, Valentina Tibaldo, and Fabio Pellacini. 2016. Vivace: A Practical Gauss-Seidel Method for Stable Soft Body Dynamics. ACM Trans. Graph. 35, 6, Article 214 (Nov. 2016), 9 pages. https://doi.org/10.1145/2980179.2982437Google ScholarDigital Library
- Erhan Gundogdu, Victor Constantin, Amrollah Seifoddini, Minh Dang, Mathieu Salzmann, and Pascal Fua. 2019. GarNet: A Two-Stream Network for Fast and Accurate 3D Cloth Draping. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 8738–8747. https://doi.org/10.1109/ICCV.2019.00883Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society, Los Alamitos, CA, USA, 1026–1034. https://doi.org/10.1109/ICCV.2015.123Google ScholarDigital Library
- Chenfanfu Jiang, Theodore Gast, and Joseph Teran. 2017. Anisotropic Elastoplasticity for Cloth, Knit and Hair Frictional Contact. ACM Trans. Graph. 36, 4, Article 152 (July 2017), 14 pages. https://doi.org/10.1145/3072959.3073623Google ScholarDigital Library
- J. P. Lewis, Matt Cordner, and Nickson Fong. 2000. Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Driven Deformation. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’00). ACM Press/Addison-Wesley Publishing Co., USA, 165–172. https://doi.org/10.1145/344779.344862Google ScholarDigital Library
- Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2015. SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graph. 34, 6, Article 248 (Oct. 2015), 16 pages. https://doi.org/10.1145/2816795.2818013Google ScholarDigital Library
- Qianli Ma, Jinlong Yang, Anurag Ranjan, Sergi Pujades, Gerard Pons-Moll, Siyu Tang, and Michael J. Black. 2020. Learning to Dress 3D People in Generative Clothing. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 6468–6477. https://doi.org/10.1109/CVPR42600.2020.00650Google ScholarCross Ref
- Eder Miguel, Derek Bradley, Bernhard Thomaszewski, Bernd Bickel, Wojciech Matusik, Miguel A. Otaduy, and Steve Marschner. 2012. Data-Driven Estimation of Cloth Simulation Models. http://www.gmrv.es/Publications/2012/MBTBMOM12Google Scholar
- Matthias Müller, Bruno Heidelberger, Marcus Hennix, and John Ratcliff. 2007. Position based dynamics. Journal of Visual Communication and Image Representation 18, 2(2007), 109–118. https://doi.org/10.1016/j.jvcir.2007.01.005Google ScholarDigital Library
- Andrew Nealen, Matthias Müller, Richard Keiser, Eddy Boxerman, and Mark Carlson. 2006. Physically Based Deformable Models in Computer Graphics. Computer Graphics Forum 25, 4 (2006), 809–836. https://doi.org/10.1111/j.1467-8659.2006.01000.xGoogle ScholarCross Ref
- Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdfGoogle ScholarDigital Library
- Chaitanya Patel, Zhouyingcheng Liao, and Gerard Pons-Moll. 2020. TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 7363–7373. https://doi.org/10.1109/CVPR42600.2020.00739Google Scholar
- Xavier Provot. 1995. Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behaviour. In Proceedings of Graphics Interface ’95 (Quebec, Quebec, Canada) (GI ’95). Canadian Human-Computer Communications Society, Toronto, Ontario, Canada, 147–154. http://graphicsinterface.org/wp-content/uploads/gi1995-17.pdfGoogle Scholar
- Igor Santesteban, Miguel A. Otaduy, and Dan Casas. 2019. Learning-Based Animation of Clothing for Virtual Try-On. Computer Graphics Forum 38, 2 (2019), 355–366. https://doi.org/10.1111/cgf.13643Google ScholarCross Ref
- Andrew Selle, Jonathan Su, Geoffrey Irving, and Ronald Fedkiw. 2009. Robust High-Resolution Cloth Using Parallelism, History-Based Collisions, and Accurate Friction. IEEE Transactions on Visualization and Computer Graphics 15, 2 (March 2009), 339–350. https://doi.org/10.1109/TVCG.2008.79Google ScholarDigital Library
- Demetri Terzopoulos, John Platt, Alan Barr, and Kurt Fleischer. 1987. Elastically Deformable Models. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’87). Association for Computing Machinery, New York, NY, USA, 205–214. https://doi.org/10.1145/37401.37427Google ScholarDigital Library
- J. Vollmer, R. Mencl, and H. Müller. 1999. Improved Laplacian Smoothing of Noisy Surface Meshes. Computer Graphics Forum 18, 3 (1999), 131–138.Google ScholarCross Ref
- Huamin Wang, Ravi Ramamoorthi, and James F. O’Brien. 2011. Data-Driven Elastic Models for Cloth: Modeling and Measurement. ACM Transactions on Graphics 30, 4 (July 2011), 71:1–11. http://graphics.berkeley.edu/papers/Wang-DDE-2011-08/ Proceedings of ACM SIGGRAPH 2011, Vancouver, BC Canada.Google ScholarDigital Library
- Huamin Wang and Yin Yang. 2016. Descent Methods for Elastic Body Simulation on the GPU. ACM Trans. Graph. 35, 6, Article 212 (Nov. 2016), 10 pages. https://doi.org/10.1145/2980179.2980236Google ScholarDigital Library
- Tuanfeng Y. Wang, Duygu Ceylan, Jovan Popović, and Niloy J. Mitra. 2018. Learning a Shared Shape Space for Multimodal Garment Design. ACM Trans. Graph. 37, 6, Article 203 (Dec. 2018), 13 pages. https://doi.org/10.1145/3272127.3275074Google ScholarDigital Library
- Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. https://doi.org/10.1109/TIP.2003.819861Google ScholarDigital Library
- Meng Zhang, Tuanfeng Y. Wang, Duygu Ceylan, and Niloy J. Mitra. 2021. Deep Detail Enhancement for Any Garment. Computer Graphics Forum 40, 2 (2021), 399–411. https://doi.org/10.1111/cgf.142642Google ScholarCross Ref
Recommendations
A Layered Model of Human Body and Garment Deformation
3DV '14: Proceedings of the 2014 2nd International Conference on 3D Vision - Volume 01In this paper we present a framework for learning a three layered model of human shape, pose and garment deformation. The proposed deformation model provides intuitive control over the three parameters independently, while producing aesthetically ...
Automatic 3D Garment Fitting Based on Skeleton Driving
Advances in Multimedia Information Processing – PCM 2018AbstractIn this paper, we propose an automated 3D garment fitting method. The proposed method can automatically position a 3D garment model onto a human model without any user manual intervention. Given a garment model and a human model with various ...
Animation of Refitted 3D Garment Models for Reshaped Bodies
ICVRV '14: Proceedings of the 2014 International Conference on Virtual Reality and VisualizationWe present a framework for a virtual try-on system. Starting with a template human body mesh, we reshape it to acquire meshes of various body dimensions according to the user specified parameters. These reshaped bodies are further processed in two ...
Comments