research-article

AutoHair: fully automatic hair modeling from a single image

Authors:

Kun ZhouAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 35, Issue 4

Article No.: 116, Pages 1 - 12

https://doi.org/10.1145/2897824.2925961

Published: 11 July 2016 Publication History

Abstract

We introduce AutoHair, the first fully automatic method for 3D hair modeling from a single portrait image, with no user interaction or parameter tuning. Our method efficiently generates complete and high-quality hair geometries, which are comparable to those generated by the state-of-the-art methods, where user interaction is required. The core components of our method are: a novel hierarchical deep neural network for automatic hair segmentation and hair growth direction estimation, trained over an annotated hair image database; and an efficient and automatic data-driven hair matching and modeling algorithm, based on a large set of 3D hair exemplars. We demonstrate the efficacy and robustness of our method on Internet photos, resulting in a database of around 50K 3D hair models and a corresponding hairstyle space that covers a wide variety of real-world hairstyles. We also show novel applications enabled by our method, including 3D hairstyle space navigation and hair-aware image retrieval.

Supplementary Material

ZIP File (a116-chai-supp.zip)

Supplemental files.

Download
113.18 MB

MP4 File (a116.mp4)

Download
274.06 MB

References

[1]

Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In CVPR, 1--8.

[2]

Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH '99, 187--194.

Digital Library

[3]

Bookstein, F. L. 1989. Principal warps: thin-plate splines and the decomposition of deformations. Trans. PAMI 11, 6, 567--585.

Digital Library

[4]

Cao, X., Wei, Y., Wen, F., and Sun, J. 2012. Face alignment by explicit shape regression. In CVPR, 2887--2894.

Digital Library

[5]

Chai, M., Wang, L., Weng, Y., Yu, Y., Guo, B., and Zhou, K. 2012. Single-view hair modeling for portrait manipulation. ACM Trans. Graph. 31, 4, 116:1--116:8.

Digital Library

[6]

Chai, M., Wang, L., Weng, Y., Jin, X., and Zhou, K. 2013. Dynamic hair manipulation in images and videos. ACM Trans. Graph. 32, 4, 75:1--75:8.

Digital Library

[7]

Chai, M., Luo, L., Sunkavalli, K., Carr, N., Hadap, S., and Zhou, K. 2015. High-quality hair modeling from a single portrait photo. ACM Trans. Graph. 34, 6, 204:1--204:10.

Digital Library

[8]

Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A. L. 2015. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR.

[9]

Cheng, M.-M., Zheng, S., Lin, W.-Y., Vineet, V., Sturgess, P., Crook, N., Mitra, N. J., and Torr, P. 2014. Imagespirit: Verbal guided image parsing. ACM Trans. Graph. 34, 1, 3:1--3:11.

Digital Library

[10]

Dai, J., He, K., and Sun, J. 2015. BoxSup: Exploiting bounding boxes to supervise convolutional networks for semantic segmentation. In ICCV, 1635--1643.

Digital Library

[11]

Desbrun, M., Meyer, M., Schröder, P., and Barr, A. H. 1999. Implicit fairing of irregular meshes using diffusion and curvature flow. In Proc. SIGGRAPH '99, 317--324.

Digital Library

[12]

Echevarria, J. I., Bradley, D., Gutierrez, D., and Beeler, T. 2014. Capturing and stylizing hair for 3D fabrication. ACM Trans. Graph. 33, 4, 125:1--125:11.

Digital Library

[13]

Electronic Arts, 2014. The Sims Resource. http://www.thesimsresource.com/.

[14]

Forney Jr, G. D. 1973. The Viterbi algorithm. Proceedings of the IEEE 61, 3, 268--278.

[15]

Gephi, 2016. The Open Graph Viz Platform. https://gephi.org.

[16]

Girshick, R., Donahue, J., Darrell, T., and Malik, J. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 580--587.

Digital Library

[17]

Herrera, T. L., Zinke, A., and Weber, A. 2012. Lighting hair from the inside: A thermal approach to hair reconstruction. ACM Trans. Graph. 31, 6, 146:1--146:9.

Digital Library

[18]

Hu, L., Ma, C., Luo, L., and Li, H. 2014. Robust hair capture using simulated examples. ACM Trans. Graph. 33, 4, 126:1--126:10.

Digital Library

[19]

Hu, L., Ma, C., Luo, L., Wei, L.-Y., and Li, H. 2014. Capturing braided hairstyles. ACM Trans. Graph. 33, 6, 225:1--225:9.

Digital Library

[20]

Hu, L., Ma, C., Luo, L., and Li, H. 2015. Single-view hair modeling using a hairstyle database. ACM Trans. Graph. 34, 4, 125:1--125:9.

Digital Library

[21]

Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.-Y., Teng, S.-H., Bao, H., Guo, B., and Shum, H.-Y. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3, 1126--1134.

Digital Library

[22]

Huang, S.-S., Shamir, A., Shen, C.-H., Zhang, H., Sheffer, A., Hu, S.-M., and Cohen-Or, D. 2013. Qualitative organization of collections of shapes via quartet analysis. ACM Trans. Graph. 32, 4, 71:1--71:10.

Digital Library

[23]

Hueting, M., Ovsjanikov, M., and Mitra, N. J. 2015. CrossLink: Joint understanding of image and 3D model collections through shape and camera pose variations. ACM Trans. Graph. 34, 6, 233:1--233:13.

Digital Library

[24]

Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1--164:9.

Digital Library

[25]

Jia, Y., 2013. Caffe: An open source convolutional architecture for fast feature embedding.

[26]

Karypis, G., and Kumar, V. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on scientific Computing 20, 1, 359--392.

Digital Library

[27]

Kim, V. G., Li, W., Mitra, N. J., DiVerdi, S., and Funkhouser, T. 2012. Exploring collections of 3D models using fuzzy correspondences. ACM Trans. Graph. 31, 4, 54:1--54:11.

Digital Library

[28]

Krähenbühl, P., and Koltun, V. 2011. Efficient inference in fully connected CRFs with Gaussian edge potentials. In NIPS, 109--117.

[29]

Li, Y., Su, H., Qi, C. R., Fish, N., Cohen-Or, D., and Guibas, L. J. 2015. Joint embeddings of shapes and images via CNN image purification. ACM Trans. Graph. 34, 6, 234:1--234:12.

Digital Library

[30]

Ling, H., and Okada, K. 2007. An efficient earth mover's distance algorithm for robust histogram comparison. Trans. PAMI 29, 5, 840--853.

Digital Library

[31]

Liu, J., Sun, J., and Shum, H.-Y. 2009. Paint selection. ACM Trans. Graph. 28, 3, 69:1--69:7.

Digital Library

[32]

Liu, Z., Li, X., Luo, P., Loy, C. C., and Tang, X. 2015. Semantic image segmentation via deep parsing network. In ICCV, 1377--1385.

Digital Library

[33]

Lloyd, S. P. 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 2, 129--137.

Digital Library

[34]

Long, J., Shelhamer, E., and Darrell, T. 2015. Fully convolutional networks for semantic segmentation. In CVPR, 3431--3440.

[35]

Luo, P., Wang, X., and Tang, X. 2012. Hierarchical face parsing via deep learning. In CVPR, 2480--2487.

Digital Library

[36]

Luo, L., Li, H., and Rusinkiewicz, S. 2013. Structure-aware hair capture. ACM Trans. Graph. 32, 4, 76:1--76:12.

Digital Library

[37]

O'Donovan, P., Lībeks, J., Agarwala, A., and Hertzmann, A. 2014. Exploratory font selection using crowdsourced attributes. ACM Trans. Graph. 33, 4, 92:1--92:9.

Digital Library

[38]

Ovsjanikov, M., Li, W., Guibas, L., and Mitra, N. J. 2011. Exploration of continuous variability in collections of 3D shapes. ACM Trans. Graph. 30, 4, 33:1--33:10.

Digital Library

[39]

Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: Geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1--30:9.

Digital Library

[40]

Rabinovich, A., Vedaldi, A., Galleguillos, C., Wiewiora, E., and Belongie, S. 2007. Objects in context. In ICCV, 1--8.

[41]

Ren, S., He, K., Girshick, R., and Sun, J. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 91--99.

Digital Library

[42]

Selle, A., Lentine, M., and Fedkiw, R. 2008. A mass spring model for hair simulation. ACM Trans. Graph. 27, 3, 64:1--64:11.

Digital Library

[43]

Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 1--15.

Digital Library

[44]

Simonyan, K., and Zisserman, A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

[45]

Smith, B. M., Zhang, L., Brandt, J., Lin, Z., and Yang, J. 2013. Exemplar-based face parsing. In CVPR, 3484--3491.

Digital Library

[46]

Uijlings, J. R., van de Sande, K. E., Gevers, T., and Smeulders, A. W. 2013. Selective search for object recognition. IJCV 104, 2, 154--171.

Digital Library

[47]

Wang, D., Chai, X., Zhang, H., Chang, H., Zeng, W., and Shan, S. 2011. A novel coarse-to-fine hair segmentation method. In IEEE Automatic Face Gesture Recognition and Workshops, 233--238.

[48]

Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: Styling, simulation, and rendering. TVCG 13, 2, 213--234.

Digital Library

[49]

Warrell, J., and Prince, S. J. D. 2009. Labelfaces: Parsing facial features by multiclass labeling with an epitome prior. In ICIP, 2481--2484.

Digital Library

[50]

Weng, Y., Wang, L., Li, X., Chai, M., and Zhou, K. 2013. Hair interpolation for portrait morphing. In Computer Graphics Forum, vol. 32, 79--84.

[51]

Yuksel, C., Schaefer, S., and Keyser, J. 2009. Hair meshes. ACM Trans. Graph. 28, 5, 166:1--166:7.

Digital Library

[52]

Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., and Torr, P. H. S. 2015. Conditional random fields as recurrent neural networks. In ICCV, 1529--1537.

Digital Library

Cited By

Zhou YChai MWang DWinberg SWood ESarkar KGross MBeeler T(2024)GroomCap: High-Fidelity Prior-Free Hair CaptureACM Transactions on Graphics10.1145/368776843:6(1-15)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687768
Bhokare GMontalvo EDiaz EYuksel C(2024)Real-Time Hair Rendering with Hair MeshesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657521(1-10)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657521
Kerbiriou GAvril QMarchal M(2024)3D Reconstruction and Semantic Modeling of EyelashesComputer Graphics Forum10.1111/cgf.1504043:2Online publication date: 24-Apr-2024
https://doi.org/10.1111/cgf.15040
Show More Cited By

Index Terms

AutoHair: fully automatic hair modeling from a single image
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Parametric curve and surface models

Recommendations

Single-view hair modeling using a hairstyle database

Human hair presents highly convoluted structures and spans an extraordinarily wide range of hairstyles, which is essential for the digitization of compelling virtual avatars but also one of the most challenging to create. Cutting-edge hair modeling ...
Hair meshes

Despite the visual importance of hair and the attention paid to hair modeling in the graphics research, modeling realistic hair still remains a very challenging task that can be performed by very few artists. In this paper we present hair meshes, a new ...
Hair meshes
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

Despite the visual importance of hair and the attention paid to hair modeling in the graphics research, modeling realistic hair still remains a very challenging task that can be performed by very few artists. In this paper we present hair meshes, a new ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 35, Issue 4

July 2016

1396 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2897824

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 July 2016

Published in TOG Volume 35, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

98
Total Citations
View Citations
1,398
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YChai MWang DWinberg SWood ESarkar KGross MBeeler T(2024)GroomCap: High-Fidelity Prior-Free Hair CaptureACM Transactions on Graphics10.1145/368776843:6(1-15)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687768
Bhokare GMontalvo EDiaz EYuksel C(2024)Real-Time Hair Rendering with Hair MeshesACM SIGGRAPH 2024 Conference Papers10.1145/3641519.3657521(1-10)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641519.3657521
Kerbiriou GAvril QMarchal M(2024)3D Reconstruction and Semantic Modeling of EyelashesComputer Graphics Forum10.1111/cgf.1504043:2Online publication date: 24-Apr-2024
https://doi.org/10.1111/cgf.15040
Wu KYang LKuang ZFeng YHan XShen YFu HZhou KZheng Y(2024)MonoHair: High-Fidelity Hair Modeling from a Monocular Video2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.02281(24164-24173)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.02281
Takimoto YTakehara HSato HZhu ZZheng B(2024)Dr.Hair: Reconstructing Scalp-Connected Hair Strands without Pre-Training via Differentiable Rendering of Line Segments2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01947(20601-20611)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01947
Kim DLee EYoo DLee H(2024)Fine-Grained Human Hair Segmentation Using a Text-to-Image Diffusion ModelIEEE Access10.1109/ACCESS.2024.335554212(13912-13922)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3355542
Wang ZNam GBozic ACao CSaragih JZollhöfer MHodgins J(2024)A Local Appearance Model for Volumetric Capture of Diverse Hairstyles2024 International Conference on 3D Vision (3DV)10.1109/3DV62453.2024.00013(190-200)Online publication date: 18-Mar-2024
https://doi.org/10.1109/3DV62453.2024.00013
Soliman MAhmed EDarwish AHassanien A(2024)Artificial intelligence powered Metaverse: analysis, challenges and future perspectivesArtificial Intelligence Review10.1007/s10462-023-10641-x57:2Online publication date: 5-Feb-2024
https://doi.org/10.1007/s10462-023-10641-x
Zakharov ESklyarova VBlack MNam GThies JHilliges O(2024)Human Hair Reconstruction with Strand-Aligned 3D GaussiansComputer Vision – ECCV 202410.1007/978-3-031-72640-8_23(409-425)Online publication date: 29-Oct-2024
https://doi.org/10.1007/978-3-031-72640-8_23
Li X(2023)Anime-like Character Face Generation: A SurveyHighlights in Science, Engineering and Technology10.54097/hset.v68i.1206968(223-232)Online publication date: 9-Oct-2023
https://doi.org/10.54097/hset.v68i.12069
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents