Feng Xu
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Recent technological advances in facial capture have made it possible to acquire high-fidelity 3D facial performance data with stunningly high spatial-temporal resolution. Current methods for facial expression transfer, however, are often limited to large-scale facial deformation. This paper introduces a novel facial expression transfer and editing technique for high-fidelity facial performance data. The key idea of our approach is to decompose high-fidelity facial performances into high-level facial feature lines, large-scale facial deformation and fine-scale motion details and transfer them appropriately to reconstruct the retargeted facial animation in an efficient optimization framework. The system also allows the user to quickly modify and control the retargeted facial sequences in the spatial-temporal domain. We demonstrate the power of our approach by transferring and editing high-fidelity facial animation data from high-resolution source models to a wide range of target models, including both human faces and non-human faces such as "monster" and "dog".
Supplemental Material
Available for Download
zip
a42-xu.zip (174.5 MB)
Supplemental material.
- Akhter, I., Simon, T., Khan, S., Matthews, I., and Sheikh, Y. 2012. Bilinear spatiotemporal basis models. ACM Trans. Graph. 31, 2 (Apr.), 17:1--17:12. Google Scholar
Digital Library
- Baran, I., Vlasic, D., Grinspun, E., and Popović, J. 2009. Semantic deformation transfer. ACM Trans. Graph. 28, 3 (July), 36:1--36:6. Google ScholarDigital Library
- Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R. W., and Gross, M. 2011. High-quality passive facial performance capture using anchor frames. ACM Trans. Graph. 30, 4, 75:1--75:10. Google ScholarDigital Library
- Bermano, A. H., Bradley, D., Beeler, T., Zund, F., Nowrouzezahrai, D., Baran, I., Sorkine-Hornung, O., Pfister, H., Sumner, R. W., Bickel, B., and Gross, M. 2014. Facial performance enhancement using dynamic shape space analysis. ACM Trans. Graph. 33, 2 (Apr.), 13:1--13:12. Google ScholarDigital Library
- Bhat, K. S., Goldenthal, R., Ye, Y., Mallet, R., and Koperwas, M. 2013. High fidelity facial animation capture and retargeting with contours. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '13, 7--14. Google ScholarDigital Library
- Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July). Google ScholarDigital Library
- Bickel, B., Lang, M., Botsch, M., Otaduy, M. A., and Gross, M. 2008. Pose-space animation and transfer of facial details. In Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '08, 57--66. Google ScholarDigital Library
- Bouaziz, S., Wang, Y., and Pauly, M. 2013. Online modeling for realtime facial animation. ACM Trans. Graph. 32, 4 (July), 40:1--40:10. Google ScholarDigital Library
- Buck, I., Finkelstein, A., Jacobs, C., Klein, A., Salesin, D., Seims, J., Szeliski, R., and Toyama, K. 2000. Performance-driven hand-drawn animation. In Proceedings of NPAR 2000. Google ScholarDigital Library
- Cao, C., Weng, Y., Lin, S., and Zhou, K. 2013. 3d shape regression for real-time facial animation. ACM Trans. Graph. 32, 4 (July), 41:1--41:10. Google ScholarDigital Library
- Chai, J., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3D facial animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 193--206. Google ScholarDigital Library
- Chuang, E., 2004. Analysis, synthesis, and retargeting of facial expressions. PhD thesis, Stanford University. Google ScholarDigital Library
- Havaldar, P. 2006. Performance driven facial animation. ACM SIGGRAPH Course Notes.Google Scholar
- Huang, H., Chai, J., Tong, X., and Wu, H.-T. 2011. Leveraging motion capture and 3d scanning for high-fidelity facial performance acquisition. ACM Trans. Graph. 30, 4, 74:1--74:10. Google ScholarDigital Library
- Image-Metrics. Live driver (face tracking). http://www.image-metrics.com.Google Scholar
- James, D. L., and Twigg, C. D. 2005. Skinning mesh animations. ACM Transactions on Graphics 24, 3, 399--407. Google ScholarDigital Library
- Joshi, P., Tien, W. C., Desbrun, M., and Pighin, F. 2003. Learning controls for blendshape based realistic facial animation. In Proceedings of ACM SIGGRAPH/Eurographics Symposium on Computer Aimation. 187--192. Google ScholarDigital Library
- Kholgade, N., Matthews, I., and Sheikh, Y. 2011. Content retargeting using parameter-parallel facial layers. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '11, 195--204. Google ScholarDigital Library
- Lau, M., Chai, J., Xu, Y.-Q., and Shum, H.-Y. 2007. Face poser: interactive facial modeling using model priors. In Proceedings of the 2007 ACM SIGGRAPH/Eurographics symposium on Computer animation. Google ScholarDigital Library
- Lau, M., Chai, J., Xu, Y.-Q., and Shum, H. 2009. Face poser: Interactive modeling of 3d facial expressions using facial priors. ACM Transactions on Graphics 29, 1, 3:1--3:17. Google ScholarDigital Library
- Lewis, J., and Anjyo, K. 2010. Direct manipulation blendshapes. IEEE Computer Graphics and Applications 30, 4, 42--50. Google ScholarDigital Library
- Li, Q., and Deng, Z. 2008. Orthogonal-blendshape-based editing system for facial motion capture data. IEEE Comput. Graph. Appl. 28, 6 (Nov.), 76--82. Google ScholarDigital Library
- Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Transactions on Graphics 29, 4, 32:1--32:6. Google ScholarDigital Library
- Li, H., Yu, J., Ye, Y., and Bregler, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Trans. Graph. 32, 4 (July), 42:1--42:10. Google ScholarDigital Library
- Meyer, M., and Anderson, J. 2007. Key point subspace acceleration and soft caching. In ACM Transactions on Graphics. 26(3): Article No. 74. Google ScholarDigital Library
- Noh, J., and Neumann, U. 2001. Expression cloning. In Proceedings of ACM SIGGRAPH 2001. 277--288. Google ScholarDigital Library
- Pighin, F., and Lewis, J. 2005. Digital face cloning. ACM SIGGRAPH Course Notes. Google ScholarDigital Library
- Seol, Y., Seo, J., Kim, P. H., Lewis, J. P., and Noh, J. 2011. Artist friendly facial animation retargeting. In Proceedings of the 2011 SIGGRAPH Asia Conference, ACM, New York, NY, USA, SA '11, 162:1--162:10. Google ScholarDigital Library
- Seol, Y., Lewis, J., Seo, J., Choi, B., Anjyo, K., and Noh, J. 2012. Spacetime expression cloning for blendshapes. ACM Trans. Graph. 31, 2 (Apr.), 14:1--14:12. Google ScholarDigital Library
- Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H. 2004. Laplacian surface editing. In Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, ACM, 175--184. Google ScholarDigital Library
- Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3 (Aug.), 399--405. Google ScholarDigital Library
- Taubin, G. 1995. A signal processing approach to fair surface design. In Proceedings of the 22Nd Annual Conference on Computer Graphics and Interactive Techniques, ACM, New York, NY, USA, SIGGRAPH '95, 351--358. Google ScholarDigital Library
- Tena, J. R., De la Torre, F., and Matthews, I. 2011. Interactive region-based linear 3d face models. ACM Trans. Graph. 30, 4 (July), 76:1--76:10. Google ScholarDigital Library
- Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Transactions on Graphics 24, 3, 426--433. Google ScholarDigital Library
- Weise, T., Li, H., Van Gool, L., and Pauly, M. 2009. Face/off: live facial puppetry. In Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, ACM, New York, NY, USA, SCA '09, 7--16. Google ScholarDigital Library
- Zhang, L., Snavely, N., Curless, B., and Seitz, S. 2004. Spacetime faces: high resolution capture for modeling and animation. ACM Transactions on Graphics 23, 3, 548--558. Google ScholarDigital Library
Index Terms
- Controllable high-fidelity facial performance transfer
Recommendations
Facial retargeting with automatic range of motion alignment
While facial capturing focuses on accurate reconstruction of an actor's performance, facial animation retargeting has the goal to transfer the animation to another character, such that the semantic meaning of the animation remains. Because of the ...
Semantic 3D motion retargeting for facial animation
APGV '06: Proceedings of the 3rd symposium on Applied perception in graphics and visualizationWe present a system for realistic facial animation that decomposes facial motion capture data into semantically meaningful motion channels based on the Facial Action Coding System. A captured performance is retargeted onto a morphable 3D face model ...
Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition
This paper introduces a new approach for acquiring high-fidelity 3D facial performances with realistic dynamic wrinkles and fine-scale facial details. Our approach leverages state-of-the-art motion capture technology and advanced 3D scanning technology ...
Comments