skip to main content
research-article

Temporal upsampling of performance geometry using photometric alignment

Authors Info & Claims
Published:21 April 2010Publication History
Skip Abstract Section

Abstract

We present a novel technique for acquiring detailed facial geometry of a dynamic performance using extended spherical gradient illumination. Key to our method is a new algorithm for jointly aligning two photographs, under a gradient illumination condition and its complement, to a full-on tracking frame, providing dense temporal correspondences under changing lighting conditions. We employ a two-step algorithm to reconstruct detailed geometry for every captured frame. In the first step, we coalesce information from the gradient illumination frames to the full-on tracking frame, and form a temporally aligned photometric normal map, which is subsequently combined with dense stereo correspondences yielding a detailed geometry. In a second step, we propagate the detailed geometry back to every captured instance guided by the previously computed dense correspondences. We demonstrate reconstructed dynamic facial geometry, captured using moderate to video rates of acquisition, for every captured frame.

Skip Supplemental Material Section

Supplemental Material

wilson

1st supplemental movie file for Temporal upsampling of performance geometry using photometric alignment

References

  1. Ahmed, N., Theobalt, C., Dobrev, P., Seidel, H.-P., and Thrun, S. 2008. Robust fusion of dynamic shape and normal capture for high-quality reconstruction of time-varying geometry. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08). 1--8.Google ScholarGoogle Scholar
  2. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-Scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3, 33: 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proceedings of the European Conference on Computer Vision. 25--36.Google ScholarGoogle Scholar
  4. Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. IEEE Trans. Patt. Anal. Mach. Intell. 27, 2, 296--302. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Hernandez, C., Vogiatzis, G., Brostow, G. J., Stenger, B., and Cipolla, R. 2007. Non-Rigid photometric stereo with colored lights. In Proceedings of the IEEE International Conference on Computer Vision. 1--8.Google ScholarGoogle Scholar
  6. Kang, S., Uyttendaele, M., Winder, S., and Szeliski, R. 2003. High dynamic range video. ACM Trans. Graph. 22, 3, 319--325. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Lim, J., Ho, J., Yang, M.-H., and Kriegman, D. 2005. Passive photometric stereo from motion. In Proceedings of the IEEE International Conference on Computer Vision. 1635--1642. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Ma, W.-C., Hawkins, T., Peers, P., Chabert, C.-F., Weiss, M., and Debevec, P. 2007. Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Proceedings of the Eurographics Symposium on Rendering. 183--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Ma, W.-C., Jones, A., Chiang, J.-Y., Hawkins, T., Frederiksen, S., Peers, P., Vukovic, M., Ouhyoung, M., and Debevec, P. 2008. Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graph. 27, 5, 121: 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Malzbender, T., Wilburn, B., Gelb, D., and Ambrisco, B. 2006. Surface enhancement using real-time photometric stereo and reflectance transformation. In Proceedings of the Eurographics Symposium on Rendering. 245--250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Nehab, D., Rusinkiewicz, S., Davis, J., and Ramamoorthi, R. 2005. Efficiently combining positions and normals for precise 3D geometry. ACM Trans. Graph. 24, 3, 536--543. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3d model acquisition. ACM Trans. Graph. 21, 3, 438--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Scharstein, D. and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 1--3, 7--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Vedula, S., Baker, S., and Kanade, T. 2005. Image based spatio-temporal modeling and view interpolation of dynamic events. ACM Trans. Graph. 24, 2, 240--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., and Matusik, W. 2009. Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph. 28, 5, 174: 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data. ACM Trans. Graph. 28, 2, 15: 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., and Debevec, P. 2005. Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Trans. Graph. 24, 3, 756--764. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. XYZRGB. 3D laser scanning—XYZ RGB Inc. http://www.xyzrgb.com/.Google ScholarGoogle Scholar
  19. Zhang, S., and Huang, P. 2006. High-Resolution, real-time three-dimensional shape measurement. Optical Engin. 45, 12, 123601: 1--8.Google ScholarGoogle Scholar
  20. Zhang, L., Curless, B., Hertzmann, A., and Seitz, S. M. 2003. Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multi-view stereo. In Proceedings of the IEEE International Conference on Computer Vision. 618--625. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. 23, 3, 548--558. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Temporal upsampling of performance geometry using photometric alignment

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Graphics
        ACM Transactions on Graphics  Volume 29, Issue 2
        March 2010
        145 pages
        ISSN:0730-0301
        EISSN:1557-7368
        DOI:10.1145/1731047
        Issue’s Table of Contents

        Copyright © 2010 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 21 April 2010
        • Accepted: 1 February 2010
        • Received: 1 November 2009
        Published in tog Volume 29, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader