Abstract
Images formed by a human face change with viewpoint. A new technique is described for synthesizing images of faces from new viewpoints, when only a single 2D image is available. A novel 2D image of a face can be computed without explicitly computing the 3D structure of the head. The technique draws on a single generic 3D model of a human head and on prior knowledge of faces based on example images of other faces seen in different poses. The example images are used to “learn” a pose-invariant shape and texture description of a new face. The 3D model is used to solve the correspondence problem between images showing faces in different poses.
The proposed method is interesting for view independent face recognition tasks as well as for image synthesis problems in areas like teleconferencing and virtualized reality.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aizawa, K. Harashima, H. and Saito, T. 1989. Model-based analysis synthesis image coding (MBASIC) system for a person'sface. Signal Processing: Image Communication, 1:139–152.
Akimoto, T., Suenaga, Y. and Wallace, R.S. 1993. Automatic creation of 3D facial models. IEEE Computer Graphics and Applications, 13(3):16–22.
Bergen, J.R. Anandan, P., Hanna, K.J. and Hingorani, R. 1992. Hierarchical model-based motion estimation. In Proceedings of the European Conference on Computer Vision, Santa Margherita Ligure, Italy, pp. 237–252.
Bergen, J.R. and Hingorani, R. 1990. Hierarchical motion-based frame rate conversion. Technical report, David Sarnoff Research Center, Princeton, NJ.
Beymer, D. 1993. Face recognition under varying pose. A.I. Memo No. 1461, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Beymer, D. and Poggio, T. 1995. Face recognition from one model view. In Proceedings of the 5th International Conference on Computer Vision.
Beymer, D. and Poggio, T. 1996. Image representation for visual learning. Science, 272:1905–1909.
Beymer, D., Shashua, A. and Poggio, T. 1993. Example-based image analysis and synthesis. A.I. Memo No. 1431, Artificial Intelligence Laboratory, Massachusetts Institute of Technology.
Burt, P.J. and Adelson, E.H. 1983. The Laplacian pyramide as a compact image code. IEEE Transactions on Communications, 31:532–540.
Burt, P.J. and Adelson, E.H. 1985. Merging images through pattern decomposition. Applications of Digital Image Processing VIII, 575:73–181. SPIE The International Society for Optical Engeneering.
Choi, C.S., Okazaki, T., Harashima, H. and Takebe, T. 1991. A system of analyzing and synthesizing facial images. In Proc. IEEE Int. Symposium of Circuit and Syatems (ISCAS91), pp. 2665– 2668.
Cootes, T.F., Taylor, C.J., Cooper, D.H. and Graham, J. 1995. Active shape models-their training and application. Computer Vision and Image Understanding, 61:38–59.
Craw, I. and Cameron, P. 1991. Parameterizing images for recognition and reconstruction. In Proc. British Machine Vision Conference, Springer, pp. 367–370.
Hallinan, P.W. 1995. A deformable model for the recognition of human faces under arbitrary illumination. Doctoral thesis, Harvard University, Cambridge, MA.
Horn, B.K.P. 1987. Robot Vision. MIT Press: Cambridge, MA.
Huang, T.S., and Lee, C.H. 1989. Motion and structure from orthographic projections. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2(5):536–540.
Jones, M., and Poggio, T. 1995. Model-based matching of line drawings by linear combination of prototypes. In Proceedings of the 5th International Conference on Computer Vision.
Lanitis, A., Taylor, C.J., Cootes, T.F., and Ahmad, T. 1995. Automatic interpretation of human faces and hand gestures using flexible models. In Proc. InternationalWorkshop on Face and Gesture Recognition, Zurich, Switzerland, pp. 98–103.
O'Toole, A.J., Deffenbacher, K.A., Valentin, D. and Abdi, H. 1994. Structural aspects of face recognition and the other-race effect. Memory and Cognition, 22:208–224.
Poggio, T. and Brunelli, R. 1992. A novel approach to graphics. Technical report 1354, MIT Media Laboratory Perceptual Computing Section.
Press, Teukolsky, Vetterling and Flannery. 1992. Numerical recipes in C: the art of scientific computing. Cambridge University Press: Cambridge.
C.A. Rothwell, D.A. Forsyth, Zissermann, A. and Mundy, J.L. 1993. Extracting projective structure from single perspective views of 3D point sets. In Proceedings of the International Conference on Computer Vision (ICCV), Berlin, Germany, pp. 573–582.
Terzopoulos, D. and Waters, K. 1993 Analysis and synthesis of facial image sequences using physical and anatomical models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(6):569–579.
Thalmann, N.D. and Thalmann, D. 1995. Digital actors for interactive television. In Proceedings of the IEEE, 83(7):1022–1031.
Vetter, T., Jones, M. and Poggio, T. 1997. A bootstrapping algorithm for learning linearized models of object classes. in IEEE Conference on Computer Vision and Pattern Recognition.
Vetter, T. and Poggio, T. 1994. Symmetric 3D objects are an easy case for 2D object recognition. Spatial Vision, 8(4):443–453.
Vetter, T. and Poggio, T. 1996. Image synthesis from a single example image. In volume 1065 of LNCS,Computer Vision – ECCV'96, Cambridge UK. Springer.
Wolberg, G. 1990. Image Warping. IEEE Computer Society Press: Los Alamitos, CA.
Xu, W. and Hauske, G. 1994. Picture quality evaluation based on error segmentation. In Proc. SPIE, Visual Communications and Image Processing, 2308:1–12.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Vetter, T. Synthesis of Novel Views from a Single Face Image. International Journal of Computer Vision 28, 103–116 (1998). https://doi.org/10.1023/A:1008058932445
Issue Date:
DOI: https://doi.org/10.1023/A:1008058932445