Abstract
Natural images are the composite consequence of multiple factors related to scene structure, illumination, and imaging. Multilinear algebra, the algebra of higher-order tensors, offers a potent mathematical framework for analyzing the multifactor structure of image ensembles and for addressing the difficult problem of disentangling the constituent factors or modes. Our multilinear modeling technique employs a tensor extension of the conventional matrix singular value decomposition (SVD), known as the N-mode SVD. As a concrete example, we consider the multilinear analysis of ensembles of facial images that combine several modes, including different facial geometries (people), expressions, head poses, and lighting conditions. Our resulting “TensorFaces” representation has several advantages over conventional eigenfaces. More generally, multilinear analysis shows promise as a unifying framework for a variety of computer vision problems.
Chapter PDF
Similar content being viewed by others
Keywords
- Singular Value Decomposition
- Independent Component Analysis
- Facial Image
- Independent Component Analysis
- Multilinear Algebra
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
P.N. Belhumeur, J. Hespanha, and D.J. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. In Proceedings of the European Conference on Computer Vision, pages 45–58, 1996.
R. Chellappa, C.L. Wilson, and S. Sirohey. Human and machine recognition of faces: A survey. Proceedings of the IEEE, 83(5):705–740, May 1995.
L. de Lathauwer. Signal Processing Based on Multilinear Algebra. PhD thesis, Katholieke Univ. Leuven, Belgium, 1997.
W. Freeman and J. Tenenbaum. Learing bilinear models for two-factor problems in vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 554–560, 1997.
A. Kapteyn, H. Neudecker, and T. Wansbeek. An approach to n-mode component analysis. Psychometrika, 51(2):269–275, June 1986.
T. G. Kolda. Orthogonal tensor decompositions. SIAM Journal on Matrix Analysis and Applications, 23(1):243–255, 2001.
J. B. Kruskal. Rank, decomposition, and uniqueness for 3-way and n-way array. In R. Coppi and S. Bolasco, editors, Multiway Data Analysis, pages 7–18, Amsterdam, 1989. North Holland.
J. R. Magnus and H. Neudecker. Matrix Differential Calculus with Applications in Statistics and Econometrics. John Wiley & Sons, New York, New York, 1988.
D.H. Marimont and B.A. Wandell. Linear models of surface and illuminance spectra. J. Optical Society of America, A., 9:1905–1913, 1992.
Y. Moses, S. Edelman, and S. Ullman. Generalization to novel images in upright and inverted faces. Perception, 25:443–461, 1996.
A. Pentland and B. Moghaddam. View-based and modular eigenspaces for face recognition. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1994.
A. Shashua and A. Levin. Linear image coding for regression and classification using the tensor-rank principle. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, page in press, Hawai, 2001.
L. Sirovich and M. Kirby. Low dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A., 4:519–524, 1987.
J.B. Tenenbaum and W.T. Freeman. Separating style and content. In M. Moser, M. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, pages 662–668. MIT Press, 1997.
M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analysers. Neural Computation, 11(2):443–482, 1999.
L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31:279–311, 1966.
M.A. Turk and A. P. Pentland. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):71–86, 1991.
M.A.O. Vasilescu. An algorithm for extracting human motion signatures. In IEEE Conference on Computer Vision and Pattern Recognition, Hawai, 2001. in press.
M. A. O. Vasilescu. Human motion signatures for character animation. In ACM SIGGRAPH 2001 Conf. Abstracts and Applications, page 200, Los Angeles, August 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Vasilescu, M.A.O., Terzopoulos, D. (2002). Multilinear Analysis of Image Ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2350. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47969-4_30
Download citation
DOI: https://doi.org/10.1007/3-540-47969-4_30
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43745-1
Online ISBN: 978-3-540-47969-7
eBook Packages: Springer Book Archive