Abstract
This paper describes an approach to use artificial reality techniques for real-time interpersonal visual communication at very low bitrate. A flexible structure is suitably adapted to the specific characteristics of the speaker’s head by means of few parameters estimated from the analysis of the real image sequence, while head motion and facial mimics are synthesized on the model by means of knowledge-based deformation rules acting on a simplified muscle structure. The analysis algorithms performed at the transmitter to estimate the model parameters are based on feature-oriented operators aimed at segmenting the real incoming frames and at the extraction of the primary facial descriptors. The system performances have been evaluated on different “head-and-shoulder” sequences and the precision, robustness and complexity of the employed analysis/synthesis algorithms have been tested. Promising results have been achieved for applications both in videophone coding and in picture animation where the facial mimics of a synthetic actor is reproduced according to the parameters extracted from a real speaking face.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aizawa, K., Harashima, H. and Saito, T. (1989) Model-Based Analysis - Synthesis Image Coding (MBASIC) System for Person’s Face. Image Communication 1:139–152.
Badique‘, E. (1990) Knowledge-based Facial Area Recognition and Improved Coding in a CClTT-Compatible Low-Bitrate Video-Codec. Proc. PCS-90, Cambridge Ma 9–1.
Buck, M. and Diehl, N. (1990) Segmentation and modeling of head and shoulder scenes. Proc. II Int. Conf. on 64Kbit/s Video Coding, Rotterdam 4–3.
Buck, M. (1990) Segmentation of moving head-and-shoulder shape. Proc. PCS-90, Cambridge Ma 1990 9–2.
CCITT Recommendation H.261: Video codec for audiovisual services at p*64 kbits/s (1990) CCITT White book
Choi, C.S., Harashima, H. and Takebe, T. (1991) Analysis and Synthesis of Facial Expressions in Knowledge-Based Coding of Facial Image Sequences. Proc. ICASSP-91, S.Francisco CA 2737–2740.
Forchheimer, R. and Kronander, T. (1989) Image Coding - from Waveforms to Animation. IEEE Trans, on ASSP 37:2008–2023.
Ekman, P. and Friesen, W.V. (1977) Facial Action Coding System. Consulting Psychologists Press, Stanford University, Palo Alto.
Gilge, M., Engelhardt, T. and Melhan, R. (1989) Coding of Arbitrarily Shaped Image Segments Based on a Generalized Orthogonal Transform. Signal Processing: Image Communication 1:103–116.
Hoetter, M. and Thoma, R. (1988) Image Segmentation Based on Object Oriented Mapping Parameter Estimation. Signal Processing 15:315–334.
Kunt, M., Ikonomopoulos, A. and Kocher, M. (1985) Second-Generation Image Coding Techniques. IEEE Proceedings 73:549–574.
Kunt, M., Benard, M. and Leonardi, R. (1987) Recent Results on High- Compression Image Coding. IEEE Trans. on Circuits and Systems 34:1306–1336.
Lavagetto, F., Grattarola, A.A., Curinga, S. and Braccini, C. (1992) Muscle Modeling for Facial Animation in Videophone Coding. Proc. IËEE Int. Workshop on Robot and Human Communication, Tokyo 369–375.
Magnenat-Thalmann, N., Primeau, E. and Thalmann, D. (1988) Abstract Muscle Action Procedures for Face Animation. The Visual Computer 3:290–297.
Morishima, S., Aizawa, K. and Harashima, H. (1988) Model-based facial image coding controlled by the speech parameter. Proc. PCS-88, Turin, I 4–4.
MPEG Video Simulation Model Three SM3 (1990) Doc. ISO-IEC/JTC1/SC2/ WG8 N/MPEG90.
Musmann, H.G., Pirsh, P. and Grallert, H.J. (1985) Advances in Picture Coding. IEEE Procedings 73:523–548.
Musmann, H.G., Hoetter, M. and Ostermann, J. (1989) Object-Oriented Analysis - Synthesis Coding of Moving Images. Signal Processing: Image Communication 1:117–138.
Nakaya, Y., Chuah, Y.C. and Harashima, H. (1991) Model-based/waveform hybrid coding for videotelephone images. Proc. ICASSP-91, S.Francisco, CA 2741–2744.
Parke, F.I. (1982) Parameterized Models for Facial Animation. IEEE Computer Graphics and Applications 2:61–68.
Pereira, F. and Masera, L. (1990) Two-layers knowledge-based videophone coding. Proc. PCS-90, Cambridge Ma 6–1.
Terzopoulos, D. and Waters, K. (1990) Analysis of Facial Images Using Physical and Anatomical Models. Proc. IEEE 3rd Int, Conf on Computer Vision, Osaka 727–732.
Waters, K. (1987) A Muscle Model for Animating Threedimensional Facial Expression. Computer Graphics 22:17–24.
Yuhas, B.P., Goldstein Jr., M.H. and Sejnowski, T.J. (1989) Integration of Acoustic and Visual Speech Signal Using Neural Networks. IEEE Communications Magazine 65–71.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1993 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Curinga, S., Grattarola, A., Lavagetto, F. (1993). Synthesis and animation of human faces: artificial reality in interpersonal video communication. In: Falcidieno, B., Kunii, T.L. (eds) Modeling in Computer Graphics. IFIP Series on Computer Graphics. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-78114-8_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-78114-8_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-78116-2
Online ISBN: 978-3-642-78114-8
eBook Packages: Springer Book Archive