Abstract
We present a system which consists of a lifelike agent animated in real-time using video and audio analysis from the user. This kind of system could be used for Instant Messaging where an avatar controlled like a puppet is displayed instead of the webcam flow. The overall system is made of video analysis based on Active Appearance Models and audio analysis based on Hidden Markov Model. The parameters from these two modules are sent to a control system driving the animation engine. The video analysis extracts the head orientation and the audio analysis provides the phonetic string used to move the lips.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active Appearance Models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, Springer, Heidelberg (1998)
Le Gallou, S., Breton, G., Garcia, C., Sguier, R.: Distance Maps: a robust illumination preprocessing for active appearance models. In: VISAPP 2006. International Conference on Computer Vision Theory and Applications (2006)
Breton, G., Bouville, C., Pel, D.: FaceEngine: A 3D Facial Animation Engine for Real Time Applications. In: Web3D Symposium, Paderborn. Germany (2000)
Breton, G., Pel, D., Garcia, C.: Modeling gaze behavior for a 3D ECA in a dialogue situation. In: Proceedings of the 11th international conference on intelligent user interfaces, Sydney, Australia (2006)
Cohen, M.M., Massaro, D.W.: Modeling Coarticulation in Synthetic Visual Speech. Models and Techniques in Computer Animation. Springer, Heidelberg (1993)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Le Gallou, S., Breton, G., Séguier, R., Garcia, C. (2007). Avatar Puppetry Using Real-Time Audio and Video Analysis. In: Pelachaud, C., Martin, JC., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds) Intelligent Virtual Agents. IVA 2007. Lecture Notes in Computer Science(), vol 4722. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74997-4_54
Download citation
DOI: https://doi.org/10.1007/978-3-540-74997-4_54
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74996-7
Online ISBN: 978-3-540-74997-4
eBook Packages: Computer ScienceComputer Science (R0)