Tatsuo Yotsukura
ABSTRACT
Towards natural human-machine communication, interface technologies by way of speech and image information have been intensively developed. An anthropomorphic dialog agent is an ideal system, which integrates spoken dialog and natural facial expressions. This paper reports on our project aiming to create a general-purpose toolkit for building an easily customizable anthropomorphic agent. There have been almost no tools so far such as intuitive, easy to understand, fully interactive, and open source. Our anthropomorphic agent is designed to fulfill these requirements. This toolkit consists four modules, multi modal dialog integration, speech recognition, speech synthesis, and face image synthesis. These modules are highly modularized and interlinked by a simple communication protocols.In this paper, we focus on the construction of an agent's face image synthesis. For this part lip movement control synchronous to the speech signal and facial emotion expression are the most important parts. We developed the face image synthesis module (FSM) that only requires one frontal face image, and can be used by any skill level of users. A user's original agent can be generated by easy adjustment of the frontal face image and the generic wire-frame model. The paper describes overall system diagram and specifically the agent's face image synthesis part.
- DARPA: Communicator Program (1998). http://fofoca.mitre.org/.Google Scholar
- Seneff, S., Hurley, E., Lau, R., Pao, C., Schmid, P. and Zue, V.: GALAXY-II: A Referece Architecture for Conversational System Development, ICSLP-1998, pp. 931--934 (1998).Google Scholar
- OAA: (The Open Agent Architecture). http://www.ai.sri.com/Eoaa/.Google Scholar
- VoiceXML: (Voice eXtensible Markup Language Ver1.0) (2000). http://www.voicexml.org.Google Scholar
- Yoshimura, T., Tokuda, K., Masuko, T.,Kobayashi, T. and Kitamura, T.: Speaker Interpolation for HMM-based Speech Synthesis System, J Acoust. Soc. Jpn. (E), Vol. 21, No. 4, pp. 199--206 (2000).Google ScholarCross Ref
- Itou, K., Hayamizu, S., Tanaka, K., Tanaka, H.: Sysstem design data collection and evaluation of a speech dialogue system, IEICE Trans. Inf. And Syst., Vol.36, No.1, pp.121--127 (1993)Google Scholar
- Morishima, S.: Face-to-face Communication in Cyberspace using Analysis and Synthesis of Facial Expression, Proceedings of '99 International Workshop on Advanced Image Technology(IWAIT99), pp.111--118 (1999) Google ScholarDigital Library
- Ekman, P., Friesen, W. V.: Manual for the Facial Action Coding System and Action Unit Photographs. Palo Alto, CA: Consulting Psychological Press. (1978)Google Scholar
Index Terms
- Model-based talking face synthesis for anthropomorphic spoken dialog agent system
Recommendations
Spontaneous spoken dialogues with the furhat human-like robot head
HRI '14: Proceedings of the 2014 ACM/IEEE international conference on Human-robot interactionFurhat [1] is a robot head that deploys a back-projected animated face that is realistic and human-like in anatomy. Furhat relies on a state-of-the-art facial animation architecture allowing accurate synchronized lip movements with speech, and the ...
Animating expressive faces across languages
This paper describes a morphing-based audio driven facial animation system. Based on an incoming audio stream, a face image is animated with full lip synchronization and synthesized expressions. A novel scheme to implement a language independent system ...
An extensible framework for interactive facial animation with facial expressions, lip synchronization and eye behavior
SPECIAL ISSUE: GamesIn this article we describe our approach to generating convincing and empathetic facial animation. Our goal is to develop a robust facial animation platform that is usable and can be easily extended. We also want to facilitate the integration of ...
Comments