Abstract
Limited animation is one of the traditional techniques for producing cartoon animations. Owing to its expressive style, it has been enjoyed around the world. However, producing high quality animations using this limited style is time-consuming and costly for animators. Furthermore, proper synchronization between the voice-actor’s voice and the character’s mouth and lip motion requires well-experienced animators. This is essential because viewers are very sensitive to audio-lip discrepancies. In this paper, we propose a method that automatically creates high-quality limited-style lip-synched animations using audio tracks. Our system can be applied for creating not only the original animations but also dubbed ones independently of languages. Because our approach follows the standard workflow employed in cartoon animation production, our system can successfully assist animators. In addition, users can implement our system as a plug-in of a standard tool for creating animations (Adobe After Effects) and can easily arrange character lip motion to suit their own style. We visually evaluate our results both absolutely and relatively by comparing them with those of previous works. From the user evaluations, we confirm that our algorithms is able to successfully generate more natural audio-mouth synchronizations in limited-style lip-synched animations than previous algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
coin: conditional inference procedures in a permutation test framework. https://cran.r-project.org/web/packages/coin/index.html. Accessed 22 Oct 2017
Bregler, C., Covell, M., Slaney, M.: Video rewrite: driving visual speech with audio. In: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, pp. 353–360. ACM Press/Addison-Wesley Publishing Co. (1997)
Cao, C., Hou, Q., Zhou, K.: Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. (TOG) 33(4), 43 (2014)
Chang, Y.J., Ezzat, T.: Transferable videorealistic speech animation. In: Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 143–151. ACM (2005)
Dunbar, D., Humphreys, G.: A spatial data structure for fast poisson-disk sample generation. ACM Trans. Graph. (TOG) 25(3), 503–508 (2006)
Dvorožňák, M., Bénard, P., Barla, P., Wang, O., Sỳkora, D.: Example-based expressive animation of 2D rigid bodies. ACM Trans. Graph 36(4), 10 (2017)
Ezzat, T., Geiger, G., Poggio, T.: Trainable videorealistic speech animation. ACM Trans. Graph. (TOG) 21(3), 388–398 (2002)
Ishi, C.T., Liu, C., Ishiguro, H., Hagita, N.: Speech-driven lip motion generation for tele-operated humanoid robots. In: Auditory-Visual Speech Processing 2011 (2011)
Kawamoto, S.I., Yotsukura, T., Anjyo, K., Nakamura, S.: Efficient lip-synch tool for 3D cartoon animation. Comput. Anim. Virtual Worlds 19(34), 247–257 (2008)
Kazi, R.H., Grossman, T., Umetani, N., Fitzmaurice, G.: Motion amplifiers: sketching dynamic illustrations using the principles of 2D animation. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (2016)
Kitamura, M., Kanamori, Y., Mitani, J., Fukui, Y., Tsuruno, R.: Motion frame omission for cartoon-like effects. In: Proceedings of International Workshop on Advanced Image Technology (IWAIT), pp. 148–152. KSBE (2014)
Morishima, S., Kuriyama, S., Kawamoto, S., Suzuki, T., Taira, M., Yotsukura, T., Nakamura, S.: Data-driven efficient production of cartoon character animation. In: ACM SIGGRAPH 2007 Sketches, p. 76. ACM (2007)
Rothauser, E.: IEEE recommended practice for speech quality measurements. IEEE Trans. Audio Electroacoust. 17, 225–246 (1969)
Wang, J., Drucker, S.M., Agrawala, M., Cohen, M.F.: The cartoon animation filter. ACM Trans. Graph. (TOG) 25, 1169–1173 (2006)
Weise, T., Bouaziz, S., Li, H., Pauly, M.: Realtime performance-based facial animation (TOG). ACM Trans. Graph. 30, 77 (2011)
Weise, T., Li, H., Van Gool, L., Pauly, M.: Face/off: live facial puppetry. In: Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer Animation, pp. 7–16. ACM (2009)
Acknowledements
This work was supported in part by the Japanese Information-Technology Promotion Agency (IPA), JST ACCEL Grant No. JPMJAC 1602, and JSPS Grant No. 17H06101, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Furukawa, S., Fukusato, T., Yamaguchi, S., Morishima, S. (2018). Voice Animator: Automatic Lip-Synching in Limited Animation by Audio. In: Cheok, A., Inami, M., Romão, T. (eds) Advances in Computer Entertainment Technology. ACE 2017. Lecture Notes in Computer Science(), vol 10714. Springer, Cham. https://doi.org/10.1007/978-3-319-76270-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-76270-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-76269-2
Online ISBN: 978-3-319-76270-8
eBook Packages: Computer ScienceComputer Science (R0)