Abstract
Our research group has been studying a speech communication system with a virtual agent (VA), to support person-centered care (PCC) of people with dementia (PWD). The current system uses an unfamiliar avatar for the VA, which causes a limitation in the care effects. In this paper, we develop a novel system that dynamically creates a VA based on a given facial image. The proposed system constructs a three-dimensional model based on facial landmarks within the image. It then stretches and transforms some portions of the 3D model to generate facial expressions. From just a given picture, the proposed system easily generates a communication agent familiar with individual PWD. Hence, it can implement (virtual, but effective) conversations with familiar partners.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cabinet Office, Government of Japan: Annual report on the aging society (2017). http://www8.cao.go.jp/kourei/whitepaper/w-2017/html/zenbun/s1_1_1.html. Accessed June 2017
Tokunaga, S., Tamamizu, K., Saiki, S., Nakamura, M., Yasuda, K.: VirtualCareGiver: personalized smart elderly care. Int. J. Softw. Innov. (IJSI) 5(1), 30–43 (2016)
Tamamizu, K., Sakakibara, S., Saiki, S., Nakamura, M., Yasuda, K.: Capturing activities of daily living for elderly at home based on environment change and speech dialog. In: International Conference on Digital Human Modeling 2017 (DHM 2017). LNCS, vol. 10287, pp. 183–194, July 2017
Sakakibara, S., Saiki, S., Nakamura, M., Yasuda, K.: Generating personalized dialogue towards daily counseling system for home dementia care. In: International Conference on Digital Human Modeling 2017 (DHM 2017). LNCS, vol. 10287, pp. 161–172, July 2017
MMDAgent. http://www.mmdagent.jp
Emotion API. https://azure.microsoft.com/en-us/services/cognitive-services/emotion/
Taigman, Y., Yang, M., Ranzato, M.A., Wolf, L.: DeepFace: closing the gap to human-level performance in face verification. In: CVPR 2014 (2014)
Fan, H., Cao, Z., Jiang, Y., Yin, Q., Doudou, C.: Learning deep face representation. arXiv:1403.2802
Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: European Conference on Computer Vision (2014)
Face++. https://www.faceplusplus.com
MotionPortrait. https://www.motionportrait.com
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Nakatani, S., Saiki, S., Nakamura, M. (2018). Integrating 3D Facial Model with Person-Centered Care Support System for People with Dementia. In: Karwowski, W., Ahram, T. (eds) Intelligent Human Systems Integration. IHSI 2018. Advances in Intelligent Systems and Computing, vol 722. Springer, Cham. https://doi.org/10.1007/978-3-319-73888-8_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-73888-8_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73887-1
Online ISBN: 978-3-319-73888-8
eBook Packages: EngineeringEngineering (R0)