Abstract
Digital plays a key role in the transformation of medicine. Beyond the simple computerisation of healthcare systems, many non-drug treatments are now possible thanks to digital technology. Thus, interactive stimulation exercises can be offered to people suffering from cognitive disorders, such as developmental disorders, neurodegenerative diseases, stroke or traumas. The efficiency of these new treatments, which are still primarily offered face-to-face by therapists, can be greatly improved if patients can pursue them at home. However, patients are left to their own devices which can be problematic. We introduce THERADIA, a 5-year project that aims to develop an empathic virtual agent that accompanies patients while receiving digital therapies at home, and that provides feedback to therapists and caregivers. We detail the architecture of our agent as well as the framework of our Wizard-of-Oz protocol, designed to collect a large corpus of interactions between people and our virtual assistant in order to train our models and improve our dialogues.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Joubert, C., Chainay, H.: Aging brain: the effect of combined cognitive and physical training on cognition as compared to cognitive and physical training alone - a systematic review. Clin. Interven. Aging 13, 1267–1301 (2018)
Klimová, B., Vališ, M.: Smartphone applications can serve as effective cognitive training tools in healthy aging. Front. Aging Neurosci. 9, 436 (2018)
van der Linden, S., Sitskoorn, M.M., Rutten, G.-J.M., Gehring, K.: Feasibility of the evidence-based cognitive telerehabilitation program Remind for patients with primary brain tumors. J. Neuro-Oncol. 137, 523–532 (2018)
Wilms, I.L.: The computerized cognitive training alliance – a proposal for alliance model for home-based computerized cognitive training. CellPress, Heliyon 6, e03254 (2020)
Turunen, M., et al.: Computer-based cognitive training for older adults: determinants of adherence. PlosOne 14(7), e0219541 (2019)
Kethuneni, S., August, S.E., Ian Vales, J.: Personal health care assistant/companion in virtual world. In: Association for the Advancement of Artificial Intelligence (AAAI), Fall Symposium Series (2009)
Vaidyam, A.N., Wisniewski, H., Halamka, J.D., Kashavan, M.S., Torous, J.B.: Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can. J. Psychiatr. 64(7), 456–464 (2019)
Cassell, J., Sullivan, J., Prevost, O., Churchill, E.: Embodied Conversational Agents. MIT Press, Cambridge (2000)
Cummins, N., Baird, A., Schuller, B.W.: Speech analysis for health: Current state-of-the-art and the increasing impact of deep learning. Methods 41–54 (2018)
Ringeval, F., et al.: AVEC 2019 workshop and challenge: state-of-mind, detecting depression with AI, and cross-cultural affect recognition. In: International Workshop on Audio/Visual Emotion Challenge, AVEC 2019, Nice, France (2019)
Swerts, M., Krahmer, E.: Audiovisual prosody and feeling of knowing. J. Memory Lang. 81–94 (2005)
Barbulescu, A., Ronfard, R., Bailly, G.: A generative audio-visual prosodic model for virtual actors. IEEE Comput. Graphics Appl. 37(6), 40–51 (2017)
Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)
Khare, A., Parthasarathy, S., Sundaram, S.: Self-Supervised learning with cross-modal transformers for emotion recognition. arXiv preprint arXiv:2011.10652 (2020)
Siriwardhana, S., Reis, A., Weerasekera, R., Nanayakkara, S.: Jointly Fine-Tuning “BERT-like” Self Supervised Models to Improve Multimodal Speech Emotion Recognition. arXiv preprint arXiv:2008.06682 (2020)
Thórisson, K.R.: Natural turn-taking needs no manual: computational theory and model, from perception to action. In: Multimodality in Language and Speech Systems, pp. 173–207. Springer, Dordrecht (2002)
Skantze, G.: Turn-taking in conversational systems and human-robot interaction: a review. Comput. Speech Lang. 67, 101–178 (2021)
Ekman, P.: Facial expressions of emotion: New findings, new questions (1992)
Russell, J.A.: Reading emotions from and into faces: resurrecting a dimensional-contextual perspective, In: Russell, J.A., Fernández-Dols, J.M. (eds.) Studies in Emotion and Social Interaction. The Psychology of Facial Expression, pp. 295–320. CUP (1997)
Scherer, K.R.: The dynamic architecture of emotion: Evidence for the component process model. Cogn. Emot. 23(7), 1307–1351 (2009)
Scherer, K.R., Dieckmann, A., Unfried, M., Ellgring, H., Mortillaro, M.: Investigating appraisal-driven facial expression and inference in emotion communication. Emotion 21(1), 73 (2019)
Shen, J., et al.: Natural TTS synthesis by conditioning wavenet on mel spectrogram predictions. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4779–4783 (2018)
Tits, N., Wang, F., El Haddad, K., Pagel, V., Dutoit, T.: Visualization and interpretation of latent spaces for controlling expressive speech synthesis through audio analysis. Interspeech, pp. 4475–4479 (2019)
Stanton, D., Wang, Y., Skerry-Ryan, R.J.: Predicting expressive speaking style from text in end-to-end speech synthesis. In: IEEE Spoken Language Technology Workshop (SLT), pp. 595–602 (2018)
Acknowledgments
This research has received funding from the Banque Publique d’Investissement (BPI) under grant agreement THERADIA, the Association Nationale de la Recherche et de la Technologie (ANRT), under grant agreement No. 2019/0729, and has been partially supported by MIAI@Grenoble-Alpes, (ANR-19-P3IA-0003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tarpin-Bernard, F. et al. (2021). THERADIA: Digital Therapies Augmented by Artificial Intelligence. In: Ayaz, H., Asgher, U., Paletta, L. (eds) Advances in Neuroergonomics and Cognitive Engineering. AHFE 2021. Lecture Notes in Networks and Systems, vol 259. Springer, Cham. https://doi.org/10.1007/978-3-030-80285-1_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-80285-1_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-80284-4
Online ISBN: 978-3-030-80285-1
eBook Packages: EngineeringEngineering (R0)