Abstract
This paper addresses the problem of accomplishing Orofacial Rehabilitation (OR) with the assistance of artificial intelligence. The main challenges involve accurately monitoring and interacting with the trainees, while preserving user experience. We analyse different approaches to solving these challenges and propose a methodology to build smart knowledge-driven OR systems that focus on automated interaction. Our proposal leverages the combination of vision-based micro and macro facial expression recognition and skill-based dialogue systems, which facilitate encapsulating the knowledge of rehabilitation professionals into natural language interactions. Experimental results of spoken keyword spotting and micro and macro facial expression recognition algorithms are provided. The OR expressions image dataset employed in our experiments is also published to support further research in the field.
Supported by SHAPES – Smart and Health Ageing through People Engaging in Supportive Systems - is funded by the Horizon 2020 Framework Programme of the European Union for Research Innovation. Grant agreement number: 857159 - SHAPES – H2020 – SC1-FA-DTS – 2018–2020.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andreu, Y., et al.: Wize mirror - a smart, multisensory cardio-metabolic risk monitoring system. Computer Vision and Image Understanding, 148:3–22. Special issue on Assistive Computer Vision and Robotics - "Assistive Solutions for Mobility, Communication and HMI" (2016)
Ben, X., et al.: Video-based facial micro-expression analysis: a survey of datasets, features and algorithms. IEEE Trans. Pattern Anal. Mach. Intell. pp. 1–1 (2021)
Ben, X., Zhang, P., Yan, R., Yang, M., Ge, G.: Gait recognition and micro-expression recognition based on maximum margin projection with tensor representation. Neural Comput. Appl. 27(8), 2629–2646 (2016)
Bickmore, T., Giorgino, T.: Methodological review: health dialog systems for patients and consumers. J. Biomed. Inform.-JBI (2021)
Bouteraa, Y., Abdallah, I.B., Alnowaiser, K., Ibrahim, A.: Smart solution for pain detection in remote rehabilitation. Alexandria Eng. J. 60(4), 3485–3500 (2021)
Chaparro, J.D.: The shapes smart mirror approach for independent living, healthy and active ageing. Sensors, 21(23) (2021)
Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. SIGKDD Explor. Newsl. 19(2), 25–35 (2017)
Thumm, P.C., Giladi, N., Hausdorff, J.M., Mirelman, A.: Tele-rehabilitation with virtual reality: a case report on the simultaneous, remote training of two patients with Parkinson disease. Am. J. Phys. Med. Rehabil. 100(5) (2021)
Gogic, I., Ahlberg, J., Pandzic, I.S.: Regression-based methods for face alignment: a survey. Signal Process. 178, 107755 (2021)
Kepuska, V., Breitfeller, J.: Wake-up-word speech recognition application for first responder communication enhancement. In: Sensors, and Command, Control, Communications, and Intelligence (C3I) Technologies for Homeland Security and Homeland Defense V, vol. 6201, pp, 431–438. SPIE (2006)
Kim, J., Lim, S., Yun, J., Kim, D.H.: Telerehabilitation needs: a bidirectional survey of health professionals and individuals with spinal cord injury in south Korea. Telemedicine Journal and e-health : the Official Journal of the American Telemedicine Association, 18(9), 713–717 (2012)
Liu, B., Mazumder, S.: Lifelong and continual learning dialogue systems: learning during conversation. In: Proceedings of the AAAI Conference on AI, 35(17) (2021)
Maags, C.: Hybridization in china’s elder care service provision. Soc. Pol. Adm. 55(1), 113–127 (2021)
Major, L., Warwick, P., Rasmussen, I., Ludvigsen, S., Cook, V.: Classroom dialogue and digital technologies: a scoping review. Educ. Inf. Technol. 23(5), 1995–2028 (2018)
Mallios, S., Bourbakis, N.: A dialogue monitoring scheme for a virtual doctor. In: 2015 National Aerospace and Electronics Conference (NAECON), pp. 249–253 (2015)
Le Ngo, A.C., Johnston, A., Phan, R.C.W., See, J.: Micro-expression motion magnification: Global lagrangian vs. local Eulerian approaches. In: 13th IEEE International Conference on Automatic Face & Gesture Recognition, pp. 650–656. IEEE Computer Society (2018)
Okur, E., Sahay, S., Nachman, L.: Data augmentation with paraphrase generation and entity extraction for multimodal dialogue system (2022)
Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. In: Chaudhuri, K., Salakhutdinov, R., (eds.), Proceedings of the 36th International Conference on Machine Learning ICML, vol. 97 of Proceedings of Machine Learning Research, pp. 6105–6114. PMLR (2019)
Terrell, E.A., Bopp, A., Neville, K., Scala, D., Zebley, K.: Telerehabilitation policy report: Interprofessional policy principles and priorities. Int. J. Telerehabilitation, 13(2) (2021)
Tradigo, G., Vizza, P., Guzzi, P.H., Fragomeni, G., Ammendolia, A., Veltri, P.: A programmable device to guide rehabilitation patients: design, testing and data collection. In: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1487–1491 (2020)
Williams, M., Evans, P.L., Serriah, M.A.: Modern maxillofacial rehabilitation, pp. 381–420. Springer International Publishing, Cham (2022)
Zak, M., et al.: Frailty syndrome-fall risk and rehabilitation management aided by virtual reality (VR) technology solutions: a narrative review of the current literature. Int. J. Environ. Res. Publ. Health, 19(5), 2985 (2022)
Zhou, L., Shao, X., Mao, Q.: A survey of micro-expression recognition. Image Vis. Comput. 105, 104043 (2021)
Zuiderveld, K.: Contrast Limited Adaptive Histogram Equalization, pp. 474–485. Academic Press Professional Inc, USA (1994)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
López-Fernández, J., Unzueta, L., Garcia, M., Aguirre, M., Méndez, A., Pozo, A.d. (2023). Knowledge-Driven Dialogue and Visual Perception for Smart Orofacial Rehabilitation. In: Tsanas, A., Triantafyllidis, A. (eds) Pervasive Computing Technologies for Healthcare. PH 2022. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 488. Springer, Cham. https://doi.org/10.1007/978-3-031-34586-9_26
Download citation
DOI: https://doi.org/10.1007/978-3-031-34586-9_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34585-2
Online ISBN: 978-3-031-34586-9
eBook Packages: Computer ScienceComputer Science (R0)