Lars Steinert
Dennis Küster
ABSTRACT
Roughly 50 million people worldwide are currently suffering from dementia. This number is expected to triple by 2050. Dementia is characterized by a loss of cognitive function and changes in behaviour. This includes memory, language skills, and the ability to focus and pay attention. However, it has been shown that secondary therapy such as the physical, social and cognitive activation of People with Dementia (PwD) has significant positive effects. Activation impacts cognitive functioning and can help prevent the magnification of apathy, boredom, depression, and loneliness associated with dementia. Furthermore, activation can lead to higher perceived quality of life. We follow Cohen's argument that activation stimuli have to produce engagement to take effect and adopt his definition of engagement as "the act of being occupied or involved with an external stimulus".
Supplemental Material
- Mehmet Berkehan Akçay and Kaya O'uz. 2020. Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers. Speech Communication 116 (2020), 56 -- 76. https://doi.org/10.1016/j.specom.2019.12.001Google Scholar
Digital Library
- S. M. Alarcão. 2017. Reminiscence therapy improvement using emotional information. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). 561--565.Google ScholarCross Ref
- Kenneth Asplund, Astrid Norberg, Rolf Adolfsson, and Howard M. Waxman. 1991. Facial expressions in severely demented patients'a stimulus--response study of four patients with dementia of the Alzheimer type. International Journal of Geriatric Psychiatry 6, 8 (1991), 599--606. https://doi.org/10.1002/gps.930060809 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/gps.930060809Google ScholarCross Ref
- Arlene J Astell, Maggie P Ellis, Lauren Bernardi, Norman Alm, Richard Dye, Gary Gowans, and Jim Campbell. 2010. Using a touch screen computer to support relationships between people with dementia and caregivers. Interacting with Computers 22, 4 (2010), 267--275.Google ScholarDigital Library
- T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L. Morency. 2018. OpenFace 2.0: Facial Behavior Analysis Toolkit. In 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018). 59--66.Google ScholarDigital Library
- Sarah Adel Bargal, Emad Barsoum, Cristian Canton Ferrer, and Cha Zhang. 2016. Emotion recognition in the wild from videos using images. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 433--436.Google ScholarDigital Library
- Hanneke Beerens, Bram de Boer, Sandra Zwakhalen, Frans Tan, Dirk Ruwaard, Jan Hamers, and Hilde Verbeek. 2016. The association between aspects of daily life and quality of life of people with dementia living in long-term care facilities: a momentary assessment study. International Psychogeriatrics -1 (04 2016). https://doi.org/10.1017/S1041610216000466Google Scholar
- Jiska Cohen-Mansfield. 2018. The impact of group activities and their content on persons with dementia attending them. Alzheimer's research & therapy 10, 1 (2018), 37.Google Scholar
- Jiska Cohen-Mansfield, Maha Dakheel-Ali, and Marcia S Marx. 2009. Engagement in persons with dementia: the concept and its measurement. The American journal of geriatric psychiatry 17, 4 (2009), 299--307.Google Scholar
- Jiska Cohen-Mansfield, Marcia S Marx, Khin Thein, and Maha Dakheel-Ali. 2011. The impact of stimuli on affect in persons with dementia. The Journal of clinical psychiatry 72, 4 (2011), 480.Google ScholarCross Ref
- Pasquale Dente, Dennis Küster, Lina Skora, and E. Krumhuber. 2017. Measures and metrics for automatic emotion classification via FACET. In Proceedings of the Conference on the Study of Artificial Intelligence and Simulation of Behaviour (AISB). 160--163.Google Scholar
- Julia Eggert, Cheryl J Dye, Ellen Vincent, Veronica Parker, Shaundra B Daily, Hiep Pham, Alison Turner Watson, Hollie Summey, and Tania Roy. 2015. Effects of viewing a preferred nature image and hearing preferred music on engagement, agitation, and mental status in persons with dementia. SAGE Open Medicine 3 (2015), 2050312115602579. https://doi.org/10.1177/2050312115602579 arXiv: https://doi.org/10.1177/2050312115602579 PMID: 26770801.Google ScholarCross Ref
- Yingruo Fan, Jacqueline C. K. Lam, and Victor O. K. Li. 2018. Video-Based Emotion Recognition Using Deeply-Supervised Neural Networks. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI '18). Association for Computing Machinery, New York, NY, USA, 584--588. https://doi.org/10.1145/3242969.3264978Google Scholar
- Maxi Freudenberg, Reginald B. Adams, Robert E. Kleck, and Ursula Hess. 2015. Through a glass darkly: facial wrinkles affect our processing of emotion in the elderly. Frontiers in Psychology 6 (2015). https://doi.org/10.3389/fpsyg.2015.01476Google Scholar
- Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 (New York, NY, USA) (ICML'16). JMLR.org, 1050--1059.Google ScholarDigital Library
- Cindy Jones, Billy Sung, and Wendy Moyle. 2015. Assessing engagement in people with dementia: a new approach to assessment using video analysis. Archives of psychiatric nursing 29, 6 (2015), 377--382.Google Scholar
- Cindy Jones, Billy Sung, and Wendy Moyle. 2018. Engagement of a Person with Dementia Scale: Establishing content validity and psychometric properties. Journal of advanced nursing 74, 9 (2018), 2227--2240.Google ScholarCross Ref
- Arvid Kappas, Eva Krumhuber, and Dennis Küster. 2013. Facial behavior. In In: Hall, Judith A.; Knapp, Mark L. (Ed.), Nonverbal communication (S. 131--166). Berlin: de Gruyter, 2013. de Gruyter, 131--166.Google Scholar
- A. Kaur, A. Mustafa, L. Mehta, and A. Dhall. 2018. Prediction and Localization of Student Engagement in the Wild. In 2018 Digital Image Computing: Techniques and Applications (DICTA). 1--8. https://doi.org/10.1109/DICTA.2018.8615851Google Scholar
- Heysem Kaya, Furkan Gürpnar, and Albert Ali Salah. 2017. Video-based emotion recognition in the wild using deep transfer learning and score fusion. Image and Vision Computing 65 (2017), 66--75.Google ScholarDigital Library
- Boris Knyazev, Roman Shvetsov, Natalia Efremova, and Artem Kuharenko. 2017. Convolutional neural networks pretrained on large face recognition datasets for emotion classification from video. (2017). arXiv:cs.CV/1711.04598Google Scholar
- Eva Krumhuber, Antony S. R. Manstead, and Arvid Kappas. 2007. Temporal Aspects of Facial Displays in Person and Expression Perception: The Effects of Smile Dynamics, Head-tilt, and Gender. Journal of Nonverbal Behavior 31, 1 (March 2007), 39--56. https://doi.org/10.1007/s10919-006-0019-xGoogle ScholarCross Ref
- M Powell Lawton, Kimberly Van Haitsma, and Jennifer Klapper. 1996. Observed affect in nursing home residents with Alzheimer's disease. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences 51, 1 (1996), P3-- P14.Google ScholarCross Ref
- Chuanhe Liu, Tianhao Tang, Kui Lv, and Minghao Wang. 2018. Multi-Feature Based Emotion Recognition for Video Clips. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI '18). Association for Computing Machinery, New York, NY, USA, 630--634. https://doi.org/10.1145/3242969.3264989Google ScholarDigital Library
- Kaixin Ma, Xinyu Wang, Xinru Yang, Mingtong Zhang, Jeffrey M Girard, and Louis-Philippe Morency. 2019. ElderReact: A Multimodal Dataset for Recognizing Emotional Response in Aging Adults. In 2019 International Conference on Multimodal Interaction (Suzhou, China) (ICMI '19). Association for Computing Machinery, New York, NY, USA, 349--357. https://doi.org/10.1145/3340555.3353747Google Scholar
- Carol Magai, Carl Cohen, David Gomberg, Chris Malatesta, and Clayton Culver. 1996. Emotional expression during mid-to late-stage dementia. International psychogeriatrics 8, 3 (1996), 383--395.Google Scholar
- Margaret McAllister, Jeanne Dayton, Florin Oprescu, Mary Katsikitis, and Christian M Jones. 0. Memory Keeper: A prototype digital application to improve engagement with people with dementia in long-term care (innovative practice). Dementia 0, 0 (0), 1471301217737872. https://doi.org/10.1177/1471301217737872 arXiv: https://doi.org/10.1177/1471301217737872 PMID: 29096546.Google Scholar
- Wendy Moyle, Cindy J Jones, Jenny E Murfield, Lukman Thalib, Elizabeth RA Beattie, David KH Shum, Siobhan T O'Dwyer, M Cindy Mervin, and Brian M Draper. 2017. Use of a robotic seal as a therapeutic tool to improve dementia symptoms: a cluster-randomized controlled trial. Journal of the American Medical Directors Association 18, 9 (2017), 766--773.Google ScholarCross Ref
- Xuesong Niu, Hu Han, Jiabei Zeng, Xuran Sun, Shiguang Shan, Yan Huang, Songfan Yang, and Xilin Chen. 2018. Automatic Engagement Prediction with GAP Feature. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI '18). ACM, New York, NY, USA, 599--603. https://doi.org/10.1145/3242969.3264982Google ScholarDigital Library
- Markus Ojala and Gemma C Garriga. 2010. Permutation tests for studying classifier performance. Journal of Machine Learning Research 11, Jun (2010), 1833--1863.Google Scholar
- Richard V Olsen, B Lynn Hutchings, and Ezra Ehrenkrantz. 2000. 'Media Memory Lane' interventions in an Alzheimer's day care center. American Journal of Alzheimer's Disease 15, 3 (2000), 163--175.Google ScholarCross Ref
- Juri Opitz and Sebastian Burst. 2019. Macro F1 and Macro F1. arXiv:cs.LG/1911.03347Google Scholar
- Viral Parekh, Pin Foong, Shengdong Zhao, and Ramanathan Subramanian. 2018. AVEID: Automatic Video System for Measuring Engagement In Dementia. 409-- 413. https://doi.org/10.1145/3172944.3173010Google Scholar
- Omkar M. Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep Face Recognition., Article 41 (September 2015), 12 pages. https://doi.org/10.5244/C. 29.41Google Scholar
- G. Perugia, M. Díaz-Boladeras, A. Català-Mallofré, E. I. Barakova, and M. Rauterberg. 2020. ENGAGE-DEM: A Model of Engagement of People with Dementia. IEEE Transactions on Affective Computing (2020), 1--1. https://doi.org/10.1109/ TAFFC.2020.2980275Google Scholar
- Giulia Perugia, Daniel Rodríguez-Martín, Marta Díaz Boladeras, Andreu Català Mallofré, Emilia Barakova, and Matthias Rauterberg. 2017. Electrodermal activity: explorations in the psychophysiology of engagement with social robots in dementia. In 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). IEEE, 1248--1254.Google ScholarCross Ref
- Giulia Perugia, Roos van Berkel, Marta Díaz-Boladeras, Andreu Català-Mallofré, Matthias Rauterberg, and Emilia Barakova. 2018. Understanding Engagement in Dementia Through Behavior. The Ethographic and Laban-Inspired Coding System of Engagement (ELICSE) and the Evidence-Based Model of EngagementRelated Behavior (EMODEB). Frontiers in psychology 9 (2018), 690. https://doi.org/10.3389/fpsyg.2018.00690Google Scholar
- M. Powell Lawton, K. Van Haitsma, M. Perkinson, and K. Ruckdeschel. 1999. Observed affect and quality of life in dementia: Further affirmations and problems. Journal of Mental Health and Aging 5, 1 (29 May 1999), 69--81.Google Scholar
- Hans Ragneskog, Kenneth Asplund, Mona Kihlgren, and Astrid Norberg. 2001. Individualized music played for agitated patients with dementia: analysis of video-recorded sessions. International journal of nursing practice 7, 3 (2001), 146--155.Google Scholar
- Susanna Re. 2003. Emotionales Ausdrucksverhalten bei schweren demenziellen Erkrankungen. Zeitschrift für Gerontologie und Geriatrie 36, 6 (2003), 447--453.Google ScholarCross Ref
- Karen Schmidt, Zara Ambadar, Jeffrey Cohn, and Lawrence Reed. 2006. Movement Differences between Deliberate and Spontaneous Facial Expressions: Zygomaticus Major Action in Smiling. Journal of nonverbal behavior 30 (02 2006), 37--52. https://doi.org/10.1007/s10919-005-0003-xGoogle Scholar
- Karen L. Schmidt and Jeffrey F. Cohn. [n.d.]. Human facial expressions as adaptations: Evolutionary questions in facial expression research. 116 ([n. d.]), 3--24. Issue S33. https://doi.org/10.1002/ajpa.20001Google Scholar
- Andrea S Schreiner, Eiko Yamamoto, and Hisako Shiotani. 2005. Positive affect among nursing home residents with Alzheimer's dementia: the effect of recreational activity. Aging & mental health 9, 2 (2005), 129--134.Google Scholar
- Tanja Schultz, Felix Putze, Timo Schulze, Lars Steinert, Ralf Mikut, Wolfgang Doneit, Andreas Kruse, Anamaria Depner, Ingo Franz, Marc Engels, Philipp Gaerte, Sebastian Jünger, Rene Linden, Christof Ziegler, Michael Ricken, Todor Dimitrov, Joachim Herzig, Irene Maucher, Keni Bernardin, and Clarissa Simon. 2018. I-CARE - Ein Mensch-Technik Interaktionssystem zur Individuellen Aktivierung von Menschen mit Demenz.Google Scholar
- Gary W Small, Peter V Rabins, Patricia P Barry, Neil S Buckholtz, Steven T DeKosky, Steven H Ferris, Sanford I Finkel, Lisa P Gwyther, Zaven S Khachaturian, Barry D Lebowitz, et al. 1997. Diagnosis and treatment of Alzheimer disease and related disorders: consensus statement of the American Association for Geriatric Psychiatry, the Alzheimer's Association, and the American Geriatrics Society. Jama 278, 16 (1997), 1363--1371.Google ScholarCross Ref
- Aimee Spector, Lene Thorgrimsen, BOB Woods, Lindsay Royan, Steve Davies, Margaret Butterworth, and Martin Orrell. 2003. Efficacy of an evidence-based cognitive stimulation therapy programme for people with dementia: randomised controlled trial. The British Journal of Psychiatry 183, 3 (2003), 248--254.Google ScholarCross Ref
- Chinchu Thomas, Nitin Nair, and Dinesh Babu Jayagopi. 2018. Predicting Engagement Intensity in the Wild Using Temporal Convolutional Network. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI '18). ACM, New York, NY, USA, 604--610. https://doi.org/10.1145/3242969.3264984Google ScholarDigital Library
- Anthony J Viera, Joanne M Garrett, et al. 2005. Understanding interobserver agreement: the kappa statistic. Fam med 37, 5 (2005), 360--363.Google Scholar
- K. Wada, Yousuke Ikeda, Kaoru Inoue, and Reona Uehara. 2010. Development and preliminary evaluation of a caregiver's manual for robot therapy using the therapeutic seal robot Paro. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication, 533--538. https://doi.org/10.1109/ROMAN.2010.5598615Google ScholarCross Ref
- WHO. 2017. Dementia. https://www.who.int/news-room/fact-sheets/detail/ dementia. [Online; accessed 27-March-2019].Google Scholar
- Bob Woods, Elisa Aguirre, Aimee E Spector, and Martin Orrell. 2012. Cognitive stimulation to improve cognitive functioning in people with dementia. Cochrane Database of Systematic Reviews 2 (2012).Google Scholar
- Jianfei Yang, Kai Wang, Xiaojiang Peng, and Yu Qiao. 2018. Deep Recurrent Multi-instance Learning with Spatio-temporal Features for Engagement Intensity Prediction. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (Boulder, CO, USA) (ICMI '18). ACM, New York, NY, USA, 594--598. https://doi.org/10.1145/3242969.3264981Google ScholarDigital Library
- Zhihong Zeng, Maja Pantic, Glenn I Roisman, and Thomas S Huang. 2009. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE transactions on pattern analysis and machine intelligence 31, 1 (2009), 39--58.Google Scholar
Index Terms
- Towards Engagement Recognition of People with Dementia in Care Settings
Recommendations
A Study on Effect of Media Therapy for the Elderly with Dementia to Nursing Care Quality
Proceedings of the 8th International Conference on Universal Access in Human-Computer Interaction. Aging and Assistive Environments - Volume 8515Japanese society contains an extremely large elderly population, unprecedented elsewhere in the world. In fact, the elderly make up about 23.3% of the Japanese population. Consequently, the number of elderly people with dementia is also increasing at an ...
Categorical processing of negative emotions from speech prosody
Everyday communication involves processing nonverbal emotional cues from auditory and visual stimuli. To characterize whether emotional meanings are processed with category-specificity from speech prosody and facial expressions, we employed a cross-...
Emotion Estimation for Elderly People with Dementia Using EEG and HRV
HCI International 2023 – Late Breaking PapersAbstractIn recent years, the number of elderly people with dementia has been increasing worldwide, and at the same time, the demand to care for elderly people with dementia has also been increasing. However, there is a chronic shortage of caregivers. ...
Comments