Hiroki Tanaka
ABSTRACT
This paper proposes an approach to automatically detect dementia from a human face. Although some works have detected dementia from speech and language attributes, there are few studies focusing on facial expression in dementia patients. We recorded the human-agent interaction data of spoken dialogues from 24 participants (12 with dementia and 12 without) and extracted the face features. Our objective was to classify dementia by L1 regularized logistic regression. The facial features and the L1 logistic regression were then used to classify the participants into two groups with 0.82 detection performance, as measured by the areas under the receiver operating characteristic curve. We also identified various contributing features, such as action units, eye gaze, and lip activity. These results demonstrate that our system has the potential to detect dementia from the face through spoken dialog systems and as such, can be of assistance to health care workers.
- [n.d.]. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition: DSM-IV-TR.Google Scholar
- Tagduda Ait Challal and Ouriel Grynszpan. 2018. What Gaze Tells Us About Personality. In Proceedings of the 6th International Conference on Human-Agent Interaction(HAI ’18). ACM, New York, NY, USA, 129–137. https://doi.org/10.1145/3284432.3284455Google ScholarDigital Library
- K. Asplund, L. Jansson, and A. Norberg. 1995. Facial expressions of patients with dementia: a comparison of two methods of interpretation. Int Psychogeriatr 7, 4 (1995), 527–534.Google ScholarCross Ref
- Kenneth Asplund, Astrid Norberg, Rolf Adolfsson, and Howard M Waxman. 1991. Facial expressions in severely demented patients—a stimulus–response study of four patients with dementia of the Alzheimer type. International Journal of Geriatric Psychiatry 6, 8 (1991), 599–606.Google ScholarCross Ref
- Tadas Baltrusaitis, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency. 2018. OpenFace 2.0: Facial Behavior Analysis Toolkit. 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) (2018), 59–66.Google ScholarDigital Library
- T. Endo, N. Ukita, H. Tanaka, N. Hagita, S. Nakamura, H. Adachi, M. Ikeda, H. Kazui, and T. Kudo. 2017. Initial response time measurement in eye movement for dementia screening test. In 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA). 262–265. https://doi.org/10.23919/MVA.2017.7986851Google ScholarCross Ref
- Kathleen C. Fraser, Jed A. Meltzer, and Frank Rudzicz. 2015. Linguistic Features Identify Alzheimer’s Disease in Narrative Speech.Journal of Alzheimer’s disease : JAD 49 2 (2015), 407–22.Google ScholarCross Ref
- D. S. Geldmacher. 2002. Cost-effective recognition and diagnosis of dementia. Semin Neurol 22, 1 (Mar 2002), 63–70.Google ScholarCross Ref
- S. Kato, H. Endo, A. Homma, T. Sakuma, and K. Watanabe. 2013. Early detection of cognitive impairment in the elderly based on Bayesian mining using speech prosody and cerebral blood flow activation. In 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 5813–5816. https://doi.org/10.1109/EMBC.2013.6610873Google ScholarCross Ref
- Yoshino Koichiro, Murase Yukitoshi, Lubis Nurul, Sugiyama Kyoshiro, Tanaka Hiroki, Sakti Sakriani, Takamichi Shinnosuke, and Nakamura Satoshi. 2019. Spoken Dialogue Robot for Watching Daily Life of Elderly People. In International Workshop on Spoken Dialogue System Technology.Google Scholar
- Masateru Matsushita, Yusuke Yatabe, Asuka Koyama, Akiko Katsuya, Daisuke Ijichi, Yusuke Miyagawa, Hiroto Ikezaki, Noboru Furukawa, Manabu Ikeda, and Mamoru Hashimoto. 2018. Are saving appearance responses typical communication patterns in Alzheimer’s disease?PLOS ONE 13, 5 (05 2018), 1–9. https://doi.org/10.1371/journal.pone.0197468Google Scholar
- Bendris Meriem, Charlet Delphine, and Chollet Gerard. 2010. Lip Activity Detection for Talking Faces Classification in TV-Content. In in 3rd International Conference on Machine Vision (ICMV). 187–190.Google Scholar
- B. Mirheidari, D. Blackburn, T. Walker, M. Reuber, and H. Christensen. 2019. Dementia detection using automatic analysis of conversations. Computer Speech and Language 53 (January 2019), 65–79. http://eprints.whiterose.ac.uk/136339/Google Scholar
- Ulrich Seidl, Ulrike Lueken, Philipp A. Thomann, Andreas Kruse, and Johannes Schroder. 2012. Facial Expression in Alzheimer’s Disease: Impact of Cognitive Deficits and Neuropsychiatric Symptoms. American Journal of Alzheimer’s Disease & Other Dementias 27, 2(2012), 100–106. https://doi.org/10.1177/1533317512440495 arXiv:https://doi.org/10.1177/1533317512440495PMID: 22495337.Google Scholar
- Daisaku Shibata, Kaoru Ito, Hiroyuki Nagai, Taro Okahisa, Ayae Kinoshita, and Eiji Aramaki. 2018. Idea density in Japanese for the early detection of dementia based on narrative speech. PLOS ONE 13, 12 (12 2018), 1–12. https://doi.org/10.1371/journal.pone.0208418Google Scholar
- Daisaku Shibata, Shoko Wakamiya, Kaoru Ito, Mai Miyabe, Ayae Kinoshita, and Eiji Aramaki. 2018. VocabChecker: Measuring Language Abilities for Detecting Early Stage Dementia. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion(IUI ’18 Companion). ACM, New York, NY, USA, Article 24, 2 pages. https://doi.org/10.1145/3180308.3180332Google ScholarDigital Library
- Vanessa Taler and Natalie A. Phillips. 2008. Language performance in Alzheimer’s disease and mild cognitive impairment: A comparative review. Journal of Clinical and Experimental Neuropsychology 30, 5(2008), 501–556. https://doi.org/10.1080/13803390701550128 arXiv:https://doi.org/10.1080/13803390701550128PMID: 18569251.Google ScholarCross Ref
- H. Tanaka, H. Adachi, N. Ukita, M. Ikeda, H. Kazui, T. Kudo, and S. Nakamura. 2017. Detecting Dementia Through Interactive Computer Avatars. IEEE Journal of Translational Engineering in Health and Medicine 5 (2017), 1–11. https://doi.org/10.1109/JTEHM.2017.2752152Google ScholarCross Ref
- Hiroki Tanaka, Hideki Negoro, Hidemi Iwasaka, and Satoshi Nakamura. 2017. Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders. PLOS ONE 12, 8 (08 2017), 1–15. https://doi.org/10.1371/journal.pone.0182151Google Scholar
- Tom N Tombaugh and Nancy J McIntyre. 1992. The mini-mental state examination: a comprehensive review. Journal of the American Geriatrics Society 40, 9 (1992), 922–935.Google ScholarCross Ref
- Tsuyoki Ujiro, Hiroki Tanaka, Hiroyoshi Adachi, Hiroaki Kazui, Manabu Ikeda, Takashi Kudo, and Satoshi Nakamura. 2018. Detection of Dementia from Responses to Atypical Questions Asked by Embodied Conversational Agents. In Proc. Interspeech 2018. 1691–1695. https://doi.org/10.21437/Interspeech.2018-1514Google ScholarCross Ref
- Jochen Weiner, Miguel Angrick, Srinivasan Umesh, and Tanja Schultz. 2018. Investigating the Effect of Audio Duration on Dementia Detection Using Acoustic Features. In Proc. Interspeech 2018. 2324–2328. https://doi.org/10.21437/Interspeech.2018-57Google ScholarCross Ref
- Detecting Dementia from Face in Human-Agent Interaction
Recommendations
Human-Computer Interaction Using Emotion Recognition from Facial Expression
EMS '11: Proceedings of the 2011 UKSim 5th European Symposium on Computer Modeling and SimulationThis paper describes emotion recognition system based on facial expression. A fully automatic facial expression recognition system is based on three steps: face detection, facial characteristic extraction and facial expression classification. We have ...
Ambient Pain Monitoring in Older Adults with Dementia to Improve Pain Management in Long-Term Care Facilities
ICMI '20 Companion: Companion Publication of the 2020 International Conference on Multimodal InteractionPainful conditions are prevalent in older adults, yet may go untreated, especially in people with severe dementia who often cannot verbally communicate their pain. Not addressing the pain can lead to the worsening of underlying conditions or lead to ...
A bi-modal face recognition framework integrating facial expression with facial appearance
Among many biometric characteristics, the facial biometric is considered to be the least intrusive technology that can be deployed in the real-world visual surveillance environment. However, in facial biometric, little research attention has been paid ...
Comments