Speech-based depression detection has attracted significant attention over the last years. A debated problem is whether it is better to use language (what people say), paralanguage (how they say it) or a combination of the two. This article addresses the question through the analysis of a Gated Multimodal Unit trained to weight modalities according to how effectively they account for the condition of a speaker (depressed or non-depressed). The experiments involved 29 individuals diagnosed with depression and 30 non-depressed participants. Besides an accuracy of 83.0% (F1 score 80.0%), the results show that the Gated Multimodal Unit tends to give more weight to paralanguage. However, the relative contribution of language tends to be higher, to a statistically significant extent, in the case of non-depressed speakers.
Cite as: Aloshban, N., Esposito, A., Vinciarelli, A. (2021) Language or Paralanguage, This is the Problem: Comparing Depressed and Non-Depressed Speakers Through the Analysis of Gated Multimodal Units. Proc. Interspeech 2021, 2496-2500, doi: 10.21437/Interspeech.2021-928
@inproceedings{aloshban21_interspeech,
author={Nujud Aloshban and Anna Esposito and Alessandro Vinciarelli},
title={{Language or Paralanguage, This is the Problem: Comparing Depressed and Non-Depressed Speakers Through the Analysis of Gated Multimodal Units}},
year=2021,
booktitle={Proc. Interspeech 2021},
pages={2496--2500},
doi={10.21437/Interspeech.2021-928}
}