Selection of Relevant Visual Feature Sets for Enhanced Depression Detection using Incremental Linear Discriminant Analysis

Abstract

Presently, while automated depression diagnosis has made great progress, most of the recent works have focused on combining multiple modalities rather than strengthening a single one. In this research work, we present a unimodal framework for depression detection based on facial expressions and facial motion analysis. We investigate a wide set of visual features extracted from different facial regions. Due to high dimensionality of the obtained feature sets, identification of informative and discriminative features is a challenge. This paper suggests a hybrid dimensionality reduction approach which leverages the advantages of the filter and wrapper methods. First, we use a univariate filter method, Fisher Discriminant Ratio, to initially reduce the size of each feature set. Subsequently, we propose an Incremental Linear Discriminant Analysis (ILDA) approach to find an optimal combination of complementary and relevant feature sets. We compare the performance of the proposed ILDA with the batch-mode LDA and also the Composite Kernel based Support Vector Machine (CKSVM) method. The experiments conducted on the Distress Analysis Interview Corpus Wizard-of-Oz (DAIC-WOZ) dataset demonstrate that the best depression classification performance is obtained by using different feature extraction methods in combination rather than individually. ILDA generates better depression classification results in comparison to the CKSVM. Moreover, ILDA based wrapper feature selection incurs lower computational cost in comparison to the CKSVM and the batch-mode LDA methods. The proposed framework significantly improves the depression classification performance, with an F1 Score of 0.805, which is better than all the video based depression detection models suggested in literature, for the DAIC-WOZ dataset. Salient facial regions and well performing visual feature extraction methods are also identified.

Appendix

1.1 1. Composite kernel support vector machine (CKSVM)

Castro et al. [9] in his work for schizophrenia detection, handled the high dimensionality of the fMRI data by using composite kernels and recursive feature elimination. The non-linear relationship between the features(voxels) within a region was captured by using the gaussian kernel, which transforms the features to an infinite dimension Hilbert space, provided with a kernel inner product. Further, to capture the linear relationship between the regions, voxels of different regions in the Hilbert space were combined using the summation kernel. This linear combination of kernels was called as the composite kernel. Composite kernel-based support vector machine (SVM) classifier parameters helped in finding the relevance of the regions instead of the individual features. Further, recursive feature elimination approach was utilized to identify the regions which distinguished the patients and the controls better. Motivated by Castro’s work, we used CKSVM to rank the 19 feature sets and used the forward selection approach to incrementally add a feature set that is the most relevant for the task of depression detection.

Let v_{i, f} denote a feature vector from the f^th feature set, 1 ≤ f ≤ F, for the i^th sample, 1 ≤ i ≤ N. In our study, N = 107 for the training set, N = 35 for the test set and F = 19. Using a non-linear transformation φ_f, feature vectors of the feature set f are mapped to a high dimensional Hilbert space provided that

$$ <{\varphi}_f\left({\mathbf{v}}_{i,f}\right).{\varphi}_f\left({\mathbf{v}}_{j,f}\right)>={k}_f\left({\mathbf{v}}_{i,f},{\mathbf{v}}_{j,f}\right) $$

(13)

where <. > denotes the inner product for a pair of feature vectors in the Hilbert space and k_f(._,.) is a Mercer’s kernel function. We used the gaussian kernel for non-linearly transforming each of the F feature sets. Corresponding to the feature set f, a kernel matrix K_f is generated. The component (i, j) of K_f is computed as

$$ {\mathbf{K}}_f\left(i,j\right)={k}_f\left({\mathbf{v}}_{i,f},{\mathbf{v}}_{j,f}\right)={e}^{-\frac{{\left\Vert {\mathbf{v}}_{i,f}-{\mathbf{v}}_{j,f}\right\Vert}^2}{2{\sigma}^2}} $$

(14)

where σis the gaussian kernel parameter. The feature sets that have been mapped individually can be concatenated into a single vector as

$$ {\varphi}_f\left({\mathbf{v}}_i\right)={\left[{\varphi_f}^T\left({\mathbf{v}}_{i,1}\right)\cdots {\varphi_f}^T\left({\mathbf{v}}_{i,F}\right)\right]}^T $$

(15)

The inner product for a pair of vectors v_i and v_jcan be given as

$$ {\displaystyle \begin{array}{c}<{\upvarphi}_{\mathrm{f}}\left({\mathrm{v}}_{\mathrm{i}}\right).{\upvarphi}_{\mathrm{f}}\left({\mathrm{v}}_{\mathrm{j}}\right)>=\left[{\upvarphi_{\mathrm{f}}}^{\mathrm{T}}\left({\mathrm{v}}_{\mathrm{i},1}\right)\cdots {\upvarphi_{\mathrm{f}}}^{\mathrm{T}}\left({\mathrm{v}}_{\mathrm{i},\mathrm{F}}\right)\right].{\left[{\upvarphi_{\mathrm{f}}}^{\mathrm{T}}\left({\mathrm{v}}_{\mathrm{i},1}\right)\cdots {\upvarphi_{\mathrm{f}}}^{\mathrm{T}}\left({\mathrm{v}}_{\mathrm{i},\mathrm{F}}\right)\right]}^{\mathrm{T}}\\ {}\kern0.5em =\sum \limits_{\mathrm{f}=1}^{\mathrm{F}}{\upvarphi_{\mathrm{f}}}^{\mathrm{T}}\left({\mathrm{v}}_{\mathrm{i},\mathrm{f}}\right).{\upvarphi}_{\mathrm{f}}\left({\mathrm{v}}_{\mathrm{j},\mathrm{f}}\right)=\sum \limits_{\mathrm{f}=1}^{\mathrm{F}}{\mathrm{k}}_{\mathrm{f}}\left({\mathrm{v}}_{\mathrm{i},\mathrm{f}},{\mathrm{v}}_{\mathrm{j},\mathrm{f}}\right)\end{array}} $$

(16)

The above result of the inner product is a composite kernel, expressed as the sum of the kernels for F feature sets. Accordingly, the optimization algorithm of the conventional support vector machine can be modified as:

$$ {\displaystyle \begin{array}{c}\underset{\upalpha}{\max}\sum \limits_{\mathrm{i}=1}^{\mathrm{N}}{\upalpha}_{\mathrm{i}}-\frac{1}{2}\sum \limits_{\mathrm{i}=1}^{\mathrm{N}}\sum \limits_{\mathrm{j}=1}^{\mathrm{N}}{\upalpha}_{\mathrm{i}}{\upalpha}_{\mathrm{j}}{\mathrm{y}}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{j}}\sum \limits_{\mathrm{f}=1}^{\mathrm{F}}{\mathrm{k}}_{\mathrm{f}}\left({\mathrm{v}}_{\mathrm{i},\mathrm{f}},{\mathrm{v}}_{\mathrm{j},\mathrm{f}}\right)\\ {}\begin{array}{c}\mathrm{s}.\mathrm{t}.\kern0.5em \sum \limits_{\mathrm{i}=1}^{\mathrm{N}}{\upalpha}_{\mathrm{i}}{\mathrm{y}}_{\mathrm{i}}=0\\ {}{\upalpha}_{\mathrm{i}}\ge 0\\ {}\begin{array}{c}1\le \mathrm{i},\mathrm{j}\le \mathrm{N}\\ {}1\le \mathrm{f}\le \mathrm{F}\end{array}\end{array}\end{array}} $$

(17)

Similarly, the equation to predict the output of SVM learning algorithm is modified as:

$$ {y}^{\ast }=\sum \limits_{i=1}^N{\alpha}_i{y}_i\sum \limits_{f=1}^F{k}_f\left({\mathbf{v}}_{i,f},{\mathbf{v}}_{\ast, f}\right)+b $$

(18)

where, α_i and b are the classifier parameters. By using composite kernels and the SVM parameter α, it is possible to compute the relevance of a particular feature set as

$$ {\left\Vert {\mathbf{w}}_f\right\Vert}^2={\alpha}^T{\mathbf{K}}_f\alpha $$

(19)

The higher the relevance of a feature set f, higher is the quadratic norm of w_f. Usingthe forward selection approach, an optimum combination of feature sets for depression detection is determined incrementally, based on the ‖w_f‖²for each distinct combination of feature sets. The CKSVM method entailed high time complexity due to the initial kernel computation and parameter tuning of σ for the gaussian kernel.