Supervised Deep Canonical Correlation Analysis for Multiview Feature Learning

Abstract

Recently, a new feature representation method called deep canonical correlation analysis (DCCA) has been proposed with high learning performance for multiview feature extraction of high dimensional data. DCCA is an effective approach to learn the nonlinear mappings of two sets of random variables that make the resulting DNN representations highly correlated. However, the DCCA learning process is unsupervised and thus lacks the class label information of training samples on the two views. In order to take full advantage of the class information of training samples, we propose a discriminative version of DCCA referred to as supervised DCCA (SDCCA) for feature learning, which explicitly considers the class information of samples. Compared with DCCA, the SDCCA method can not only guarantee the nonlinear maximal correlation between two views, but also minimize within-class scatter of the samples. With supervision, SDCCA can extract more discriminative features for pattern classification tasks. We test SDCCA on the handwriting recognition and speech recognition using two popular MNIST and XRMB datasets. Experimental results show that SDCCA gets higher performance than several related algorithms.