Multiview dimension reduction via Hessian multiset canonical correlations
Introduction
Many computer vision tasks employ multiple information rather than a single representation to achieve more robust performance. For example, an image can be described with color, shape, and texture features [1], [2], [3]. The extracted visual features usually have high dimensions of up to hundreds or thousands, which often causes the problem called the curse-of-dimensionality [4], [5]. Hence multiview dimension reduction algorithms [6], [7] have been subsequently proposed with the purpose of finding an appropriate low-dimensional feature subspace from multiview high-dimensional features. Canonical correlation analysis (CCA) [8] is one of the most representative techniques and has been widely applied to many multiview learning applications including classification, retrieval, regression and clustering.
Canonical correlation analysis (CCA) proposed by Hotelling [8] seeks a pair of linear transformation for two view high-dimensional features such that the corresponding low-dimensional projections are maximally correlated. In recent years, dozens of CCA extensions have been developed and these algorithms can be roughly categorised into the following four groups namely discriminative CCA, sparse CCA, kernel CCA and locality preserving CCA.
Discriminative CCA [9], [10], [11] considers the combination of within-class/between-class information and correlated information of training samples to improve the discrimination of the low-dimensional subspace. For example, Kim et al. [9] developed a linear discriminant function of CCA similarly to linear discriminant analysis (LDA) by maximizing the canonical correlations of within-class sets and minimizing the canonical correlations of between-class sets. Wang et al. [10] proposed an unsupervised discriminant CCA based on spectral clustering by utilizing the correlation information between the samples in the same class including the correlation between paired data, the correlation across views, and the correlation within views. Sun et al. [11] added the uncorrelation constraint into multiview linear discriminant analysis (MLDA) and proposed the multiview ULDA (MULDA) to seek the project directions with minimum redundancy. Besides, they also extended the MULDA to its kernel version, i.e., kernel multiview uncorrelated discriminant analysis (KMUDA), to tackle the nonlinear case.
Sparse CCA [12], [13], [14] seeks two sparse canonical vectors to maximize the correlation between the two views. Hardoon and Shawe-Taylor [12] proposed a sparse CCA by minimizing the number of features while maximizing the correlation between the primal view and the dual view. Witten et al. [13] employed penalized matrix decomposition to optimize l1-regularized sparse CCA. And Chen et al. [14] developed a structured sparse CCA by incorporating the structured-sparsity-inducing penalty to exploit both pre-given group structure and unknown group structure.
Kernel CCA [15], [16], [17], [18] finds maximally nonlinear correlations via an implicit nonlinear mapping to increase the flexibility of the feature selection. Lai and Fyfe [15] used a kernel trick to map the data to a high-dimensional feature space and then performed the traditional CCA on the feature space. Akaho [17] developed regularized kernel CCA to avoid the overfitting by incorporating regularization technique into kernel CCA. Zhu et al. [18] utilized a mixture of kernels rather than a single kernel as the nonlinear mapping and improved both interpolation ability and extrapolation ability of kernel CCA.
Locality preserving CCA incorporates locality information into CCA and then preserves the local manifold structure [19] while obtaining the canonical correlation [20]. Sun and Chen [21] decomposed the global nonlinear structure into many locally linear ones, then conducted linear CCA on each small neighborhood and finally integrated the local sub-problems to obtain the canonical correlation.
Although CCA and its variants have achieved practical success, they are in nature a two-view dimension reduction techniques and cannot handle the problem of multiview dimension reduction that often exists in many real-world applications. Multiset canonical correlation analysis (MCCA) [22] then has been developed to tackle the correlation analysis of multiple variables. Luo et al. [23] proposed tensor CCA to maximize the correlation between the multiple canonical variables by finding the best rank-1 approximation of the high-order covariance tensor. And recently, Yuan et al. [24] proposed Laplacian multiset canonical correlations (LapMCC) to discover the nonlinear correlations among multiview features by combining many locally within-view and between-view correlations together.
In spite of LapMCC has achieved promising performance compared with the traditional MCCA algorithms, it has been identified that Laplacian will bias the solution towards a constant function and then lead to poor extrapolating power [25]. On the other hand, Hessian predicts a high order derivatives and has a richer null space, which makes it can steer the prediction varying smoothly along the underlying manifold. In this paper, we integrate Hessian into the multiset canonical correlations and derive Hessian multiset canonical correlations (HesMCC). HesMCC takes the advantage of Hessian and provides superior extrapolating capability. Therefore, HesMCC can significantly leverage the performance. Fig. 1 graphically demonstrates the whole procedure of the proposed HesMCC. Briefly speaking, the contribution of this paper includes the following three folds: (1) we derive the locality preserving canonical correlation by using Hessian; (2) we further formulate HesMCC to analyze multiset canonical correlation; (3) we present the algorithm of HesMCC and conduct extensive experiments to verify the proposed HesMCC.
Finally, we carefully implement HesMCC for multiview dimension reduction and conduct extensive experiments on USPS database for handwritten digits classification, Yale-B database and ChokePoint database for face recognition and ETH-80 database for object classification respectively. We also compare HesMCC with TCCA, KMUDA, MCCA and LapMCC algorithms to evaluate the performance of HesMCC. The experimental results verify the effectiveness of HesMCC by comparison with the baseline algorithms.
The rest of this paper is organized as follows. We firstly review some related works in Section 2. Then we derive the proposed HesMCC algorithm in Section 3. Section 4 details the implementation of HesMCC. And experimental results are discussed in Section 5, followed by the conclusion in Section 6.
Section snippets
Related work
In this section, we first briefly summarize the related work for the multiview canonical correlation analysis including multiset canonical correlation analysis (MCCA), multiset integrated canonical correlation analysis (MICCA), tensor canonical correlation analysis (TCCA), and Laplacian multiset canonical correlation (LapMCC). And then, we give a brief null space description of Laplacian and Hessian.
Hessian multiset canonical correlations (HesMCC)
In this section, we firstly describe the local preserve CCA by Hessian and then extend it to HesMCC for the multiview case.
Algorithm
In this section, we describe the optimization of problem (4) using Lagrange multiplier technique. Consider λ as the Lagrange multiplier, the Lagrangian function of problem (4) can be written as
The partial derivative of w.r.t. αi has the following expression
Set , we have
The m equations of (7) can be finally reformulated aswhere SH is a block matrix with
Experiments
To evaluate the performance of HesMCC, we implement HesMCC for multiview dimension reduction and compare it with the traditional MCCA and the representative LapMCC algorithm. Particularly, we conduct four experiments which are handwritten digits classification on USPS database [30], face recognition on a combinatorial subset selected from Yale-B and Extend Yale-B databases [31], a more challenging face dataset, i.e., ChokePoint dataset [32], and object classification on ETH-80 database [33].
The
Conclusion
Canonical correlation analysis has proven a successful technique for two-view dimension reduction in both theoretical foundation and practical applications. However, the traditional CCA cannot reveal the nonlinear correlation relationship between multiview features. In this paper, we propose Hessian multiset canonical correlations to tackle this disadvantage of CCA. Hessian has richer null space and can properly exploit the intrinsic local geometry of the data manifold in contrast to Laplacian.
Acknowledgement
This study was supported by the National Natural Science Foundation of China under Grants 61671480, 61572486; the Fundamental Research Funds for the Central Universities under Grant 14CX02203A and the Graduate Student Innovation Project Funding of China University of Petroleum (East China) under Grants YCXJ2016075.
References (33)
- et al.
Face recognition based on pixel-level and feature-level fusion of the top-level's wavelet sub-bands
Inf. Fusion
(2015) - et al.
Using multi-modal semantic association rules to fuse keywords and visual features automatically for web image retrieval
Inf. Fusion
(2011) - et al.
PAC-Bayes analysis of multi-view learning
Inf. Fusion
(2017) - et al.
Unsupervised discriminant canonical correlation analysis based on spectral clustering
Neurocomputing
(2016) - et al.
Dimensionality reduction by mixed kernel canonical correlation analysis
Pattern Recognit.
(2012) - et al.
Fusion of multispectral and panchromatic images via sparse representation and local autoregressive model
Inf. Fusion
(2014) - et al.
Visual feature coding based on heterogeneous structure fusion for image classification
Inf. Fusion
(2017) - et al.
Locality preserving CCA with applications to data visualization and pose estimation
Image Vis. Comput.
(2007) - et al.
A novel multiset integrated canonical correlation analysis framework and its application in feature fusion
Pattern Recognit.
(2011) - et al.
Multiview Spectral Embedding
IEEE Transactions on Systems Man & Cybernetics Part B
(2010)
On the ``dimensionality curse'' and the ``self-similarity blessing''
IEEE Trans. Knowl. Data Eng.
Multi-View Dimensionality Reduction Via Canonical Correlation Analysis
Sparse unsupervised dimensionality reduction for multiple view data
IEEE Trans. Circuits Syst. Video Technol.
Relations between two sets of variates
Biometrika
Discriminative learning and recognition of image set classes using canonical correlations
IEEE Trans. Pattern Anal. Mach. Intell.
Multiview uncorrelated discriminant analysis
IEEE Trans. Cybernet.
Cited by (92)
A review of feature set partitioning methods for multi-view ensemble learning
2023, Information FusionSparse robust subspace learning via boolean weight
2023, Information FusionADT: AI-Driven network Telemetry processing on routers
2023, Computer NetworksNeighbor-consistent multi-modal canonical correlations for feature fusion
2022, Infrared Physics and TechnologyOPLS-SR: A novel face super-resolution learning method using orthonormalized coherent features
2021, Information Sciences