Elsevier

Pattern Recognition

Volume 93, September 2019, Pages 392-403
Pattern Recognition

Structured general and specific multi-view subspace clustering

https://doi.org/10.1016/j.patcog.2019.05.005Get rights and content

Highlights

  • We propose a structured general and specific multi-view subspace clustering method for image clustering.

  • The structural general representation matrix keeps the similarity relationship of data and the specific representation matrices exploit the diversity between different matrices.

  • We present an effective optimization algorithm to solve the proposed objective function.

  • Compared with most state-of-the-arts, experimental results demonstrate that our proposed methods obtain superior performances on four benchmark datasets.

Abstract

In this paper, we propose a structured general and specific multi-view subspace clustering method for image clustering. Unlike most existing multi-view subspace clustering methods which harness the shared cluster structure to preserve the consistence between different views or utilize the diversity regularization to exploit the complementary information from different views, our method learns the structured general and specific representation matrices to obtain the common and specific characteristics of different views with structure consistence and diversity regularization. The general representation matrix guarantees the consistence between different views and the specific representation matrices indicate the diversity among different views. Hence, our method can well exploit the common structure and diversity information of multi-view data. Specifically, the proposed framework can be applied into many existing multi-view subspace clustering methods. Moreover, we develop an efficient and effective optimization approach to solve the objective function of which the time and convergence analyses are also provided. Experimental results on four benchmark datasets are presented to show the effectiveness of proposed method.

Introduction

Subspace clustering has been an important topic in computer vision for many years due to the extensive growth of applications such as image and motion segmentation [1], [2], face clustering [3], [4], and image representation [5], [6]. Generally, the objective of subspace clustering is to segment data points drawn from multiple low-dimensional subspaces into different clusters [7]. Numerous subspace clustering methods have been proposed in recent years such as sparse subspace clustering (SSC) [1], low-rank representation (LRR) [8], least squares regression (LSR) [2], and structured sparse subspace clustering (S3C) [9]. While these subspace clustering methods have achieved encouraging performance on single-view data, it remains a challenge to deal with multi-view data as both specific and generic information coexists in multi-view data.

In computer vision, the requirement to exploit the comprehensive information from multiple distinct features arises as the feature descriptors are easy to obtain and various multi-view datasets are created. Motivated by this, several multi-view subspace clustering methods have been proposed in recent years [10], [11]. Essentially, multi-view subspace clustering methods exploit intrinsic properties, i.e., complementarity and consistency, among different views to jointly enhance the generalization ability of learning models. Representative algorithms in multi-view subspace learning include multi-view low-rank sparse subspace clustering (MLRSSC) [12], diversity-induced multi-view subspace clustering (DiMSC) [13], exclusivity-consistency regularized multi-view subspace clustering (ECMSC) [3], and consistent and specific multi-view subspace clustering (CSMSC) [14]. These multi-view methods have greatly improved performance than single-view methods. However, the consistency based methods fail to utilize the diversity among different views and the complementary based methods insufficiently preserve the common cluster structure by enforcing the affinity matrices to be different. Hence, these methods cannot well utilize the consistency and diversity characteristics among different views.

We propose a novel structured multi-view subspace clustering approach to learn the general and specific self-representation matrices for image clustering in this paper. Fig. 1 illustrates the basic idea of our proposed approach. The general self-representation matrix Z0 is a matrix shared by affinity matrices Di, which follows the consistence principle to maintain the general property that when data points are close in original feature space, their corresponding representations in a new embedding space should be close. While the specific representation matrices Zi hold specific property that the matrices Zi are different to each other and complementary to the general self-representation matrix for representing their specific similarities of corresponding spaces. Fig. 1 shows that D1 and Dv share Z0, Z1 and Zv are complementary to Z0 to build D1 and Dv. Generally, the data point can be approximatively reconstructed by the weighted sum of neighboring points, and the weights which are invariant to linear transformation of data reflect the inherent geometric and similarity properties. Particularly, the same weights are preserved in different embedding spaces [15]. Thus, the weight matrices or similarity matrices in different feature spaces share the intrinsic and common similarity which is entitled as general representation matrix in this paper. And the learnt general representation matrix also keeps the similarity relationship of the original multi-view data. Subspace self-representation builds the affinity matrix to measure the similarity between data points when the sparse or low-rank constraint is imposed on regularization term [1], [8]. For multi-view subspace self-representations, they have the structural shared and general property and also the specific relationship with the diversity information. Motivated by these findings, our proposed method learns the structured general and specific representation matrices to represent the intrinsic similarity relationship and explore the diversity of different views simultaneously. Experimental results are presented to demonstrate the effectiveness of our proposed method.

The contributions of our proposed approach are summarized as follows:

  • 1.

    We propose a structured multi-view subspace clustering approach to learn the structural general representation matrix and specific representation matrices.

  • 2.

    The general representation matrix keeps the similarity relationship of data and the specific representation matrices hold diversity between different matrices.

  • 3.

    We present an effective optimization algorithm to solve the proposed objective function.

  • 4.

    We conduct experiments on four benchmark datasets and the experimental results demonstrate the effectiveness of our proposed method.

Section snippets

Related work

In this section, we briefly review two related works: 1) multi-view learning, and 2) subspace clustering.

Proposed approach

In this section, we elaborate the details of our proposed approach. The notations are summarized in Table 1.

As shown in Fig. 1, we propose a structured general and specific multi-view subspace clustering method. First, we learned the affinity matrices Di in the subspace self-representation stage by using low-rank subspace clustering (LRR). The affinity matrices capture the low-rank structure of each view. Then, we obtained the general representation matrix Z0 and specific representation

Optimization

We propose an optimization method to approximatively solve the objective function in (9) and apply the Augmented Lagrange Multiplier (ALM) [31], [32] to iteratively update all the variables Di, Zi, Ei and Z0, i=1,2,,v. We also study the model complexity following the optimization approach.

Experiments and results

First, we describe the experimental settings including datasets, comparison methods, evaluation metrics and parameter settings. Second, we show experimental results and analyses on benchmark datasets.

Conclusion

In this paper, we have presented a multi-view subspace clustering method. Our proposed method models the common and complementary information among each view via the general and specific representation matrices. The general representation matrix holds the similarity relationship between the feature space and representation space while the specific representation matrices follow the diversity constraint. Due to this, our method is quite different from most existing multi-view subspace clustering

Acknowledgements

This work was supported in part by the National Key Research and Development Program of China under Grant 2017YFA0700802, in part by the National Natural Science Foundation of China under Grant 61672306, Grant U1713214, Grant 61572271, and Grant 61527808, in part by the National 1000 Young Talents Plan Program, and in part by the Shenzhen Fundamental Research Fund (Subject Arrangement) under Grant JCYJ20170412170602564.

Wencheng Zhu received the B.S. degree and the M.Eng. degree both in computer science and technology from Tianjin University, China, in 2014 and 2017, respectively. He is currently pursuing the Ph.D. degree at the Department of Automation, Tsinghua University. His research interests include video summarization and deep learning.

References (40)

  • L.v.d. Maaten et al.

    Visualizing data using t-SNE

    J. Mach. Learn. Res.

    (2008)
  • E. Elhamifar et al.

    Sparse subspace clustering: algorithm, theory, and applications

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • C. Lu et al.

    Subspace clustering by block diagonal representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2019)
  • Z. Ding et al.

    Robust multi-view data analysis through collective low-rank subspace

    IEEE Trans. Neural Netw. Learn. Syst.

    (2018)
  • G. Liu et al.

    Robust recovery of subspace structures by low-rank representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • C.-G. Li et al.

    Structured sparse subspace clustering: a joint affinity learning and subspace clustering framework

    IEEE Trans. Image Process.

    (2017)
  • H. Gao et al.

    Multi-view subspace clustering

    IEEE International Conference on Computer Vision

    (2015)
  • C. Tang et al.

    Learning joint affinity graph for multi-view subspace clustering

    IEEE Trans. Multimed.

    (2018)
  • M. Brbic et al.

    Multi-view low-rank sparse subspace clustering

    Pattern Recognit.

    (2018)
  • S. Luo et al.

    Consistent and specific multi-view subspace clustering

    AAAI Conference on Artificial Intelligence

    (2018)
  • Cited by (0)

    Wencheng Zhu received the B.S. degree and the M.Eng. degree both in computer science and technology from Tianjin University, China, in 2014 and 2017, respectively. He is currently pursuing the Ph.D. degree at the Department of Automation, Tsinghua University. His research interests include video summarization and deep learning.

    Jiwen Lu received the B.Eng. degree in mechanical engineering and the M.Eng. degree in electrical engineering from the Xi’an University of Technology, Xi’an, China, and the Ph.D. degree in electrical engineering from the Nanyang Technological University, Singapore, in 2003, 2006, and 2012, respectively. He is currently an Associate Professor with the Department of Automation, Tsinghua University, Beijing, China. From March 2011 to November 2015, he was a Research Scientist with the Advanced Digital Sciences Center, Singapore. His current research interests include computer vision, pattern recognition, deep learning, and machine learning. He has authored/co-authored over 200 scientific papers in these areas, where more than 100 papers are published in the IEEE Transactions journals and top-tier computer vision conferences. He serves as an Associate Editor of the IEEE Transactions on Image Processing, the IEEE Transactions on Circuits and Systems for Video Technology, the IEEE Transactions on Biometrics, Behavior, and Identity Science, Pattern Recognition, and the Journal of Visual Communication and Image Representation. He is a senior member of the IEEE.

    Jie Zhou received the B.S. and M.S. degrees both from the Department of Mathematics, Nankai University, Tianjin, China, in 1990 and 1992, respectively, and the Ph.D. degree from the Institute of Pattern Recognition and Artificial Intelligence, Huazhong University of Science and Technology (HUST), Wuhan, China, in 1995.From then to 1997, he served as a postdoctoral fellow in the Department of Automation, Tsinghua University, Beijing, China. Since 2003, he has been a full professor in the Department of Automation, Tsinghua University. His research interests include computer vision, pattern recognition, and image processing. In recent years, he has authored more than 200 papers in peer-reviewed journals and conferences. Among them, more than 60 papers have been published in top journals and conferences such as the IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Image Processing, and CVPR. He is an associate editor for the IEEE Transactions on Pattern Analysis and Machine Intelligence, the International Journal of Robotics and Automation and two other journals. He received the National Outstanding Youth Foundation of China Award. He is a senior member of the IEEE.

    View full text