Elsevier

Information Sciences

Volume 609, September 2022, Pages 816-837
Information Sciences

Consensus graph learning for auto-weighted multi-view projection clustering

https://doi.org/10.1016/j.ins.2022.07.119Get rights and content

Abstract

Graph-based learning approaches have achieved remarkable success in clustering prevalent multi-view data owing to their capacities to reveal the relation between data and discover its underlying structure. However, real multi-view data is not only simply high-dimensional, but also contains noise and redundant information, so the learned affinity graphs may be unreliable, let alone optimal, and produce inaccurate clustering results. Moreover, existing graph learning based multi-view projection models only learn a common graph or a shared low-dimensional embedding matrix, which fails to preserve the flexible local manifold geometry of each view. To alleviate these problems, a novel consensus graph-based auto-weighted multi-view projection clustering (CGAMPC) is developed, which performs dimensionality reduction, manifold structure preservation and consensus structured graph learning simultaneously. To be specific, the 2,1-norm is leveraged to resist noise and adaptively select discriminative features. Meanwhile, to preserve the manifold structure information of all views, we construct informative similarity graphs for the projection data, and fuse them into a consensus structured graph via an auto-weighted synthesis strategy. Furthermore, an effective alternating iterative algorithm is presented to optimize our CGAMPC. Finally, numerical studies on several multi-view benchmark datasets justify the superiority of the proposed approach over other state-of-the-art clustering approaches.

Introduction

Clustering aims to divide unlabeled data points into their corresponding groups so that data in the same group have the great similarities, and vice versa. Over the past decades, it has been one of the most vital research topics in the realms of data mining, pattern recognition and bioinformatics due to the convenience of obtaining unlabeled data information in real application. Recently, a flurry of effective clustering algorithms were developed, and they can be loosely split into spectral clustering-based approaches [13], [20], matrix factorization based approaches [38], [39], [44] and graph-based approaches [24], [25].

To the best of our knowledge, the data structures are typically described in the form of graphs, which are extensively used in computer vision, image retrieval, image segmentation, social network analysis, etc. Moreover, it is essential to take the structure information of data into consideration during the clustering process to boost the performance. In view of this, we focus on graph-based clustering approaches in this study, and they have obtained impressive performance and received widespread attention among scholars. Traditional graph-based data clustering models [20] work in two sequential steps: 1) constructing the similarity matrix from the original data points beforehand; and 2) subsequently performing spectral clustering (SC) to acquire the final cluster labels. The quality of the informative graphs greatly affects the clustering performance of these algorithms. In other words, if we define a poor graph, the ultimate clustering result might be far from optimal. As a result, similarity measurement is critical to the clustering results. Considerable efforts have been expended to build a high-quality similarity matrix from the original data, such as the binary function, cosine similarity, Euclidean distance, and Gaussian function, to list a few. Nevertheless, different metrics will yield various results, and even these results may vary greatly. Presently, there is no way to judge which metric is the best for a specific dataset. To tackle this issue, automatic similarity learning was proposed, which can be roughly split into self-expression based methods [13], [50] and adaptive neighbor based approaches [24], [34], where the former code each data point as a weighted combination of other data points, and the similarity is essentially the representation coefficient. The latter learn the similarity information by allocating the adaptive neighbors to each data point based on the local distances. Thanks to their capacity to preserve the manifold geometry structure, the latter outperform the former, which only capture the linear structure information. In addition, the latter combine graph construction and data clustering, which increases the flexibility and improves the robustness of algorithms, while SC separates the data clustering process. Under this circumstance, the adaptive neighbor based methods [24], [34] gained increasing popularity due to their ability to recognize arbitrarily shaped clusters and their well-defined mathematical framework. However, they are single-view clustering methods, which are incapable of excavating the correlation between features from different sources.

With the advancement of information technology, real-world data is usually collected from different sources or observed from different perspectives, which is referred to as multi-view data. Some examples of multi-view data are given in Fig. 1. As exhibited in Fig. 1, an object can be depicted by image, audio, text and video, and a particular word can be translated into different languages. Compared with traditional single-view data that cannot accurately grasp the comprehensive information of objects, multi-view data is semantically richer and more useful. Furthermore, the coding information of distinct views is consistent and complementary, which helps to improve clustering performance. Thus, clustering on multiple views by effectively integrating underlying information becomes a major study. In recent years, a variety of graph-based multi-view clustering techniques [10], [21], [22], [38] were presented and have attained excellent performance. Although these models consider that different views generally have different discriminative capabilities and assign ideal weight to each view, they simply learn a common graph [21], [38] or a shared low-dimensional embedding matrix [10], [22], which does not preserve the flexible manifold geometry of each view. Besides, for the ubiquitous high-dimensional multi-view data, the above clustering methods may not be effective since this kind of data contains outliers, noise, and redundant features, and can lead to the curse of dimensionality.

In view of the above analysis, and inspired by single-view projection [24], [30], [35] and multi-view latent embedding learning [1], [43], we develop a joint optimization framework, termed consensus graph-based auto-weighted multi-view projection clustering (CGAMPC), to circumvent the previous issues and learn the discriminative projection for clustering. Different from the two-stage projection learning scheme [50], CGAMPC simultaneously performs projection learning and clustering in a synchronous way, which guarantees the optimum. Notice that, unlike the multi-view projected clustering [4], which learned orthogonal projection matrices to reduce dimensionality, we learn a general projection matrix for each view, and introduce orthogonal constraints to make the projection data statistically independent. In addition, we adopt the 2,1-norm to constrain the projection matrices to select important features and resist unimportant features; accordingly, the adverse influence of noise will be reduced. To make full use of the complementary and consistent information among multiple views as well as preserve the flexible manifold structures of all views, instead of learning a common graph [4], [31], [33], CGAMPC constructs an informative data graph of each view from the projection data and uses an auto-weighted strategy to synthesize them according to the importance of different views. In such a way, large weights are assigned to useful data graphs, whereas small weights are allocated to ineffective data graphs, so that we can get an optimal consensus affinity graph. Meanwhile, the Laplacian matrix rank constraint is introduced to make the consensus graph have a block diagonal structure, which directly displays the final cluster labels without any subsequent clustering operations. The flowchart of our CGAMPC model is depicted in Fig. 2.

To summarize, the primary contributions of our study are as below:

  • We present a novel clustering framework for high-dimensional multi-view data, called CGAMPC, which elegantly combines feature selection, individual similarity graph leaning, manifold structure preservation, consensus structured graph learning and data clustering.

  • In CGAMPC, embedding space learning is implemented by projecting the original multi-view data matrix, which can effectively reduce dimensionality and relieve the curse of dimensionality. Moreover, the 2,1-norm constraint is imposed on the projection matrices to realize effective feature selection and alleviate the adverse impact of noise.

  • Different from the existing multi-view projection clustering algorithms, which only learn a common affinity matrix shared by all views, CGAMPC not only conducts view-specific data graphs and consensus affinity graph learning simultaneously to preserve local manifold structure of each view, but also trade-off the differences in contribution between various view-specific graphs via an auto-weighted fusion scheme.

  • With the Laplacian matrix rank constraint, the learned consensus graph has an explicit clustering structure that can directly display the cluster label for each data point without any subsequent processing steps.

  • An effective iterative algorithm with a theoretical convergence guarantee is proposed to solve our CGAMPC. Furthermore, extensive experimental evaluations substantiate the effectiveness and ascendancy of CGAMPC in dealing with high-dimensional multi-view data clustering in comparison with several competitive methods.

The rest of this paper is structured as follows: Section 2 gives an outline of some notations and recapitulates the related graph-based multi-view clustering approaches. Section 3 presents our CGAMPC model and its optimization algorithm with complexity and convergence analysis. Section 4 presents numerous experiments and analysis. Finally, the conclusion and future works are offered in Section 5.

Section snippets

Related works

In this section, we first give some symbolic conventions, and then revisit the related graph-based multi-view clustering approaches in brief.

The proposed CGAMPC

In this section, the presented method is described in detail. The motivation of the proposed approach is first stated, and then its mathematical model expression is presented. Subsequently, we devise an effective alternating iterative algorithm to optimize our model, and summarize the entire algorithm. Additionally, the computational complexity and convergence are elaborately analyzed. Finally, we compare our work with some related existing techniques.

Experiments and analysis

To verify the effectiveness and superiority of CGAMPC, in this section, we carry out abundant experiments on six widely used real-world datasets, and also compare CGAMPC with ten state-of-the-art clustering algorithms.

Conclusion

In this study, a novel auto-weighted multi-view projection clustering model based on consensus graph learning, named CGAMPC, is developed, which seamlessly aggregates dimensionality reduction, manifold structure preservation, and a self-weighed graph synthesis strategy into a unified optimization framework, and each subtask can be mutually promoted. Different from the preexisting multi-view projection data clustering approaches, we manage to construct a consensus affinity graph from multiple

CRediT authorship contribution statement

Xiaoshuang Sang: Conceptualization, Methodology, Formal analysis, Software, Investigation, Writing - original draft. Jianfeng Lu: Writing - review & editing, Supervision, Data curation, Funding acquisition. Hong Lu: Validation, Formal analysis, Visualization.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgment

This work were supported by the National Key Research and Development Program of China (No. 2018YFB1004900), 111 Project (No. B13022) and the Research Project of Anhui University of Finance and Economics (No. ACKYB22021).

References (50)

  • J. Wen et al.

    Low-rank representation with adaptive graph regularization

    Neural Networks

    (2018)
  • J. Wen et al.

    Consensus guided incomplete multi-view spectral clustering

    Neural Networks

    (2021)
  • D.Y. Xie et al.

    Adaptive latent similarity learning for multi-view clustering

    Neural Networks

    (2020)
  • X. Yu et al.

    Fine-grained similarity fusion for multi-view spectral clustering

    Inf. Sci.

    (2021)
  • X.Q. Zhang et al.

    Robust low-rank kernel multi-view subspace clustering based on the schatten p)norm and correntropy

    Inf. Sci.

    (2019)
  • Y.P. Zhang et al.

    Low-rank preserving embedding

    Pattern Recogn.

    (2017)
  • M.S. Chen et al.

    Multi-view clustering in latent embedding space, in

  • K. Fan

    On a theorem of weyl concerning eigenvalues of linear transformations I

    Proc. Natl. Acad. Sci. U.S.A.

    (1949)
  • A. Kumar et al.

    Co-regularized multi-view spectral clustering

    Adv. Neural Inform. Process. Syst.

    (2011)
  • F.F. Li et al.

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories

    Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshop

    (2004)
  • Y.Q. Li et al.

    Large-scale multi-view spectral clustering via bipartite graph

  • Y.W. Liang et al.

    Consistency meets inconsistency: A unified graph learning framework for multi-view clustering

    Proceedings of IEEE International Conference on Data Mining

    (2019)
  • B.Y. Liu et al.

    Multi-view consensus proximity learning for clustering

    IEEE Trans. Knowl. Data Eng.

    (2020)
  • C.Y. Lu et al.

    Subspace clustering by block diagonal representation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2018)
  • S. Luo et al.

    Consistent and specific multi-view subspace clustering, in

  • Cited by (0)

    View full text