Abstract
Clustering is a fundamental task in the field of data analysis. With the development of deep learning, deep clustering focuses on learning meaningful representation with neural networks. Ensemble clustering algorithms combine multiple base partitions into a robust and better consensus clustering. Current deep ensemble clustering algorithms usually neglect shallow and original features. Besides, rarel algorithms use graph attention networks to explore clustering structures. This paper proposes a novel Clustering algorithm based on Multi-layer Features and Graph attention Networks (CMFGN). CMFGN obtains multi-layer features through the hierarchical convolutional layers. Moreover, CMFGN combines the co-association matrix with original features as the Graph Attention Networks (GAT) input to obtain consensus clustering, which reuses original information and leverages GAT to inherit a good clustering structure. Extensive experimental results show that CMFGN remarkably outputs competitive methods on four challenging image datasets. In particular, CMFGN achieves the ACC of 82.14% on the Digits dataset, which is almost up to 6% performance improvement compared with the best baseline.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
A data availability statement is mandatory for publication in this journal. Please confirm that this statement is accurate, or provide an alternative.
References
Abasi AK, Khader AT, Al-Betar MA et al (2021) An improved text feature selection for clustering using binary grey wolf optimizer. In: Proceedings of the 11th national technical seminar on unmanned system technology 2019. Springer, Singapore, pp 503–516
Al Khafaf N, Jalili M, Sokolowski P (2020) A Novel clustering index to find optimal clusters size with application to segmentation of energy consumers. IEEE Trans Ind Inf 17(1):346–355
Bianchi FM, Grattarola D, Alippi C (2020) Spectral clustering with graph neural networks for graph pooling. In: PMLR, pp 874–883
Bo D, Wang X, Shi C et al (2020) Structural deep clustering network. In: Proceedings of the web conference, pp 1400–1410
Bruna, Joan et al (2013) Spectral networks and locally connected networks on graphs. arXiv:1312.6203
Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph Representations. AAAI, pp 1145–1152
Case Western Reserve University. Bearing data center (seeded fault test data) http://www.csegroupscaseedu/bearingdatacenter/home
Charytanowicz M, Perzanowski K, Januszczak M et al (2020) Application of complete gradient clustering algorithm for analysis of wildlife spatial distribution. Ecol Ind 113:106216
Defferrard M, Bresson X, Vandergheynst P (2016a) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf ProceSs Syst 29:3844–3852
Dong X, Yu Z, Cao W et al (2020) A survey on ensemble learning. Front Comp Sci 14(2):241–258
Fred ALN, Jain AK (2002) Data clustering using evidence accumulation. In: 16th International conference on pattern recognition. ICPR, pp 276–280
Fred ALN, Jain AK (2005) Combining multiple clusterings using evidence accumulation. IEEE Trans Pattern Anal Mach Intell 27(6):835–850
Fu Z, Zhao Y, Chang D et al (2021) A hierarchical weighted low-rank representation for image clustering and classification. Pattern Recogn 112:107736
Dizaji GK, Herandi A, Deng C et al (2017) Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: Proceedings of the IEEE international conference on computer vision, pp 5736–5745
Ghosal A, Nandy A, Das AK et al (2020) A short review on different clustering techniques and their applications. In: Emerging technology in modelling and graphics. Springer, Singapore, pp 69–83
Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings of 2005 IEEE international joint conference on neural networks, pp 729–734
Guo X, Gao L, Liu X et al (2017a) Improved deep embedded clustering with local structure preservation. In: IJCAI, pp 1753–1759
Guo X, Liu X, Zhu E et al (2017b) Deep clustering with convolutional autoencoders. In: International conference on neural information processing. Springer, Cham, pp 373–382
Huang D, Wang CD, Lai JH (2017) Locally weighted ensemble clustering. IEEE Trans Cybern 48(5):1460–1473
Huo G, Zhang Y, Gao J et al (2021) CaEGCN: Cross-attention fusion based enhanced graph convolutional network for clustering. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2021.3125020
Ilc N (2020) Weighted cluster ensemble based on partition relevance analysis with reduction step. IEEE Access 8:113720–113736
Jiang Z, Hou Y, Min WU (2018) Clustering ensemble with weighted voting based on feature correlation. Comput Eng Appl 54(3):150–159
Jing P, Su Y, Li Z et al (2021) Learning robust affinity graph representation for multi-view clustering. Inf Sci 544:155–167
Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks. ICLR, 2017:1–14.
Li C, Kulwa F, Zhang J et al (2021) A review of clustering methods in microorganism image analysis. In: Information technology in biomedicine. Springer, Kamien Slaski, Poland, pp 13–25
Liang Y, Ren Z, Wu Z et al (2020) Scalable spectral ensemble clustering via building representative co-association matrix. Neurocomputing 390:158–167
Luo H, Kong F, Li Y (2006) Clustering mixed data based on evidence accumulation. In: International conference on advanced data mining and applications. Springer, Berlin, Heidelberg, pp 348–355
McConville R, Santos-Rodriguez R, Piechocki RJ et al (2021) N2d:(not too) deep clustering via clustering the local manifold of an autoencoded embedding. In: 2020 25th international conference on pattern recognition (ICPR), p 514
Pan S, Hu R, Long G et al (2018) Adversarially regularized graph autoencoder for graph embedding. In: IJCAI, pp 2609–2615
Qi C, Zhang J, Jia H et al (2021) Deep face clustering using residual graph convolutional network. Knowl-Based Syst 211:106561
Saxena A, Prasad M, Gupta A et al (2017) A review of clustering techniques and developments. Neurocomputing 267:664–681
Smith WA, Randall RB (2015) Rolling element bearing diagnostics using the Case Western Reserve University data: a benchmark study. Mechanical systems and signal processing. Elsevier, Sydney, pp 100–131
Strehl A, Ghosh J (2002) Cluster ensembles–-a knowledge reuse framework for combining multiple partitions. J Mach Learn Res 3(9):583–617
Velickovic P, Cucurull G, Casanova A et al (2018) Graph attention networks, vol.1050. In: ICLR, p 4
Wang T, Wei W, Wang F (2019a) Sample pairwise weighting co⁃association matrix based ensemble clustering algorithm. J Nanjing Univ (nat Sci) 55(4):592–600
Wang C, Pan S, Hu R et al (2019b) Attributed graph clustering: a deep attentional embedding approach. In: International joint conference on artificial intelligence, pp 3670–3676
Wu Z, Pan S, Chen F et al (2020) A comprehensive survey on graph neural networks. IEEE Trans Neural Netw Learn Syst 32(1):4–24
Wu X, Guo C, Hu T (2021) The research on clustering ensembles selection algorithm based on semi-supervised K-means clustering. J Phys Conf Ser 1732(1):012074
Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning. PMLR, pp 478–487
Yang B, Fu X, Sidiropoulos ND et al (2017) Towards K-means-friendly spaces: simultaneous deep learning and clustering. In: International conference on machine learning. PMLR, pp 3861–3870
Yen GG, Lin KC (2000) Wavelet packet feature extraction for vibration monitoring[J]. IEEE Trans Ind Electron 47(3):650–667
Zhang X, Zhang Y, Zhang Z. Multi-granularity Recurrent Attention Graph Neural Network for Few-Shot Learning. International Conference on Multimedia Modeling. Springer, Cham, 2021: 147–158.
Zhong R, Wang R, Zou Y et al (2021) Graph attention networks adjusted Bi-LSTM for video summarization. IEEE Signal Process Lett 28:663–667
Funding
This study was funded by the National Natural Science Foundations of China (No. 61976216, No.62276265 and No. 61672522).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Informed consent
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). Additional informed consent was obtained from all patients for which identifying information is included in this article.
Human and animal rights
This article does not contain any studies with human or animal subjects performed by the any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Hou, H., Ding, S., Xu, X. et al. A novel clustering algorithm based on multi-layer features and graph attention networks. Soft Comput 27, 5553–5566 (2023). https://doi.org/10.1007/s00500-023-07848-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-07848-z