Skip to main content
SpringerLink
Log in

Attention-based hierarchical denoised deep clustering network

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Clustering is a basic task of data analysis and decision making. Recently, graph convolution network (GCN) based deep clustering frameworks have produced the state-of-the-art performance. However, the traditional GCN has not fully learnt the structural information of the neighbors. Therefore, in this paper, we propose an attention-based hierarchical denoised deep clustering (AHDDC) algorithm to solve the problem, which enables GCN to learn multiple layers of hidden information and uses the attention mechanism to strengthen the information. Besides, we use a denoising autoencoder to reduce the influence of the data noise on the clustering. In AHDDC, Firstly, we input the feature vector of the original data into a denoising autoencoder (DAE) to learn the hidden representation; secondly, the representation information of the autoencoder and the structure information constructed by the KNN graph are passed into a hierarchical attentional graph convolutional network; finally, a self-supervision module is used to optimize the clustering results. Experimental results show the superiorities of our method over most advanced algorithms. Besides, the effectiveness of the proposed hierarchical, attention based and denoising improving strategies are also verified experimentally.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Affeldt, S., Labiod, L., Nadif, M.: Spectral clustering via ensemble deep autoencoder learning (SC-EDAE). Pattern Recognit. 108, 107522 (2020)

    Article  Google Scholar 

  2. Affeldt, S., Labiod, L., Nadif, M.: Spectral clustering via ensemble deep autoencoder learning (sc-edae). Pattern Recognit. 108, 107522 (2020)

    Article  Google Scholar 

  3. Bo, D., Wang, X., Shi, C., Zhu, M., Lu, E., Cui, P.: Structural deep clustering network. In: WWW, pp 1400–1410 (2020)

  4. Cai, J., Wang, S., Guo, W.: Stacked sparse auto-encoder for deep clustering. In: ISPA/BDCloud/Socialcom/Sustaincom, pp 1532–1538 (2019)

  5. Cai, T., Li, J., Mian, A.S., Li, R., Yu, J.X.: Target-aware holistic influence maximization in spatial social networks. IEEE Trans. Knowl. Data Eng. PP(99), 1–1 (2020)

    Article  Google Scholar 

  6. Chen, J., Zhong, M., Li, J., Wang, D., Tu, H.: Effective deep attributed network representation learning with topology adapted smoothing. IEEE Trans. Cybern. PP(99), 1–12 (2021)

    Google Scholar 

  7. Cleuziou, G., Moreno, J.G.: Kernel methods for point symmetry-based clustering. Pattern Recognit. 48(9), 2812–2830 (2015)

    Article  Google Scholar 

  8. Du, J., Michalska, S., Subramani, S., Wang, H., Zhang, Y.: Neural attention with character embeddings for hay fever detection from twitter. Health Inf. Sci. Syst. 7(1), 1–7 (2019)

    Article  Google Scholar 

  9. Haeffele, B.D., You, C., Vidal, R.: A critique of self-expressive deep subspace clustering. In: ICLR (2021)

  10. Hasanzadeh, A., Hajiramezanali, E., Narayanan, K.R., Duffield, N., Zhou, M., Qian, X.: Semi-implicit graph variational auto-encoders. In: NeurIPS, pp 10711–10722 (2019)

  11. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006)

    Article  MATH  Google Scholar 

  12. Hu, G., Lu, G., Zhao, Y.: Fss-gcn: a graph convolutional networks with fusion of semantic and structure for emotion cause analysis. Knowl. Based Syst. 212, 106584 (2021)

    Article  Google Scholar 

  13. Huang, F., Zhang, S., He, M., Wu, X.: Clustering Web documents using hierarchical representation with multi-granularity. World Wide Web 17 (1), 105–126 (2014)

    Article  Google Scholar 

  14. Huang, S., Kang, Z., Tsang, I.W., Xu, Z.: Auto-weighted multi-view clustering via kernelized graph learning. Pattern Recognit. 88, 174–184 (2019)

    Article  Google Scholar 

  15. Ji, J., Liang, Y., Lei, M.: Deep attributed graph clustering with self-separation regularization and parameter-free cluster estimation. Neural Netw. 142, 522–533 (2021)

    Article  Google Scholar 

  16. Karpagalingam, T., Karuppaiah, M.: A hybrid approach for text document clustering using jaya optimization algorithm. Expert Syst. Appl. 178, 115040 (2021)

    Article  Google Scholar 

  17. Kipf, T.N., Welling, M.: Variational graph auto-encoders. coRR (2016)

  18. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: ICLR (2017)

  19. Kou, S., Xia, W., Zhang, X., Gao, Q., Gao, X.: Self-supervised graph convolutional clustering by preserving latent distribution. Neurocomputing 437, 218–226 (2021)

    Article  Google Scholar 

  20. Le Cun, Y., Matan, O., Boser, B.: Denker: handwritten zip code recognition with multilayer networks. In: ICPR, vol. 2, pp 35–40 (1990)

  21. Lewis, D.D., Yang, Y., Russell-Rose, T., Li, F.: Rcv1: a new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361–397 (2004)

    Google Scholar 

  22. Li, Q., Han, Z., Wu, X.: Deeper insights into graph convolutional networks for semi-supervised learning. In: AAAI, pp 3538–3545 (2018)

  23. Li, X., Yin, H., Zhou, K., Zhou, X.: Semi-supervised clustering with deep metric learning and graph embedding. World Wide Web 23(2), 781–798 (2020)

    Article  Google Scholar 

  24. Li, X., Hu, Y., Sun, Y., Hu, J., Zhang, J., Qu, M.: A deep graph structured clustering network. IEEE Access 8, 161727–161738 (2020)

    Article  Google Scholar 

  25. Li, Z., Wang, X., Li, J., Zhang, Q.: Deep attributed network representation learning of complex coupling and interaction. Knowl. Based Syst. 212, 106618 (2021)

    Article  Google Scholar 

  26. Liu, Q., Wu, H., Xu, Z.: Consensus model based on probability k-means clustering algorithm for large scale group decision making. Int. J. Mach. Learn. Cybern. 12(6), 1609–1626 (2021)

    Article  Google Scholar 

  27. Lu, R., Liu, J., Zuo, X.: Attentive multi-view deep subspace clustering net. Neurocomputing 435, 186–196 (2021)

    Article  Google Scholar 

  28. Lu, H., Liu, S., Wei, H., Chen, C., Geng, X.: Deep multi-kernel auto-encoder network for clustering brain functional connectivity data. Neural Netw. 135, 148–157 (2021)

    Article  Google Scholar 

  29. Pan, X., Wang, Y., Chin, K.: Dynamic programming algorithm-based picture fuzzy clustering approach and its application to the large-scale group decisionmaking problem. Comput. Ind. Eng. 157, 107330 (2021)

    Article  Google Scholar 

  30. Rathore, P., Kumar, D., Bezdek, J.C., Rajasegarar, S., Palaniswami, M.: A rapid hybrid clustering algorithm for large volumes of high dimensional data. IEEE Trans. Knowl. Data Eng. 31(4), 641–654 (2019)

    Article  Google Scholar 

  31. Song, X., Li, J., Tang, Y., Zhao, T., Guan, Z.: Jkt: a joint graph convolutional network based deep knowledge tracing. Inf. Sci. 580(4698) (2021)

  32. Stisen, A., Blunck, H.: Bhattacharya: smart devices are different: assessing and mitigatingmobile sensing heterogeneities for activity recognition. In: ACM Conference on Embedded Networked Sensor Systems, pp 127–140 (2015)

  33. Supriya, S., Siuly, S., Wang, H., Zhang, Y.: Automated epilepsy detection techniques from electroencephalogram signals: a review study. Health Inf. Sci. Syst. 8(1), 1–15 (2020)

    Article  Google Scholar 

  34. Tran, D.H., Meunier, M., Cheriet, F.: Deep image clustering using self-learning optimization in a variational auto-encoder. In: ICPR, vol. 12662, pp 736–749 (2020)

  35. Tu, L., Lv, Y., Zhang, Y., Cao, X.: Logistics service provider selection decision making for healthcare industry based on a novel weighted density-based hierarchical clustering. Adv. Eng. Inform. 48, 101301 (2021)

    Article  Google Scholar 

  36. Wang, C., Pan, S., Hu, R., Long, G., Jiang, J., Zhang, C.: Attributed graph clustering: a deep attentional embedding approach. In: IJCAI, pp 3670–3676 (2019)

  37. Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., Yu, P.S.: Heterogeneous graph attention network. In: The World Wide Web Conference, pp 2022–2032 (2019)

  38. Wang, Z., Zuo, L., Ma, J., Chen, S., Li, J., Kang, Z., Zhang, L.: Exploring nonnegative and low-rank correlation for noise-resistant spectral clustering. World Wide Web 23(3), 2107–2127 (2020)

    Article  Google Scholar 

  39. Wang, J., Jiang, J.: Unsupervised deep clustering via adaptive GMM modeling and optimization. Neurocomputing 433, 199–211 (2021)

    Article  Google Scholar 

  40. Wen, G., Zhu, Y., Cai, Z., Zheng, W.: Self-tuning clustering for highdimensional data. World Wide Web 21(6), 1563–1573 (2018)

    Article  Google Scholar 

  41. Wen, J., Zhang, Z., Xu, Y., Zhang, B., Fei, L., Xie, G.: Cdimc-Net: cognitive deep incomplete multi-view clustering network. In: IJCAI, pp 3230–3236 (2020)

  42. Xu, C., Dai, Y., Lin, R., Wang, S.: Deep clustering by maximizing mutual information in variational auto-encoder. Knowl. Based Syst. 205, 106260 (2020)

    Article  Google Scholar 

  43. Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In: ICML. Proceedings of Machine Learning Research, vol. 70, pp 3861–3870 (2017)

  44. Yang, X., Deng, C., Zheng, F., Yan, J., Liu, W.: Deep spectral clustering using dual autoencoder network. In: CVPR, pp 4066–4075 (2019)

  45. Yang, Y., Guan, Z., Li, J., Zhao, W., Cui, J., Wang, Q.: Interpretable and efficient heterogeneous graph convolutional network. IEEE Trans. Knowl. Data Eng. (2021)

  46. Yin, J., Tang, M., Cao, J., Wang, H., You, M., Lin, Y.: Vulnerability exploitation time prediction: an integrated framework for dynamic imbalanced learning, pp. 1–23 (2021)

  47. Yu, H., Zhang, C., Wang, G.: A tree-based incremental overlapping clustering method using the three-way decision theory. Knowl.-Based Syst. 91, 189–203 (2016)

    Article  Google Scholar 

  48. Zhang, X., Liu, H., Li, Q., Wu, X.: Attributed graph clustering via adaptive graph convolution. In: IJCAI, pp 4327–4333 (2019)

  49. Zhang, F., Wang, Y., Liu, S., Wang, H.: Decision-based evasion attacks on tree ensemble classifiers. World Wide Web 23(5), 2957–2977 (2020)

    Article  Google Scholar 

  50. Zhang, Y., Chen, Y., Yang, J., Wang, J., Hu, H., Xing, C., Zhou, X.: Clustering enhanced error-tolerant top-k spatio-textual search. World Wide Web 24(4), 1185–1214 (2021)

    Article  Google Scholar 

  51. Zhao, F., Yang, Y., Zhao, W.: Adaptive clustering algorithm based on max-min distance and bayesian decision theory. Int. J. Comput. Sci. 44(2), 24 (2017)

    Google Scholar 

  52. Zhao, Z., Hu, D., Wang, H., Yu, X.: Minimum distance constrained sparse autoencoder network for hyperspectral unmixing. J. Appl. Remote Sens. 14(4), 048501 (2020)

    Google Scholar 

  53. Zheng, Y., Xu, Z., He, Y., Tian, Y.: A hesitant fuzzy linguistic bi-objective clustering method for large-scale group decision-making. Expert Syst. Appl. 168, 114355 (2021)

    Article  Google Scholar 

  54. Zhou, P., Lu, C., Feng, J., Lin, Z., Yan, S.: Tensor low-rank representation for data recovery and clustering. IEEE Trans. Pattern Anal. Mach. Intell. 43(5), 1718–1732 (2021)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61902106, in part by the Natural Science Foundation of Tianjin under Grant 19JCZDJC40000, in part by the Natural Science Foundation of Hebei Province under Grant F2020202028 and in part Beihang Beidou technology achievement transformation and industrialization funding project under Grant BARI2001.

Author information

Authors and Affiliations

  1. School of Artifcial Intelligence, Hebei University of Technology, Tianjin, 300401, China

    Yongfeng Dong, Ziqiu Wang, Jiapeng Du & Linhao Li

  2. Hebei Province Key Laboratory of Big Data Computing, Tianjin, 300401, China

    Yongfeng Dong, Ziqiu Wang & Linhao Li

  3. Hebei Engineering Research Center of Data-Driven Industrial Intelligent, Tianjin, 300401, China

    Yongfeng Dong, Ziqiu Wang & Linhao Li

  4. Science and Technology on Micro-system Laboratory, Shanghai Institute of Micro-system and Information Technology, Chinese Academy of Sciences, Shanghai, 201800, China

    Weidong Fang

Authors
  1. Yongfeng Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziqiu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiapeng Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Weidong Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Linhao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Linhao Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Decision Making in Heterogeneous Network Data Scenarios and Applications Guest Editors: Jianxin Li, Chengfei Liu, Ziyu Guan, and Yinghui Wu

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, Y., Wang, Z., Du, J. et al. Attention-based hierarchical denoised deep clustering network. World Wide Web 26, 441–459 (2023). https://doi.org/10.1007/s11280-022-01007-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-022-01007-4

Keywords

Search

Navigation