Skip to main content
Log in

Heterogeneous clustering via adversarial deep Bayesian generative model

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

This paper aims to study the deep clustering problem with heterogeneous features and unknown cluster number. To address this issue, a novel deep Bayesian clustering framework is proposed. In particular, a heterogeneous feature metric is first constructed to measure the similarity between different types of features. Then, a feature metric-restricted hierarchical sample generation process is established, in which sample with heterogeneous features is clustered by generating it from a similarity constraint hidden space. When estimating the model parameters and posterior probability, the corresponding variational inference algorithm is derived and implemented. To verify our model capability, we demonstrate our model on the synthetic dataset and show the superiority of the proposed method on some real datasets. Our source code is released on the website: Github.com/yexlwh/Heterogeneousclustering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Jiang Z, Zheng Y, Tan H, Tang B, Zhou H. Variational deep embedding: an unsupervised and generative approach to clustering. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1965–1972

  2. Bhattacharjee P, Mitra P. A survey of density based clustering algorithms. Frontiers of Computer Science, 2021, 15(1): 151308

    Article  Google Scholar 

  3. Xue H, Li S, Chen X, Wang Y. A maximum margin clustering algorithm based on indefinite kernels. Frontiers of Computer Science, 2019, 13(4): 813–827

    Article  Google Scholar 

  4. Ghasedi K, Wang X, Deng C, Huang H. Balanced self-paced learning for generative adversarial clustering network. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 4386–4395

  5. Wen J, Zhang Z, Xu Y, Zhang B, Fei L, Xie G S. CDIMC-net: cognitive deep incomplete multi-view clustering network. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2021, 447

  6. Xie J, Girshick R, Farhadi A. Unsupervised deep embedding for clustering analysis. In: Proceedings of the 3rd International Conference on Machine Learning. 2016, 478–487

  7. Zhou P, Hou Y, Feng J. Deep adversarial subspace clustering. In: Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 1596–1604

  8. Peng X, Xiao S, Feng J, Yau W Y, Yi Z. Deep subspace clustering with sparsity prior. In: Proceedings of the 25th International Joint Conference on Artificial Intelligence. 2016, 1925–1931

  9. Guo X, Gao L, Liu X, Yin J. Improved deep embedded clustering with local structure preservation. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence. 2017, 1753–1759

  10. Ji P, Zhang T, Li H, Salzmann M, Reid I. Deep subspace clustering networks. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. 2017, 23–32

  11. Yu Y, Zhou W J. Mixture of GANs for clustering. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2018, 3047–3053

  12. Yang X, Deng C, Zheng F, Yan J, Liu W. Deep spectral clustering using dual autoencoder network. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 4061–4070

  13. Shaham U, Stanton K P, Li H, Basri R, Nadler B, Kluger Y. SpectralNet: spectral clustering using deep neural networks. In: Proceedings of the 6th International Conference on Learning Representation. 2018

  14. Cheng J, Wang Q, Tao Z, Xie D, Gao Q. Multi-view attribute graph convolution networks for clustering. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2021, 411

  15. Menapace W, Lathuilière S, Ricci E. Learning to cluster under domain shift. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 736–752

  16. Tapaswi M, Law M T, Fidler S. Video face clustering with unknown number of clusters. In: Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. 2019, 5026–5035

  17. Yang L, Zhan X, Chen D, Yan J, Boy C C, Lin D. Learning to cluster faces on an affinity graph. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 2293–2301

  18. Li J, Lu K, Huang Z, Zhu L, Shen H T. Heterogeneous domain adaptation through progressive alignment. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(5): 1381–1391

    Article  MathSciNet  Google Scholar 

  19. Yang S, Song G, Jin Y, Du L. Domain adaptive classification on heterogeneous information networks. In: Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2021, 196

  20. Wang C, Mahadevan S. Heterogeneous domain adaptation using manifold alignment. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence. 2011, 1541–1546

  21. Tsai Y H H, Yeh Y R, Wang Y C F. Learning cross-domain landmarks for heterogeneous domain adaptation. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 5081–5090

  22. Yeh Y R, Huang C H, Wang Y C F. Heterogeneous domain adaptation and classification by exploiting the correlation subspace. IEEE Transactions on Image Processing, 2014, 23(5): 2009–2018

    Article  MathSciNet  Google Scholar 

  23. Wang M, Deng W. Deep visual domain adaptation: a survey. Neurocomputing, 2018, 312: 135–153

    Article  Google Scholar 

  24. Day O, Khoshgoftaar T M. A survey on heterogeneous transfer learning. Journal of Big Data, 2017, 4(1): 29

    Article  Google Scholar 

  25. Wang H, Yang Y, Liu B. GMC: graph-based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, 2020, 32(6): 1116–1129

    Article  Google Scholar 

  26. Shi S, Nie F, Wang R, Li X. Fast multi-view clustering via prototype graph. IEEE Transactions on Knowledge and Data Engineering, 2021, doi: https://doi.org/10.1109/TKDE.2021.3078728

  27. Li Z, Nie F, Chang X, Yang Y, Zhang C, Sebe N. Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(12): 6323–6332

    Article  MathSciNet  Google Scholar 

  28. Yin J, Sun S. Incomplete multi-view clustering with reconstructed views. IEEE Transactions on Knowledge and Data Engineering, 2021, doi: https://doi.org/10.1109/TKDE.2021.3112114

  29. Li L, Wan Z, He H. Incomplete multi-view clustering with joint partition and graph learning. IEEE Transactions on Knowledge and Data Engineering, 2021, doi: https://doi.org/10.1109/TKDE.2021.3082470

  30. Wang Y, Zhu J. DP-space: Bayesian nonparametric subspace clustering with small-variance asymptotics. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 862–870

  31. Gholami B, Pavlovic V. Probabilistic temporal subspace clustering. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 4313–4322

  32. Simo-Serra E, Torras C, Moreno-Noguer F. 3D human pose tracking priors using geodesic mixture models. International Journal of Computer Vision, 2017, 122(2): 388–408

    Article  MathSciNet  Google Scholar 

  33. Straub J, Freifeld O, Rosman G, Leonard J J, Fisher J W. The Manhattan frame model—Manhattan world inference in the space of surface normals. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(1): 235–249

    Article  Google Scholar 

  34. Ye X, Zhao J. Multi-manifold clustering: a graph-constrained deep nonparametric method. Pattern Recognition, 2019, 93: 215–227

    Article  Google Scholar 

  35. Ye X, Zhao J, Zhang L, Guo L. A nonparametric deep generative model for multimanifold clustering. IEEE Transactions on Cybernetics, 2019, 49(7): 2664–2677

    Article  Google Scholar 

  36. Hannah L A, Blei D M, Powell W B. Dirichlet process mixtures of generalized linear models. The Journal of Machine Learning Research, 2011, 12: 1923–1953

    MathSciNet  Google Scholar 

  37. Wang Y, Zhu J. Small-variance asymptotics for Dirichlet process mixtures of SVMs. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014, 2135–2141

  38. Blei D M, Jordan M I. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 2006, 1(1): 121–143

    Article  MathSciNet  Google Scholar 

  39. Li Z, Cheong L F, Yang S, Toh K C. Simultaneous clustering and model selection: algorithm, theory and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(8): 1964–1978

    Article  Google Scholar 

  40. Liang J, Yang J, Cheng M M, Rosin P L, Wang L. Simultaneous subspace clustering and cluster number estimating based on triplet relationship. IEEE Transactions on Image Processing, 2019, 28(8): 3973–3985

    Article  MathSciNet  Google Scholar 

  41. Rodriguez A, Laio A. Clustering by fast search and find of density peaks. Science, 2014, 344(6191): 1492–1496

    Article  Google Scholar 

  42. Ye X L, Zhao J, Chen Y, Guo L J. Bayesian adversarial spectral clustering with unknown cluster number. IEEE Transactions on Image Processing, 2020, 29: 8506–8518

    Article  MathSciNet  Google Scholar 

  43. Mukherjee S, Asnani H, Lin E, Kannan S. ClusterGAN: latent space clustering in generative adversarial networks. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 4610–4617

  44. Chen W Y, Hsu T M H, Tsai Y H H, Wang Y C F, Chen M S. Transfer neural trees for heterogeneous domain adaptation. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 399–414

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Grant Nos. 62006131, 62071260), the National Natural Science Foundation of Zhejiang Province (LQ21F020009, LQ18F020001).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xulun Ye.

Additional information

Xulun Ye received the MSc and PhD degrees from Ningbo University, China in 2016 and 2019, respectively, where he is currently a lecturer. His research interests include Bayesian learning, deep learning, nonparametric clustering and convex analysis.

Jieyu Zhao received the BS and MSc degrees from Zhejiang University, China and the PhD degree from Royal Holloway University of London, UK in 1985, 1988 and 1995 respectively. He is currently a full professor at Ningbo University, China. His research interests include deep learning, and computer vision.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ye, X., Zhao, J. Heterogeneous clustering via adversarial deep Bayesian generative model. Front. Comput. Sci. 17, 173322 (2023). https://doi.org/10.1007/s11704-022-1376-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-1376-2

Keywords

Navigation