Skip to main content

Advertisement

ULDC: uncertainty-based learning for deep clustering

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep clustering has gained prominence due to its impressive capability to handle high-dimensional real-world data. However, in the absence of ground-truth labels, existing clustering methods struggle to discern false positives that resemble the target cluster and false negatives that visually differ but maintain semantic consistency. The unreliable projections caused by visual ambiguity disrupt representation learning, leading to sub-optimal clustering outcomes. To address this challenge, we propose a novel method called uncertainty-based learning for deep clustering (ULDC), which aims to discover more optimal cluster structures within data from an uncertainty perspective. Specifically, we utilize the Dirichlet distribution to quantify the uncertainty of feature projections in the latent space, providing a probabilistic framework for modeling uncertainty during the clustering process. We then develop uncertainty-based learning to mitigate the interference caused by false positives and negatives in the clustering tasks. Additionally, a semantic calibration module is introduced to achieve a global alignment of cross-instance semantics, facilitating the learning of clustering-favorite representations. Extensive experiments on five widely-used benchmarks demonstrate the effectiveness of ULDC. The source code is available from https://github.com/YL616/ULDC.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

Data openly available in a public repository. 1. CIFAR-10 and CIFAR-100 are openly available at http://www.cs.toronto.edu/kriz/cifar.html 2. STL-10 is openly available at https://cs.stanford.edu/acoates/stl10/ 3. ImageNet-Dogs is openly available at https://www.kaggle.com/c/dog-breed-identification/overview 4. ImageNet-10 is openly available at https://docs.ultralytics.com/zh/datasets/classify/imagenet10/

References

  1. Wen J, Varol E, Sotiras A et al (2022) Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes. Med Image Anal 75:102304

    Article  Google Scholar 

  2. Chang Y, Tu Z, Xie W et al (2022) Video anomaly detection with spatio-temporal dissociation. Pattern Recognit 122:108213

    Article  Google Scholar 

  3. Bhaskaran S, Marappan R (2023) Design and analysis of an efficient machine learning based hybrid recommendation system with enhanced density-based spatial clustering for digital e-learning applications. Complex Intell Syst 9(4):3517–3533

    Article  Google Scholar 

  4. MacQueen J, et al (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the 15th berkeley symposium on mathematical statistics and probability, pp 281–297

  5. Schubert E, Sander J, Ester M et al (2017) Dbscan revisited, revisited: why and how you should (still) use dbscan. ACM Trans Database Syst 42:1–21. https://doi.org/10.1145/3068335

    Article  MathSciNet  Google Scholar 

  6. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17:395–416. https://doi.org/10.1007/S11222-007-9033-Z

    Article  MathSciNet  Google Scholar 

  7. Peng X, Feng J, Xiao S et al (2018) Structured autoencoders for subspace clustering. IEEE Trans Image Process 27(10):5076–5086

    Article  MathSciNet  Google Scholar 

  8. Ren Y, Hu K, Dai X et al (2019) Semi-supervised deep embedded clustering. Neurocomputing 325:121–130. https://doi.org/10.1016/J.NEUCOM.2018.10.016

    Article  Google Scholar 

  9. Song C, Liu F, Huang Y, et al (2013) Auto-encoder based data clustering. In: Proceedings of the 18th iberoamerican congress on pattern recognition, Part I, pp 117–124. https://doi.org/10.1007/978-3-642-41822-8_15

  10. Kingma DP, Welling M (2014) Auto-encoding variational bayes. In: Proceedings of the 2nd international conference on learning representations

  11. Li Z, Liu F, Yang W, et al (2021) A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Trans Neural Netw Learn Syst 6999–7019. https://doi.org/10.1109/TNNLS.2021.3084827

  12. Caron M, Bojanowski P, JoDCGANulin A, et al (2018) Deep clustering for unsupervised learning of visual features. In: Proceedings of the european conference on vomputer vision, pp 132–149. https://doi.org/10.1007/978-3-030-01264-9_9

  13. Van Gansbeke W, Vandenhende S, Georgoulis S, et al (2020) Scan: Learning to classify images without labels. In: Proceedings of the european conference on computer vision, pp 268–285. https://doi.org/10.1007/978-3-030-58607-2_16

  14. Huang J, Gong S, Zhu X (2020) Deep semantic clustering by partition confidence maximisation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8849–8858. https://doi.org/10.1109/CVPR42600.2020.00887

  15. Chang J, Meng G, Wang L et al (2020) Deep self-evolution clustering. IEEE Trans Pattern Anal Mach intell 42:809–823. https://doi.org/10.1109/TPAMI.2018.2889949

    Article  Google Scholar 

  16. Zhan X, Xie J, Liu Z, et al (2020) Online deep clustering for unsupervised representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. https://doi.org/10.1109/CVPR42600.2020.00672

  17. Cai J, Fan J, Guo W, et al (2022) Efficient deep embedded subspace clustering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1–10. https://doi.org/10.1109/CVPR52688.2022.00012

  18. Chen C, Lu H, Wei H et al (2023) Deep subspace image clustering network with self-expression and self-supervision. Appl Intell 53:4859–4873. https://doi.org/10.1007/S10489-022-03654-6

    Article  Google Scholar 

  19. Li Y, Hu P, Liu Z, et al (2021) Contrastive clustering. In: Proceedings of the AAAI conference on artificial intelligence, pp 8547–8555. https://doi.org/10.1609/AAAI.V35I10.17037

  20. Sadeghi M, Hojjati H, Armanfard N (2023) C3: cross-instance guided contrastive clustering. In: Proceedings of the 34th british machine vision conference, pp 899–900

  21. Huang Z, Chen J, Zhang J et al (2023) Learning representation for clustering via prototype scattering and positive sampling. IEEE Trans Pattern Anal Mach Intell 45:7509–7524. https://doi.org/10.1109/TPAMI.2022.3216454

    Article  Google Scholar 

  22. Zhong Y, Huang D, Wang CD (2023) Deep temporal contrastive clustering. Neural Process Lett 55:7869–7885. https://doi.org/10.1007/S11063-023-11287-0

    Article  Google Scholar 

  23. Zhao Y, Bai L (2024) Contrastive clustering with a graph consistency constraint. Pattern Recognit 146:110032. https://doi.org/10.1016/J.PATCOG.2023.110032

    Article  Google Scholar 

  24. Deng X, Huang D, Chen DH et al (2023) Strongly augmented contrastive clustering. Pattern Recognit 139:109470. https://doi.org/10.1016/J.PATCOG.2023.109470

    Article  Google Scholar 

  25. Molchanov D, Ashukha A, Vetrov D (2017) Variational dropout sparsifies deep neural networks. In: Proceedings of the international conference on machine learning, pp 2498–2507

  26. Zhang C, Bütepage J, Kjellström H et al (2018) Advances in variational inference. IEEE Trans Pattern Anal Mach Intell 41(8):2008–2026. https://doi.org/10.1109/TPAMI.2018.2889774

    Article  Google Scholar 

  27. Malinin A, Gales M (2018) Predictive uncertainty estimation via prior networks. In: Proceedings of the advances in neural information processing systems

  28. Sensoy M, Kaplan L, Kandemir M (2018) Evidential deep learning to quantify classification uncertainty. In: Proceedings of the advances in neural information processing systems

  29. Zhao X, Chen F, Hu S, et al (2020) Uncertainty aware semi-supervised learning on graph data. In: Advances in neural information processing systems, pp 12827–12836

  30. Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resourc Res 31:1351–1373

    Article  Google Scholar 

  31. Hu W, Wang H, He M et al (2023) Uncertainty-aware hierarchical reinforcement learning for long-horizon tasks. Appl Intell 53:28555–28569. https://doi.org/10.1007/S10489-023-05022-4

    Article  Google Scholar 

  32. Sensoy M, Kaplan L, Kandemir M (2018) Evidential deep learning to quantify classification uncertainty. Adv Neural Inf Process Syst 31

  33. Krizhevsky A, Hinton G, et al (2009) Learning multiple layers of features from tiny images. Toronto

  34. Coates A, Ng A, Lee H (2011) An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of the 14th international conference on artificial intelligence and statistics, pp 215–223

  35. Russakovsky O, Deng J, Su H et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  36. Chang J, Wang L, Meng G, et al (2017) Deep adaptive image clustering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5879–5887. https://doi.org/10.1109/ICCV.2017.626

  37. He K, Zhang X, Ren S, et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  38. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

  39. Davies DL, Bouldin DW (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 2:224–227

    Article  Google Scholar 

  40. Zelnik-Manor L, Perona P (2004) Self-tuning spectral clustering. In: Proceedings of the advances in neural information processing systems

  41. Gowda KC, Krishna G (1978) Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recognit 10:105–112. https://doi.org/10.1016/0031-3203(78)90018-3

  42. Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: Proceedings of the international conference on machine learning, pp 478–487

  43. Mehralian M, Karasfi B (2018) Rdcgan: Unsupervised representation learning with regularized deep convolutional generative adversarial networks. In: Proceedings of the 19th conference on artificial intelligence and robotics and 2nd asia-pacific international symposium, pp 31–38

  44. Haeusser P, Plapp J, Golkov V, et al (2019) Associative deep clustering: Training a classification network with no labels. In: Proceedings of the pattern recognition: 14th german conference, pp 18–32. https://doi.org/10.1007/978-3-030-12939-2_2

  45. Xu C, Dai Y, Lin R et al (2020) Deep clustering by maximizing mutual information in variational auto-encoder. Knowl Based Syst 205:106260. https://doi.org/10.1016/J.KNOSYS.2020.106260

    Article  Google Scholar 

  46. Li Y, Yang M, Peng D et al (2022) Twin contrastive learning for online clustering. Int J Comput Vis 130:2205–2221. https://doi.org/10.1007/S11263-022-01639-Z

    Article  Google Scholar 

  47. Tan D, Huang Z, Peng X, et al (2023) Deep adaptive fuzzy clustering for evolutionary unsupervised representation learning. IEEE Trans Neural Netw Learn Syst

  48. Hu Z, Wang Y, Ning H et al (2023) Mutual-taught deep clustering. Knowl Based Syst 282:111100. https://doi.org/10.1016/J.KNOSYS.2023.111100

    Article  Google Scholar 

  49. Wang L, Zhang Y, Feng J (2005) On the euclidean distance of images. IEEE Trans Pattern Anal Mach Intell 27(8):1334–1339

    Article  Google Scholar 

  50. Martinez M, Stiefelhagen R (2019) Taming the cross entropy loss. In: Proceedings of the pattern recognition: 40th german conference, pp 628–637. https://doi.org/10.1007/978-3-030-12939-2_43

  51. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9

  52. Selvaraju RR, Cogswell M, Das A, et al (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE international conference on computer vision, pp 618–626

Download references

Funding

This work was supported by the National Natural Science Foundation of China under grant No.62272087, and the Science and Technology Planning Project of Sichuan Province under grant No.2023YFG0161.

Author information

Authors and Affiliations

Authors

Contributions

Luyao Chang: Methodology and Writing. Xinzheng Niu: Conceptualization and Funding acquisition. Zhenghua Li: Formal analysis. Zhiheng Zhang: Investigation. Shenshen Li: Visualization. Philippe Fournier-Viger: Writing - Review & Editing.

Corresponding author

Correspondence to Xinzheng Niu.

Ethics declarations

Competing Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical and informed consent for data used

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chang, L., Niu, X., Li, Z. et al. ULDC: uncertainty-based learning for deep clustering. Appl Intell 55, 223 (2025). https://doi.org/10.1007/s10489-024-06125-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-06125-2

Keywords