Abstract
Recently, contrastive clustering has demonstrated high performance in the field of deep clustering due to its powerful feature extraction capabilities. However, existing contrastive clustering methods suffer from inter-class conflicts and often produce suboptimal clustering outcomes due to the disregard of latent class information. To address this issue, we propose a novel method called Contrastive learning using Hierarchical data similarities for Deep Clustering (CHDC), consisting of three modules, namely the inter-class separation enhancer, the intra-class compactness enhancer, and the clustering module. Specifically, to induct the latent class information by utilizing the sample pairs with data similarities, the inter-class separation enhancer and the intra-class compactness enhancer handle negative and positive sample pairs, respectively, with distinct hierarchical similarities. Additionally, the clustering module aims to ensure the alignment of cluster assignments between samples and their neighboring samples. By designing these three modules that work collaboratively, inter-class conflicts are alleviated, allowing CHDC to learn more discriminative features. Lastly, we design a novel update method for positive sample pairs to reduce the likelihood of introducing erroneous information. To evaluate the performance of CHDC, we conduct extensive experiments on five widely adopted image classification datasets. The experimental results demonstrate the superiority of CHDC compared to state-of-the-art methods. Moreover, ablation studies demonstrate the effectiveness of the proposed modules.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2, 433–459 (2010)
Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. In: Proceedings of the Advances in Neural Information Processing System, vol. 19 (2006)
Bi, F., Wang, W., Chen, L.: DSCAN: density-based spatial clustering of applications with noise. J. Nanjing Univ. (Nat. Sci.) 48, 491–498 (2012)
Cai, D., He, X., Wang, X., Bao, H., Han, J.: Locality preserving nonnegative matrix factorization. In: Proceedings of the International Joint Conference on Artificial Intelligence (2009)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: Proceedings of the European Conference on Computer Vision, pp. 132–149 (2018)
Chang, J., Guo, Y., Wang, L., Meng, G., Xiang, S., Pan, C.: Deep discriminative clustering analysis (2019)
Chang, J., Wang, L., Meng, G., Xiang, S., Pan, C.: Deep adaptive image clustering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5879–5887 (2017)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: Proceedings of the International Conference on Machine Learning, pp. 1597–1607 (2020)
Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: practical automated data augmentation with a reduced search space. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop, pp. 702–703 (2020)
Gowda, K.C., Krishna, G.: Agglomerative clustering using the concept of mutual nearest neighbourhood. Pattern Recogn. 10, 105–112 (1978)
Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1753–1759 (2017)
Guo, X., Liu, X., Zhu, E., Yin, J.: Deep clustering with convolutional autoencoders. In: Proceedings of International Conference on Neural Information Processing, pp. 373–382 (2017)
Guo, Y., et al.: HCSC: hierarchical contrastive selective coding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9706–9715 (2022)
Haeusser, P., Plapp, J., Golkov, V., Aljalbout, E., Cremers, D.: Associative deep clustering: training a classification network with no labels. In: Proceedings of the German Conference on Pattern Recognition, pp. 18–32 (2019)
Hsu, C.C., Lin, C.W.: CNN-based joint clustering and representation learning with feature drift compensation for large-scale image data. IEEE Trans. Multimedia 20, 421–429 (2017)
Huang, J., Gong, S., Zhu, X.: Deep semantic clustering by partition confidence maximisation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8849–8858 (2020)
Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9865–9874 (2019)
Jiang, Z., Zheng, Y., Tan, H., Tang, B., Zhou, H.: Variational deep embedding: an unsupervised and generative approach to clustering. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1965–1972 (2017)
Khosla, P., et al.: Supervised contrastive learning. In: Proceedings of the Advances in Neural Information Processing System, vol. 33, pp. 18661–18673 (2020)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: Proceedings of the International Conference on Learning Representations (2013)
Kiselev, V.Y., Andrews, T.S., Hemberg, M.: Challenges in unsupervised clustering of single-cell RNA-SEQ data. Nat. Rev. Genet. 20, 273–282 (2019)
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images. Handbook of Systemic Autoimmune Diseases 1 (2009)
Kruskal, J.B., Wish, M.: Multidimensional Scaling, vol. 11. Sage, Thousand Oaks (1978)
Kung, S.Y.: Kernel Methods and Machine Learning. Cambridge University Press, Cambridge (2014)
Li, Y., Hu, P., Liu, Z., Peng, D., Zhou, J.T., Peng, X.: Contrastive clustering. In: Proceedings of the Association for the Advancement of Artificial Intelligence, vol. 35, pp. 8547–8555 (2021)
Ma, X., Kim, W.H.: Locally normalized soft contrastive clustering for compact clusters. In: Proceedings of the International Joint Conference on Artificial Intelligence (2022)
McQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
Mehralian, M., Karasfi, B.: RDCGAN: unsupervised representation learning with regularized deep convolutional generative adversarial networks. In: Proceedings of the Conference on Artificial Intelligence and Robotics and Asia-Pacific International Symposium, pp. 31–38 (2018)
Meng, Q., Qian, H., Liu, Y., Xu, Y., Shen, Z., Cui, L.: MHCCL: masked hierarchical cluster-wise contrastive learning for multivariate time series. In: Proceedings of the Association for the Advancement of Artificial Intelligence (2022)
Ng, A., Jordan, M., Weiss, Y.: On spectral clustering: analysis and an algorithm. In: Proceedings of the Advances in Neural Information Processing System, vol. 14 (2001)
Regatti, J.R., Deshmukh, A.A., Manavoglu, E., Dogan, U.: Consensus clustering with unsupervised representation learning. In: Proceedings of the International Joint Conference on Neural Networks, pp. 1–9 (2021)
Ren, Y., et al.: Deep clustering: a comprehensive survey. arXiv preprint arXiv:2210.04142 (2022)
Reynolds, D.A., et al.: Gaussian mixture models. Encycl. Biomet. 741, 659–663 (2009)
Sarfraz, S., Sharma, V., Stiefelhagen, R.: Efficient parameter-free clustering using first neighbor relations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8934–8943 (2019)
Saunshi, N., Plevrakis, O., Arora, S., Khodak, M., Khandeparkar, H.: A theoretical analysis of contrastive unsupervised representation learning. In: Proceedings of the International Conference on Machine Learning, vol. 97, pp. 5628–5637 (2019)
Shen, S., et al.: Structure-aware face clustering on a large-scale graph with 107 nodes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9085–9094 (2021)
Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks (2015)
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: SCAN: learning to classify images without labels. In: Proceedings of the European Conference on Computer Vision, pp. 268–285 (2020)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A., Bottou, L.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Wang, L., Xiao, Y., Li, J., Feng, X., Li, Q., Yang, J.: IIRWR: internal inclined random walk with restart for lncRNA-disease association prediction. IEEE Access 7, 54034–54041 (2019)
Wu, J., et al.: Deep comprehensive correlation mining for image clustering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8150–8159 (2019)
Xiao, Y., et al.: Reinforcement learning-based non-autoregressive solver for traveling salesman problems. arXiv preprint arXiv:2308.00560 (2023)
Xiao, Y., Xiao, Z., Feng, X., Chen, Z., Kuang, L., Wang, L.: A novel computational model for predicting potential lncRNA-disease associations based on both direct and indirect features of lncRNA-disease pairs. BMC Bioinform. 21, 1–22 (2020)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: Proceedings of the International Conference on Machine Learning, pp. 478–487 (2016)
Yang, J., Parikh, D., Batra, D.: Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5147–5156 (2016)
Zhong, H., et al.: Graph contrastive clustering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9224–9233 (2021)
Acknowledgements
This work is supported by the National Key Research and Development Program of China (2021YFF1201200), the Jilin Provincial Department of Science and Technology Project (20230201083GX, 20220201145GX, and 20230201065GX), the National Natural Science Foundation of China (62072212, 61972174, 61972175, and 12205114), the Guangdong Universities’ Innovation Team Project (2021KCXTD015), and Key Disciplines (2021ZDJS138) Projects.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, Y., Xiao, Y., Wu, X., Song, L., Liang, Y., Zhou, Y. (2024). Leveraging Hierarchical Similarities for Contrastive Clustering. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1962. Springer, Singapore. https://doi.org/10.1007/978-981-99-8132-8_12
Download citation
DOI: https://doi.org/10.1007/978-981-99-8132-8_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8131-1
Online ISBN: 978-981-99-8132-8
eBook Packages: Computer ScienceComputer Science (R0)