Skip to main content
SpringerLink
Log in

Unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

With the development of the times, people generate a huge amount of data every day, most of which are unlabeled data, but manual labeling needs a lot of time and effort, so unsupervised algorithms are being used more often. This paper proposes an unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors (CLKNN). CLKNN is trained in two steps, which are the representation learning step and the clustering step. Contrastive learning and K-nearest neighbors have a huge impact on CLKNN. In the representation learning step, firstly CLKNN processes the image by double data augmentation to get two different augmented images; then CLKNN uses double contrastive loss to extract the high-level feature information of the augmented images, maximizing the similarity of row space and maximizing the similarity of column space to ensure the invariance of information. In the clustering step, CLKNN finds the nearest neighbors of each image by K-nearest neighbors, then it maximizes the similarity between each image and its nearest neighbors to get the final result. To test the performance of CLKNN, the experiments are conducted on CIFAR-10, CIFAR-100 and STL-10 in this paper. From the final results, it is clear that CLKNN has better performance than other advanced algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 4888–4897

  2. Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. Lect Notes Comput Sci 20:139–156

    Article  Google Scholar 

  3. Zeng S, Zhang B, Zhang Y, Gou J (2020) Dual sparse learning via data augmentation for robust facial image classification. Int J Mach Learn Cybernet 11:1717–1734

    Article  Google Scholar 

  4. Chen Z, Ding S, Hou H (2021) A novel self-attention deep subspace clustering. Int J Mach Learn Cybernet. https://doi.org/10.1007/s13042-021-01318-4

    Article  Google Scholar 

  5. Wang W, Song H, Zhao S et al (2019) Learning unsupervised video object segmentation through visual attention. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 3059–3069

  6. Vo HV, Bach F, Cho M et al (2019) Unsupervised image matching and object discovery as optimization. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8279–8288

  7. Fischer A, Igel C (2012) An introduction to restricted Boltzmann machines. Lect Notes Comput Sci 20:14–36

    Article  Google Scholar 

  8. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536

    Article  Google Scholar 

  9. Lecun Y, Bottou L, Bengio Y et al (1988) Gradient-based learning applied to document recognition. Proc IEEE Int Conf Comput Vis 86(11):533–536

    Google Scholar 

  10. Dosovitskiy A, Springenberg JT, Riedmiller M, Brox T (2014) Discriminative unsupervised feature learning with convolutional neural networks. Adv Neural Inf Process Syst 20:766–774

    Google Scholar 

  11. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  Google Scholar 

  12. Masci J, Meier U, Cireşan D et al (2011) Stacked convolutional auto-encoders for hierarchical feature extraction. In: The 21th international conference on artificial neural networks, pp 52–59

  13. Goodfellow I, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial nets. arXiv:1406.2601v1

  14. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. arxiv: 1511.06434

  15. Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: 33rd international conference on machine learning, vol 1, pp 740–749

  16. Dizaji KG, Herandi A, Deng C et al (2017) Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: Proceedings of the IEEE international conference on computer vision, pp 5747–5756

  17. Li F, Qiao H, Zhang B (2018) Discriminatively boosted image clustering with fully convolutional auto-encoders. Pattern Recogn 83:161–173

    Article  Google Scholar 

  18. Yang J, Parikh D, Batra D (2016) Joint unsupervised learning of deep representations and image clusters. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 5147–5156

  19. Caron M, Bojanowski P, Mairal J et al (2019) Unsupervised pre-training of image features on noncurated data. In: Proceedings of the IEEE international conference on computer vision, pp 2959–2968

  20. Huang J, Gong S, Zhu X (2020) Deep semantic clustering by partition confidence maximisation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 8846–8855

  21. Ji X, Vedaldi A, Henriques J (2019) Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 9864–9873

  22. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn 2:1735–1742

    Google Scholar 

  23. Mnih A, Teh YW (2012) A fast and simple algorithm for training neural probabilistic language models. In: Proceedings of the 29th international conference on machine learning, vol 2, pp 1751–1758

  24. Oord A van den, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding. arXiv:1807.03748

  25. He K, Fan H, Wu Y, et al (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 9726–9735

  26. Henaff OJ, Srinivas A, De Fauw J et al (2020) Data-efficient image recognition with contrastive predictive coding. In: 37th international conference on machine learning, pp 4130–4140

  27. Chen T, Kornblith S, Norouzi M et al (2020) A simple framework for contrastive learning of visual representations. arXiv:2002.05709v3

  28. Van Gansbeke W, Vandenhende S, Georgoulis S et al (2020) SCAN: learning to classify images without labels. Lect Notes Comput Sci 20:268–285

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Hebei Provincial Department of education in 2021 provincial postgraduate demonstration course project construction under Grant KCJSX2021024.

Author information

Authors and Affiliations

  1. Engineering Research Center of the Ministry of Education for Intelligent Control System and Intelligent Equipment, Yanshan University, Qinhuangdao, 066004, China

    Xiuling Zhang, Shuo Wang, Ziyun Wu & Xiaofei Tan

  2. Key Laboratory of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao, 066004, China

    Xiuling Zhang

Authors
  1. Xiuling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shuo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaofei Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuo Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Wang, S., Wu, Z. et al. Unsupervised image clustering algorithm based on contrastive learning and K-nearest neighbors. Int. J. Mach. Learn. & Cyber. 13, 2415–2423 (2022). https://doi.org/10.1007/s13042-022-01533-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01533-7

Keywords

Search

Navigation