Abstract
This passage introduces a new clustering approach called BYOL network-based Contrastive Clustering (BCC). This methodology builds on the BYOL framework, which consists of two co-optimized networks: the online and target networks. The online network aims to predict the outputs of the target network while maintaining the similarity relationship between views. The target network is stop-gradient and only updated by EMA of the online network. Additionally, the study incorporates the concept of adversarial learning into the approach to further refine the cluster assignments. The effectiveness of BCC is demonstrated on several mainstream image datasets, achieving impressive results without the need for negative samples or a large batch size. This research showcases the feasibility of using the BYOL architecture for clustering and proposes a novel clustering method that eliminates the problems bring by negative samples and reduce the computational complexity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yang, M.S., Wu, K.L.: A similarity-based robust clustering method. IEEE Trans. Pattern Anal. Mach. Intell. 26(4), 434–448 (2004)
Atilgan, C., Nasibov, E.N.: A space efficient minimum spanning tree approach to the fuzzy joint points clustering algorithm. IEEE Trans. Fuzzy Syst. 27(6), 1317–1322 (2018)
Dixon, W.J., Massey, Jr. F.J.: Introduction to statistical analysis. New York, NY, USA: McGraw-Hill, 344, (1951)
Horn, D., Gottlieb, A.: Algorithm for data clustering in pattern recognition problems based on quantum mechanics. Phys. Rev. Lett. 88(1), 018702 (2001)
Abdi, H., Williams, L.J.: Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2(4), 433–459 (2010)
Hofmann, T., Schölkopf, B., Smola, A.J.: Kernel methods in machine learning. Ann. Stat. 36(3), 1171–1220 (2008)
Bernardi, C., Maday, Y.: Spectral methods. Handbook of numerical analysis 5, 209–485 (1997)
Song, C., Liu, F., Huang, Y., Wang, L., Tan, T.: Auto-encoder based data clustering. In: Ruiz-Shulcloper, J., Sanniti di Baja, G. (eds.) CIARP 2013. LNCS, vol. 8258, pp. 117–124. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-41822-8_15
Chen, T., Kornblith, S., Norouzi, M., et al.: A simple framework for contrastive learning of visual representations. Int. Conf. Mach. Learn., 1597–1607 (2020)
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: SCAN: learning to classify images without labels. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12355, pp. 268–285. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58607-2_16
Li, Y., Hu, P., Liu, Z., et al.: Contrastive clustering. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, issue 10, pp. 8547–8555 (2021)
Grill, J.B., Strub, F., Altché, F., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural. Inf. Process. Syst. 33, 21271–21284 (2020)
Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9865–9874 (2019)
Wu, J., Long, K., Wang, F., et al.: Deep comprehensive correlation mining for image clustering. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 8150–8159 (2019)
Zhao, J., Lu, D., Ma, K., Zhang, Y., Zheng, Y.: Deep image clustering with category-style representation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 54–70. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_4
Darlow, L.N., Storkey, A.: Dhog: deep hierarchical object grouping. arXiv preprint arXiv:2003.08821 (2020)
Niu, C., Zhang, J., Wang, G., Liang, J.: GATCluster: self-supervised gaussian-attention network for image clustering. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 735–751. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_44
Acknowledgment
This work was supported in part by the National Natural Science Foundation of China under Grant No. 62273164 and the Joint Fund of Natural Science Foundation of Shandong Province under Grant No. ZR2020LZH009.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, X. et al. (2023). BYOL Network Based Contrastive Clustering. In: Huang, DS., Premaratne, P., Jin, B., Qu, B., Jo, KH., Hussain, A. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2023. Lecture Notes in Computer Science, vol 14086. Springer, Singapore. https://doi.org/10.1007/978-981-99-4755-3_61
Download citation
DOI: https://doi.org/10.1007/978-981-99-4755-3_61
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-4754-6
Online ISBN: 978-981-99-4755-3
eBook Packages: Computer ScienceComputer Science (R0)