Skip to main content
SpringerLink
Log in

Deep Clustering with Spherical Distance in Latent Space

  • Conference paper
  • First Online:
Advanced Computational Methods for Knowledge Engineering (ICCSAMA 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1121))

Abstract

This paper studies the problem of deep joint-clustering using auto-encoder. For this task, most algorithms solve a multi-objective optimization problem, where it is then transformed into a sing-objective problem by linear scalarization techniques. However, it introduces the scaling problem in latent space in a class of algorithms. We propose an extension to solve this problem by using scale invariance distance functions. The advantage of this extension is demonstrated for a particular case of joint-clustering with MSSC (minimizing sum-of-squares clustering). Numerical experiments on several benchmark datasets illustrate the superiority of our extension over state-of-the-art algorithms with respect to clustering accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://pytorch.org/docs/stable/autograd.html.

  2. 2.

    https://pytorch.org/.

  3. 3.

    https://github.com/JennyQQL/DeepClusterADMM-Release.

  4. 4.

    https://github.com/MaziarMF/deep-k-means.

References

  1. Affeldt, S., Labiod, L., Nadif, M.: Spectral clustering via ensemble deep autoencoder learning (SC-EDAE). arXiv:1901.02291 [cs, stat], January 2019

  2. Aytekin, C., Ni, X., Cricri, F., Aksu, E.: Clustering and unsupervised anomaly detection with \(\ell _2\) normalized deep auto-encoder representations. arXiv:1802.00187 [cs], February 2018

  3. Cai, D., He, X., Han, J.: Locally consistent concept factorization for document clustering. IEEE Trans. Knowl. Data Eng. 23(6), 902–913 (2011)

    Article  Google Scholar 

  4. Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. Lecture Notes in Computer Science, pp. 139–156. Springer, Cham (2018)

    Chapter  Google Scholar 

  5. Chen, D., Lv, J., Zhang, Y.: Unsupervised multi-manifold clustering by learning deep representation. In: Workshops at the Thirty-First AAAI Conference on Artificial Intelligence, March 2017

    Google Scholar 

  6. Das, D., Ghosh, R., Bhowmick, B.: Deep representation learning characterized by inter-class separation for image clustering. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 628–637, January 2019

    Google Scholar 

  7. Dhillon, I.S., Modha, D.S.: Concept decompositions for large sparse text data using clustering. Mach. Learn. 42(1), 143–175 (2001)

    Article  Google Scholar 

  8. Ghasedi Dizaji, K., Herandi, A., Deng, C., Cai, W., Huang, H.: Deep clustering via joint convolutional autoencoder embedding and relative entropy minimization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5736–5745 (2017)

    Google Scholar 

  9. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256, March 2010

    Google Scholar 

  10. Guo, X.: Deep embedded clustering with data augmentation. In: ACML, p. 16 (2018)

    Google Scholar 

  11. Guo, X., Gao, L., Liu, X., Yin, J.: Improved deep embedded clustering with local structure preservation. In: International Joint Conference on Artificial Intelligence (IJCAI-17), pp. 1753–1759 (2017)

    Google Scholar 

  12. Huang, P., Huang, Y., Wang, W., Wang, L.: Deep embedding network for clustering. In: 2014 22nd International Conference on Pattern Recognition, pp. 1532–1537, August 2014

    Google Scholar 

  13. Ji, P., Zhang, T., Li, H., Salzmann, M., Reid, I.: Deep subspace clustering networks. In: NIPS, September 2017

    Google Scholar 

  14. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 [cs], December 2014

  15. Le Tan, D.K., Le, H., Hoang, T., Do, T.T., Cheung, N.M.: DeepVQ: a deep network architecture for vector quantization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 2579–2582 (2018)

    Google Scholar 

  16. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  17. Lovász, L., Plummer, M.D.: Matching Theory. American Mathematical Society, Providence (2009)

    MATH  Google Scholar 

  18. Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML 2010, pp. 807–814. Omnipress, USA (2010)

    Google Scholar 

  19. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323(6088), 533 (1986)

    Article  Google Scholar 

  20. Shah, S.A., Koltun, V.: Deep continuous clustering. arXiv:1803.01449 [cs], March 2018

  21. Shaol, X., Ge, K., Su, H., Luo, L., Peng, B., Li, D.: Deep discriminative clustering network. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE, Rio de Janeiro, July 2018

    Google Scholar 

  22. Song, C., Liu, F., Huang, Y., Wang, L., Tan, T.: Auto-encoder based data clustering. In: Progress in Pattern Recognition. Image Analysis, Computer Vision, and Applications, pp. 117–124. Lecture Notes in Computer Science, Springer, Heidelberg, November 2013

    Chapter  Google Scholar 

  23. Steinbach, M., Ertöz, L., Kumar, V.: The challenges of clustering high dimensional data. In: Wille, L.T. (ed.) New Directions in Statistical Physics: Econophysics, Bioinformatics, and Pattern Recognition, pp. 273–309. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  24. Strehl, A., Ghosh, J.: Cluster ensembles - a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002). http://www.jmlr.org/papers/v3/strehl02a.html

  25. Tian, K., Zhou, S., Guan, J.: DeepCluster: a general clustering framework based on deep learning. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 809–825. Springer (2017)

    Google Scholar 

  26. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747 [cs, stat], August 2017

  27. Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487 (2016)

    Google Scholar 

  28. Yang, B., Fu, X., Sidiropoulos, N.D., Hong, M.: Towards k-means-friendly spaces: simultaneous deep learning and clustering. In: International Conference on Machine Learning, pp. 3861–3870, 17 July 2017. http://proceedings.mlr.press/v70/yang17b.html

  29. Yang, J., Parikh, D., Batra, D.: Joint unsupervised learning of deep representations and image clusters. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5147–5156, June 2016

    Google Scholar 

  30. Zhang, P., Gong, M., Zhang, H., Liu, J.: DRLNet: deep difference representation learning network and an unsupervised optimization framework. In: IJCAI (2017)

    Google Scholar 

  31. Zhang, T., Ji, P., Harandi, M., Hartley, R., Reid, I.: Scalable deep k-subspace clustering. In: Jawahar, C., Li, H., Mori, G., Schindler, K. (eds.) Computer Vision – ACCV 2018. Lecture Notes in Computer Science, pp. 466–481. Springer International Publishing, Cham (2019)

    Chapter  Google Scholar 

  32. Zhang, T., Ji, P., Harandi, M., Huang, W., Li, H.: Neural collaborative subspace clustering. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 7384–7393. PMLR, June 2019

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Applications Department, LGIPM, University of Lorraine, Metz, France

    Bach Tran & Hoai An Le Thi

Authors
  1. Bach Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hoai An Le Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bach Tran .

Editor information

Editors and Affiliations

  1. Computer Science and Applications Department LGIPM, University of Lorraine, Metz Cedex 03, France

    Hoai An Le Thi

  2. Computer Science and Applications Department LGIPM, University of Lorraine, Metz Cedex 03, France

    Hoai Minh Le

  3. Laboratory of Mathematics, National Institute for Applied Sciences, Saint-Étienne-du-Rouvray Cedex, France

    Tao Pham Dinh

  4. Department of Information Systems, Wroclaw University of Science and Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tran, B., Le Thi, H.A. (2020). Deep Clustering with Spherical Distance in Latent Space. In: Le Thi, H., Le, H., Pham Dinh, T., Nguyen, N. (eds) Advanced Computational Methods for Knowledge Engineering. ICCSAMA 2019. Advances in Intelligent Systems and Computing, vol 1121. Springer, Cham. https://doi.org/10.1007/978-3-030-38364-0_21

Download citation

Publish with us

Policies and ethics

Search

Navigation