Skip to main content
SpringerLink
Log in

Image Clustering and Generation with HDGMVAE-I

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14554))

Included in the following conference series:

  • 334 Accesses

Abstract

Generating high-quality and realistic images has emerged as a popular research direction in the field of computer vision. We propose the Hierarchically Disentangled Gaussian Mixture Variational Autoencoder model with Importance sampling (HDGMVAE-I) for image generation and clustering. To enhance clustering performance, we introduce total correlation (TC) as a key to computing latent features between samples and cluster centroids and use slack variables to narrow down the feasible solution space. We also introduce Fisher discriminant as a regularization term to minimize within-class distance and maximize between-class distance, particularly effective for samples that are difficult to classify accurately. Moreover, we introduce importance sampling to reduce the information gap between the ELBO and the log-likelihood function, which leads to more realistic generated data. Experimental results demonstrate that our proposed method significantly outperforms the baseline in both clustering and generation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aubry, M., Maturana, D., Efros, A.A., Russell, B.C., Sivic, J.: Seeing 3D chairs: exemplar part-based 2d–3d alignment using a large dataset of cad models. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3762–3769 (2014)

    Google Scholar 

  2. Burda, Y., Grosse, R., Salakhutdinov, R.: Importance weighted autoencoders. arXiv preprint arXiv:1509.00519 (2015)

  3. Caliński, T., Harabasz, J.: A dendrite method for cluster analysis. Commun. Stat. Theory Methods 3(1), 1–27 (1974)

    Article  MathSciNet  Google Scholar 

  4. Chen, R.T., Li, X., Grosse, R.B., Duvenaud, D.K.: Isolating sources of disentanglement in variational autoencoders. In: Advances in Neural Information Processing Systems, vol. 31 (2018)

    Google Scholar 

  5. Chen, S., Huang, J.: Fec: three finetuning-free methods to enhance consistency for real image editing. arXiv preprint arXiv:2309.14934 (2023)

  6. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., Abbeel, P.: InfoGAN: interpretable representation learning by information maximizing generative adversarial nets. In: Advances in Neural Information Processing Systems, vol. 29 (2016)

    Google Scholar 

  7. Davies, D.L., Bouldin, D.W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell. 2, 224–227 (1979)

    Article  Google Scholar 

  8. Dilokthanakul, N., et al.: Deep unsupervised clustering with Gaussian mixture variational autoencoders. arXiv preprint arXiv:1611.02648 (2016)

  9. Esmaeili, B., et al.: Structured disentangled representations. In: The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2525–2534. PMLR (2019)

    Google Scholar 

  10. Goodfellow, I., et al.: Generative adversarial nets. In: Neural Information Processing Systems (2014)

    Google Scholar 

  11. Goyal, P., et al.: Accurate, large minibatch SGD: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)

  12. Higgins, I., et al.: Beta-VAE: learning basic visual concepts with a constrained variational framework. In: International Conference on Learning Representations (2017)

    Google Scholar 

  13. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851 (2020)

    Google Scholar 

  14. Huang, J., Liu, Y., Huang, Y., Chen, S.: Seal2real: prompt prior learning on diffusion model for unsupervised document seal data generation and realisation. arXiv preprint arXiv:2310.00546 (2023)

  15. Huang, J., Liu, Y., Qin, J., Chen, S.: KV inversion: KV embeddings learning for text-conditioned real image action editing. arXiv preprint arXiv:2309.16608 (2023)

  16. Jiang, J., Xia, G.G., Carlton, D.B., Anderson, C.N., Miyakawa, R.H.: Transformer VAE: a hierarchical model for structure-aware and interpretable music representation learning. In: ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 516–520. IEEE (2020)

    Google Scholar 

  17. Kim, H., Mnih, A.: Disentangling by factorising. In: International Conference on Machine Learning, pp. 2649–2658. PMLR (2018)

    Google Scholar 

  18. Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)

  19. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  20. Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3730–3738 (2015)

    Google Scholar 

  21. Liu, Z., Luo, P., Wang, X., Tang, X.: Large-scale celebfaces attributes (celeba) dataset. Retrieved August 15(2018), 11 (2018)

    Google Scholar 

  22. Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(11) (2008)

    Google Scholar 

  23. Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models. In: International Conference on Machine Learning, pp. 1278–1286. PMLR (2014)

    Google Scholar 

  24. Rousseeuw, P.J.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20, 53–65 (1987)

    Article  Google Scholar 

  25. Satheesh, C., Kamal, S., Mujeeb, A., Supriya, M.: Passive sonar target classification using deep generative \(beta \)-VAE. IEEE Sig. Process. Lett. 28, 808–812 (2021)

    Article  Google Scholar 

  26. Shao, J., Li, X.: Generalized zero-shot learning with multi-channel gaussian mixture VAE. IEEE Sig. Process. Lett. 27, 456–460 (2020)

    Article  Google Scholar 

  27. Suekane, K., et al.: Personalized fashion sequential recommendation with visual feature based on conditional hierarchical VAE. In: 2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 362–365. IEEE (2022)

    Google Scholar 

  28. Xiao, H., Rasul, K., Vollgraf, R.: Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

  29. Zacherl, J., Frank, P., Enßlin, T.A.: Probabilistic autoencoder using fisher information. Entropy 23(12), 1640 (2021)

    Article  MathSciNet  Google Scholar 

  30. Zheng, H., Yao, J., Zhang, Y., Tsang, I.W., Wang, J.: Understanding VAEs in Fisher-Shannon plane. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 5917–5924 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Artificial Intelligence, Wuhan Textile University, Wuhan, China

    Yongqi Liu, Jiashuang Zhou & Xiaoqin Du

  2. Engineering Research Center of Hubei Province for Clothing Information, Wuhan, Hubei, China

    Xiaoqin Du

  3. Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion, Wuhan, Hubei, China

    Xiaoqin Du

Authors
  1. Yongqi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiashuang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaoqin Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoqin Du .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Stevan Rudinac

  2. Delft University of Technology, Delft, The Netherlands

    Alan Hanjalic

  3. Delft University of Technology, Delft, The Netherlands

    Cynthia Liem

  4. University of Amsterdam, Amsterdam, The Netherlands

    Marcel Worring

  5. Reykjavik University, Reykjavik, Iceland

    Björn Þór Jónsson

  6. Microsoft Research Lab – Asia, Beijing, China

    Bei Liu

  7. The University of Tokyo, Tokyo, Japan

    Yoko Yamakata

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Liu, Y., Zhou, J., Du, X. (2024). Image Clustering and Generation with HDGMVAE-I. In: Rudinac, S., et al. MultiMedia Modeling. MMM 2024. Lecture Notes in Computer Science, vol 14554. Springer, Cham. https://doi.org/10.1007/978-3-031-53305-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53305-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53304-4

  • Online ISBN: 978-3-031-53305-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation