skip to main content
10.1145/3340531.3412103acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
short-paper

Multimodal Clustering via Deep Commonness and Uniqueness Mining

Published: 19 October 2020 Publication History

Abstract

Deep multimodal clustering have shown their competitiveness among different multimodal clustering algorithms. Existing algorithms usually boost the multimodal clustering by exploring the common knowledge among multiple modalities, which underutilizes the uniqueness of multiple modalities. In this paper, we enhance the mining of modality-common knowledge by extracting the modality-unique knowledge of each modality simultaneously. Specifically, we first utilize autoencoders to extract the modality-common and modality-unique features of each modality respectively. Meanwhile, the cross reconstruction is used to build latent connections among different modalities, i.e., maintain the consistency of modality-common features of each modality as well as heightening the diversity of modality-unique features of each modality. After that, modality-common features are fused to cluster the multimodal data. Experimental results on several benchmark datasets demonstrate that the proposed method outperforms state-of-art works obviously.

References

[1]
Mahdi Abavisani and Vishal M Patel. 2018. Deep multimodal subspace clustering networks. IEEE Journal of Selected Topics in Signal Processing, Vol. 12, 6 (2018), 1601--1614.
[2]
Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. 2015. Diversity-induced multi-view subspace clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586--594.
[3]
Li Fei-Fei and Pietro Perona. 2005. A bayesian hierarchical model for learning natural scene categories. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 2. IEEE, 524--531.
[4]
Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. 2017. Improved deep embedded clustering with local structure preservation. In IJCAI. 1753--1759.
[5]
Zhenyu Huang, Joey Tianyi Zhou, Xi Peng, Changqing Zhang, Hongyuan Zhu, and Jiancheng Lv. 2019. Multi-view Spectral Clustering Network. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, 2563--2569.
[6]
Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. 2011. Multimodal Deep Learning. In ICML.
[7]
Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. 2016. Learning deep representations of fine-grained visual descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 49--58.
[8]
Uri Shaham, Kelly Stanton, Henry Li, Boaz Nadler, Ronen Basri, and Yuval Kluger. 2018. Spectralnet: Spectral clustering using deep neural networks. arXiv preprint arXiv:1801.01587 (2018).
[9]
Nitish Srivastava and Ruslan Salakhutdinov. 2012. Multimodal Learning with Deep Boltzmann Machines. J. Mach. Learn. Res., Vol. 15 (2012), 2949--2980.
[10]
Weiran Wang, Raman Arora, Karen Livescu, and Jeff A. Bilmes. 2015. On Deep Multi-View Representation Learning. In ICML.
[11]
Xiaobo Wang, Xiaojie Guo, Zhen Lei, Changqing Zhang, and Stan Z Li. 2017. Exclusivity-consistency regularized multi-view subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 923--931.
[12]
Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In International conference on machine learning. 478--487.
[13]
Handong Zhao, Zhengming Ding, and Yun Fu. 2017. Multi-view clustering via deep matrix factorization. In Thirty-First AAAI Conference on Artificial Intelligence.

Cited By

View all
  • (2024)Multi-modal data clustering using deep learning: A systematic reviewNeurocomputing10.1016/j.neucom.2024.128348(128348)Online publication date: Aug-2024
  • (2023)Mutual Information-Driven Multi-View ClusteringProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614986(3268-3277)Online publication date: 21-Oct-2023
  • (2023)On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clustering2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02296(23976-23985)Online publication date: Jun-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CIKM '20: Proceedings of the 29th ACM International Conference on Information & Knowledge Management
October 2020
3619 pages
ISBN:9781450368599
DOI:10.1145/3340531
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 October 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. consensus
  2. deep clustering
  3. diversity
  4. multimodal clustering

Qualifiers

  • Short-paper

Funding Sources

  • National Natural Science Foundation of China

Conference

CIKM '20
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)31
  • Downloads (Last 6 weeks)2
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Multi-modal data clustering using deep learning: A systematic reviewNeurocomputing10.1016/j.neucom.2024.128348(128348)Online publication date: Aug-2024
  • (2023)Mutual Information-Driven Multi-View ClusteringProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3614986(3268-3277)Online publication date: 21-Oct-2023
  • (2023)On the Effects of Self-supervision and Contrastive Alignment in Deep Multi-view Clustering2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.02296(23976-23985)Online publication date: Jun-2023
  • (2023)DaCFN: divide-and-conquer fusion network for RGB-T object detectionInternational Journal of Machine Learning and Cybernetics10.1007/s13042-022-01771-914:7(2407-2420)Online publication date: 11-Jan-2023
  • (2022)Gromov-Wasserstein Multi-modal Alignment and ClusteringProceedings of the 31st ACM International Conference on Information & Knowledge Management10.1145/3511808.3557339(603-613)Online publication date: 17-Oct-2022
  • (2022)Robust multi-view subspace clustering based on consensus representation and orthogonal diversityNeural Networks10.1016/j.neunet.2022.03.009150:C(102-111)Online publication date: 18-May-2022
  • (2021)CformerProceedings of the 30th ACM International Conference on Information & Knowledge Management10.1145/3459637.3482073(3078-3082)Online publication date: 26-Oct-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media