skip to main content
10.1145/3664647.3681600acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

EMVCC: Enhanced Multi-View Contrastive Clustering for Hyperspectral Images

Published: 28 October 2024 Publication History

Abstract

Cross-view consensus representation plays a critical role in hyperspectral images (HSIs) clustering. Recent multi-view contrastive cluster methods utilize contrastive loss to extract contextual consensus representation. However, these methods have a fatal flaw: contrastive learning may treat similar heterogeneous views as positive sample pairs and dissimilar homogeneous views as negative sample pairs. At the same time, the data representation via self-supervised contrastive loss is not specifically designed for clustering. Thus, to tackle this challenge, we propose a novel multi-view clustering method, i.e., Enhanced Multi-View Contrastive Clustering (EMVCC). First, the spatial multi-view is designed to learn the diverse features for contrastive clustering, and the globally relevant information of spectrum-view is extracted by Transformer, enhancing the spatial multi-view differences between neighboring samples. Then, a joint self-supervised loss is designed to constrain the consensus representation from different perspectives to efficiently avoid false negative pairs. Specifically, to preserve the diversity of multi-view information, the features are enhanced by using probabilistic contrastive loss, and the data is projected into a semantic representation space, ensuring that the similar samples in this space are closer in distance. Finally, we design a novel clustering loss that aligns the view feature representation with high confidence pseudo-labels for promoting the network to learn cluster-friendly features. In the training process, the joint self-supervised loss is used to optimize the cross-view features.Abundant experiment studies on numerous benchmarks verify the superiority of EMVCC in comparison to some state-of-the-art clustering methods. The codes are available at https://github.com/YiLiu1999/EMVCC.

References

[1]
Pranjal Awasthi, Nishanth Dikkala, and Pritish Kamath. 2022. Do More Negative Samples Necessarily Hurt In Contrastive Learning?. In Proceedings of the 39th International Conference on Machine Learning, Vol. 162. PMLR, 1101--1116.
[2]
Yaoming Cai, Zijia Zhang, Zhihua Cai, Xiaobo Liu, Xinwei Jiang, and Qin Yan. 2020. Graph convolutional subspace clustering: A robust subspace clustering framework for hyperspectral image. IEEE Transactions on Geoscience and Remote Sensing, Vol. 59, 5 (2020), 4191--4202.
[3]
Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. 2020. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, Vol. 33 (2020), 9912--9924.
[4]
Jianlong Chang, Gaofeng Meng, Lingfeng Wang, Shiming Xiang, and Chunhong Pan. 2018. Deep self-evolution clustering. IEEE transactions on pattern analysis and machine intelligence, Vol. 42, 4 (2018), 809--823.
[5]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597--1607.
[6]
Liang Du, Peng Zhou, Lei Shi, Hanmo Wang, Mingyu Fan, Wenjian Wang, and Yi-Dong Shen. 2015. Robust multiple kernel K-means using ell_2,1-norm. In Proceedings of the 24th International Conference on Artificial Intelligence (Buenos Aires, Argentina) (IJCAI'15). AAAI Press, 3476--3482.
[7]
Uno Fang, Jianxin Li, Naveed Akhtar, Man Li, and Yan Jia. 2023. Gomic: Multi-view image clustering via self-supervised contrastive heterogeneous graph co-learning. World Wide Web, Vol. 26, 4 (2023), 1667--1683.
[8]
Chuan Fu, Bo Du, and Liangpei Zhang. 2024. Do We Need Learnable Classifiers? A Hyperspectral Image Classification Algorithm Based on Attention-Enhanced ResBlock-in-ResBlock and ETF Classifier. IEEE Trans. Geosci. Remote. Sens., Vol. 62 (2024), 1--13. https://doi.org/10.1109/TGRS.2024.3353383
[9]
Chuan Fu, Bo Du, and Liangpei Zhang. 2024. ReSC-net: Hyperspectral Image Classification Based on Attention-Enhanced Residual Module and Spatial-Channel Attention. IEEE Trans. Geosci. Remote. Sens., Vol. 62 (2024), 1--15. https://doi.org/10.1109/TGRS.2024.3402364
[10]
Lianru Gao, Degang Wang, Lina Zhuang, Xu Sun, Min Huang, and Antonio Plaza. 2023. BS3LNet: A New Blind-Spot Self-Supervised Learning Network for Hyperspectral Anomaly Detection. IEEE Transactions on Geoscience and Remote Sensing, Vol. 61, 5504218 (2023), 1--18.
[11]
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. 2020. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, Vol. 33 (2020), 21271--21284.
[12]
Renxiang Guan, Zihao Li, Teng Li, Xianju Li, Jinzhong Yang, and Weitao Chen. 2022. Classification of heterogeneous mining areas based on rescapsnet and gaofen-5 imagery. Remote Sensing, Vol. 14, 13 (2022), 3216.
[13]
Renxiang Guan, Zihao Li, Xianju Li, and Chang Tang. 2024. Pixel-Superpixel Contrastive Learning and Pseudo-Label Correction for Hyperspectral Image Clustering. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 6795--6799.
[14]
Renxiang Guan, Zihao Li, Chujia Song, Guo Yu, Xianju Li, and Ruyi Feng. 2024. S2RC-GCN: A Spatial-Spectral Reliable Contrastive Graph Convolutional Network for Complex Land Cover Classification Using Hyperspectral Images. arXiv preprint arXiv:2404.00964 (2024).
[15]
Renxiang Guan, Zihao Li, Wenxuan Tu, Jun Wang, Yue Liu, Xianju Li, Chang Tang, and Ruyi Feng. 2024. Contrastive Multiview Subspace Clustering of Hyperspectral Images Based on Graph Convolutional Networks. IEEE Transactions on Geoscience and Remote Sensing, Vol. 62, 5510514 (2024), 1--14.
[16]
Kaveh Hassani and Amir Hosein Khasahmadi. 2020. Contrastive multi-view representation learning on graphs. In International conference on machine learning. PMLR, 4116--4126.
[17]
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9729--9738.
[18]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.
[19]
Shizhe Hu, Guoliang Zou, Chaoyang Zhang, Zhengzheng Lou, Ruilin Geng, and Yangdong Ye. 2023. Joint contrastive triple-learning for deep multi-view clustering. Information Processing & Management, Vol. 60, 3 (2023), 103284.
[20]
Xiang Hu, Teng Li, Tong Zhou, and Yuanxi Peng. 2021. Deep spatial-spectral subspace clustering for hyperspectral images based on contrastive learning. Remote Sensing, Vol. 13, 21 (2021), 4418.
[21]
Xuelong Li, Han Zhang, Rong Wang, and Feiping Nie. 2022. Multiview Clustering: A Scalable and Parameter-Free Bipartite Graph Fusion Method. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 1 (2022), 330--344.
[22]
Zhenglai Li, Chang Tang, Xinwang Liu, Xiao Zheng, Wei Zhang, and En Zhu. 2022. Consensus Graph Learning for Multi-View Clustering. IEEE Transactions on Multimedia, Vol. 24 (2022), 2461--2472.
[23]
Youwei Liang, Dong Huang, and Chang-Dong Wang. 2019. Consistency Meets Inconsistency: A Unified Graph Learning Framework for Multi-view Clustering. In 2019 IEEE International Conference on Data Mining (ICDM). 1204--1209.
[24]
Yijie Lin, Yuanbiao Gou, Zitao Liu, Boyun Li, Jiancheng Lv, and Xi Peng. 2021. COMPLETER: Incomplete Multi-View Clustering via Contrastive Prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11174--11183.
[25]
Jie Liu, Renxiang Guan, Zihao Li, Jiaxuan Zhang, Yaowen Hu, and Xueyong Wang. 2023. Adaptive Multi-Feature Fusion Graph Convolutional Network for Hyperspectral Image Classification. Remote Sensing, Vol. 15, 23 (2023), 5483.
[26]
Shujun Liu and Huajun Wang. 2022. Graph Convolutional Optimal Transport for Hyperspectral Image Spectral Clustering. IEEE Transactions on Geoscience and Remote Sensing, Vol. 60 (2022), 1--13.
[27]
Xinwang Liu, Yong Dou, Jianping Yin, Lei Wang, and En Zhu. 2016. Multiple kernel k-means clustering with matrix-induced regularization. In Proceedings of the AAAI conference on artificial intelligence, Vol. 30.
[28]
Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, and Xi Peng. 2024. Decoupled contrastive multi-view clustering with high-order random walks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 14193--14201.
[29]
Fulin Luo, Yi Liu, Yule Duan, Tan Guo, Lefei Zhang, and Bo Du. 2024. SDST: Self-Supervised Double-Structure Transformer for Hyperspectral Images Clustering. IEEE Transactions on Geoscience and Remote Sensing, Vol. 62 (2024), 1--14.
[30]
Fulin Luo, Tianyuan Zhou, Jiamin Liu, Tan Guo, Xiuwen Gong, and Xinbo Gao. 2024. DCENet: Diff-Feature Contrast Enhancement Network for Semi-Supervised Hyperspectral Change Detection. IEEE Transactions on Geoscience and Remote Sensing, Vol. 62 (2024), 1--14.
[31]
Shirui Luo, Changqing Zhang, Wei Zhang, and Xiaochun Cao. 2018. Consistent and specific multi-view subspace clustering. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32.
[32]
Feiping Nie, Jing Li, and Xuelong Li. 2016. Parameter-free auto-weighted multiple graph learning: a framework for multiview clustering and semi-supervised classification. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (New York, New York, USA) (IJCAI'16). AAAI Press, 1881--1887.
[33]
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018).
[34]
Erlin Pan and Zhao Kang. 2021. Multi-view contrastive graph clustering. Advances in neural information processing systems, Vol. 34 (2021), 2148--2159.
[35]
Craig Rodarmel and Jie Shan. 2002. Principal component analysis for hyperspectral image classification. Surveying and Land Information Science, Vol. 62, 2 (2002), 115--122.
[36]
Mengjing Sun, Pei Zhang, Siwei Wang, Sihang Zhou, Wenxuan Tu, Xinwang Liu, En Zhu, and Changjian Wang. 2021. Scalable multi-view subspace clustering with unified anchors. In Proceedings of the 29th ACM International Conference on Multimedia. 3528--3536.
[37]
Long Tian, Qian Du, Ivica Kopriva, and Nicolas Younan. 2018. Spatial-spectral Based Multi-view Low-rank Sparse Sbuspace Clustering for Hyperspectral Imagery. In IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium. 8488--8491.
[38]
Yonglong Tian, Dilip Krishnan, and Phillip Isola. 2020. Contrastive multiview coding. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XI 16. Springer, 776--794.
[39]
Daniel J. Trosten, Sigurd Løkse, Robert Jenssen, and Michael C. Kampffmeyer. 2023. On the Effects of Self-Supervision and Contrastive Alignment in Deep Multi-View Clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 23976--23985.
[40]
Bing Tu, Wangquan He, Qianming Li, Yishu Peng, and Antonio Plaza. 2023. A New Context-Aware Framework for Defending Against Adversarial Attacks in Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, Vol. 61, 5505114 (2023), 1--14.
[41]
Hsiao-Yu Tung, Hsiao-Wei Tung, Ersin Yumer, and Katerina Fragkiadaki. 2017. Self-supervised learning of motion capture. Advances in neural information processing systems, Vol. 30 (2017).
[42]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, 11 (2008).
[43]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, Vol. 30 (2017).
[44]
Siwei Wang, Xinwang Liu, Xinzhong Zhu, Pei Zhang, Yi Zhang, Feng Gao, and En Zhu. 2022. Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance. IEEE Transactions on Image Processing, Vol. 31 (2022), 556--568.
[45]
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. 2022. Multi-Level Feature Learning for Contrastive Multi-View Clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16051--16060.
[46]
Jie Xu, Huayi Tang, Yazhou Ren, Liang Peng, Xiaofeng Zhu, and Lifang He. 2022. Multi-level Feature Learning for Contrastive Multi-view Clustering. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16030--16039.
[47]
Weiqing Yan, Yuanyang Zhang, Chenlei Lv, Chang Tang, Guanghui Yue, Liang Liao, and Weisi Lin. 2023. GCFAgg: Global and Cross-View Feature Aggregation for Multi-View Clustering. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 19863--19872.
[48]
Weiqing Yan, Yuanyang Zhang, Chang Tang, Wujie Zhou, and Weisi Lin. 2024. Anchor-Sharing and Clusterwise Contrastive Network for Multiview Representation Learning. IEEE Transactions on Neural Networks and Learning Systems (2024), 1--11.
[49]
Gang Yang, Ke Huang, Weiwei Sun, Xiangchao Meng, Dehua Mao, and Yong Ge. 2022. Enhanced mangrove vegetation index based on hyperspectral images for mapping mangrove. ISPRS Journal of Photogrammetry and Remote Sensing, Vol. 189 (2022), 236--254.
[50]
Xihong Yang, Jin Jiaqi, Siwei Wang, Ke Liang, Yue Liu, Yi Wen, Suyuan Liu, Sihang Zhou, Xinwang Liu, and En Zhu. 2023. Dealmvc: Dual contrastive calibration for multi-view clustering. In Proceedings of the 31st ACM International Conference on Multimedia. 337--346.
[51]
Ming Yin, Weitian Huang, and Junbin Gao. 2020. Shared generative latent representation learning for multi-view clustering. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 6688--6695.
[52]
Changqing Zhang, Qinghua Hu, Huazhu Fu, Pengfei Zhu, and Xiaochun Cao. 2017. Latent Multi-view Subspace Clustering. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 4333--4341.
[53]
Changqing Zhang, Yeqing Liu, and Huazhu Fu. 2019. Ae2-nets: Autoencoder in autoencoder networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2577--2585.
[54]
Hongming Zhang, Liwei Qiu, Lingling Yi, and Yangqiu Song. 2018. Scalable multiplex network embedding. In IJCAI, Vol. 18. 3082--3088.
[55]
Yang Zhao and Xuelong Li. 2023. Deep Spectral Clustering With Regularized Linear Embedding for Hyperspectral Image Clustering. IEEE Transactions on Geoscience and Remote Sensing, Vol. 61 (2023), 1--11.

Cited By

View all
  • (2025)STMNet: Single-Temporal Mask-Based Network for Self-Supervised Hyperspectral Change DetectionIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.352354163(1-12)Online publication date: 2025
  • (2024)Spatial–Spectral Graph Contrastive Clustering With Hard Sample Mining for Hyperspectral ImagesIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.346464862(1-16)Online publication date: 2024

Index Terms

  1. EMVCC: Enhanced Multi-View Contrastive Clustering for Hyperspectral Images

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '24: Proceedings of the 32nd ACM International Conference on Multimedia
    October 2024
    11719 pages
    ISBN:9798400706868
    DOI:10.1145/3664647
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 October 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. contrastive learning
    2. hyperspectral images (hsis)
    3. multi-view clustering
    4. self-supervised learning

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China

    Conference

    MM '24
    Sponsor:
    MM '24: The 32nd ACM International Conference on Multimedia
    October 28 - November 1, 2024
    Melbourne VIC, Australia

    Acceptance Rates

    MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)107
    • Downloads (Last 6 weeks)25
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)STMNet: Single-Temporal Mask-Based Network for Self-Supervised Hyperspectral Change DetectionIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.352354163(1-12)Online publication date: 2025
    • (2024)Spatial–Spectral Graph Contrastive Clustering With Hard Sample Mining for Hyperspectral ImagesIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.346464862(1-16)Online publication date: 2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media