Federated Fuzzy C-means with Schatten-p Norm Minimization
Pages 9407 - 9416
Abstract
Multi-view clustering has emerged as an important unsupervised method to process unlabelled multi-view data that provides a comprehensive description of an object. Existing multi-view clustering methods focus on centralized settings but ignore the fact that real-world multi-view data may be distributed across different entities. The sensitive information embedded in multi-view data hinders the cooperative training of multi-view clustering, since data of different views cannot be directly shared, leading to a great challenge to cooperatively exploit the consistent and complementary information of different views. To validate the multi-view clustering in distributed scenarios, in this paper, we propose a novel federated multi-view method named Federated Multi-View Fuzzy C-means with Schatten-p Norm Minimization (FMVFCMSP) which is based on fuzzy C-means and tensor Schatten p-norm. Specifically, we utilize the membership degrees to replace conventional hard clustering assignment in K-means, enabling improved uncertainty handling and less information loss. Moreover, we introduce a tensor Schatten p-norm-based regularizer to fully explore the inter-view complementary information and global spatial structure. We also develop a federated optimization algorithm enabling clients to collaboratively learn the clustering results. Extensive experiments on several datasets demonstrate that our proposed method exhibits superior performance in federated multi-view clustering.
References
[1]
James C Bezdek, Robert Ehrlich, and William Full. 1984. FCM: The fuzzy c-means clustering algorithm. Computers & geosciences, Vol. 10, 2--3 (1984), 191--203.
[2]
Maria Brbić and Ivica Kopriva. 2018. Multi-view low-rank sparse subspace clustering. Pattern Recognition, Vol. 73 (2018), 247--258.
[3]
Xiaochun Cao, Changqing Zhang, Huazhu Fu, Si Liu, and Hua Zhang. 2015. Diversity-induced multi-view subspace clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586--594.
[4]
Sicong Che, Zhaoming Kong, Hao Peng, Lichao Sun, Alex Leow, Yong Chen, and Lifang He. 2022. Federated multi-view learning for private medical data integration and analysis. ACM Transactions on Intelligent Systems and Technology (TIST), Vol. 13, 4 (2022), 1--23.
[5]
Man-Sheng Chen, Ling Huang, Chang-Dong Wang, and Dong Huang. 2020. Multi-view clustering in latent embedding space. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 3513--3520.
[6]
Tianyi Chen, Xiao Jin, Yuejiao Sun, and Wotao Yin. 2020. Vafl: a method of vertical asynchronous federated learning. arXiv preprint arXiv:2007.06081 (2020).
[7]
Wenlan Chen, Hong Wang, and Cheng Liang. 2023. Deep multi-view contrastive learning for cancer subtype identification. Briefings in Bioinformatics, Vol. 24, 5 (2023), bbad282.
[8]
Xinyue Chen, Jie Xu, Yazhou Ren, Xiaorong Pu, Ce Zhu, Xiaofeng Zhu, Zhifeng Hao, and Lifang He. 2023. Federated Deep Multi-View Clustering with Global Self-Supervision. In Proceedings of the 31st ACM International Conference on Multimedia. 3498--3506.
[9]
Chenhang Cui, Yazhou Ren, Jingyu Pu, Jiawei Li, Xiaorong Pu, Tianyi Wu, Yutao Shi, and Lifang He. 2024. A novel approach for effective multi-view clustering with information-theoretic perspective. Advances in Neural Information Processing Systems, Vol. 36 (2024).
[10]
Marco F Duarte and Yu Hen Hu. 2004. Vehicle classification in distributed sensor networks. J. Parallel and Distrib. Comput., Vol. 64, 7 (2004), 826--838.
[11]
Ali Mamdouh Elkahky, Yang Song, and Xiaodong He. 2015. A multi-view deep learning approach for cross domain user modeling in recommendation systems. In Proceedings of the 24th international conference on world wide web. 278--288.
[12]
Si-Guo Fang, Dong Huang, Xiao-Sha Cai, Chang-Dong Wang, Chaobo He, and Yong Tang. 2023. Efficient multi-view clustering via unified and discrete bipartite graph learning. IEEE Transactions on Neural Networks and Learning Systems (2023).
[13]
Uno Fang, Man Li, Jianxin Li, Longxiang Gao, Tao Jia, and Yanchun Zhang. 2023. A Comprehensive Survey on Multi-view Clustering. IEEE Transactions on Knowledge and Data Engineering (2023).
[14]
Li Fei-Fei, Rob Fergus, and Pietro Perona. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop. IEEE, 178--178.
[15]
Dashan Gao, Ce Ju, Xiguang Wei, Yang Liu, Tianjian Chen, and Qiang Yang. 2019. Hhhfl: Hierarchical heterogeneous horizontal federated learning for electroencephalography. arXiv preprint arXiv:1909.05784 (2019).
[16]
Quanxue Gao, Pu Zhang, Wei Xia, Deyan Xie, Xinbo Gao, and Dacheng Tao. 2020. Enhanced tensor RPCA and its application. IEEE transactions on pattern analysis and machine intelligence, Vol. 43, 6 (2020), 2133--2140.
[17]
Derek Greene and Pádraig Cunningham. 2006. Practical solutions to the problem of diagonal dominance in kernel document clustering. In Proceedings of the 23rd international conference on Machine learning. 377--384.
[18]
Xingchen Hu, Jindong Qin, Yinghua Shen, Witold Pedrycz, Xinwang Liu, and Jiyuan Liu. 2023. An Efficient Federated Multi-view Fuzzy C-Means Clustering Method. IEEE Transactions on Fuzzy Systems (2023).
[19]
Dong Huang, Chang-Dong Wang, and Jian-Huang Lai. 2023. Fast multi-view clustering via ensembles: Towards scalability, superiority, and simplicity. IEEE Transactions on Knowledge and Data Engineering (2023).
[20]
Shudong Huang, Wei Shi, Zenglin Xu, Ivor W Tsang, and Jiancheng Lv. 2022. Efficient federated multi-view learning. Pattern Recognition, Vol. 131 (2022), 108817.
[21]
Shudong Huang, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. 2021. Measuring diversity in graph learning: A unified framework for structured multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 34, 12 (2021), 5869--5883.
[22]
Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, and Yong Zhang. 2021. Personalized cross-silo federated learning on non-iid data. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35. 7865--7873.
[23]
Yi Jin, Chenning Li, Yidong Li, Peixi Peng, and George A Giannopoulos. 2021. Model latent views with multi-center metric learning for vehicle re-identification. IEEE Transactions on Intelligent Transportation Systems, Vol. 22, 3 (2021), 1919--1931.
[24]
Hao Kong, Xingyu Xie, and Zhouchen Lin. 2018. t-Schatten- p norm for low-rank tensor recovery. IEEE Journal of Selected Topics in Signal Processing, Vol. 12, 6 (2018), 1405--1419.
[25]
Jing Li, Quanxue Gao, Qianqian Wang, Ming Yang, and Wei Xia. 2024. Orthogonal non-negative tensor factorization based multi-view clustering. Advances in Neural Information Processing Systems, Vol. 36 (2024).
[26]
Jiangmeng Li, Wenwen Qiang, Changwen Zheng, and Bing Su. 2022. RHMC: Modeling consistent information from deep multiple views via Regularized and Hybrid Multiview Coding. Knowledge-Based Systems, Vol. 241 (2022), 108201.
[27]
Jianqiang Li, Guoxu Zhou, Yuning Qiu, Yanjiao Wang, Yu Zhang, and Shengli Xie. 2020. Deep graph regularized non-negative matrix factorization for multi-view clustering. Neurocomputing, Vol. 390 (2020), 108--116.
[28]
Ruihuang Li, Changqing Zhang, Huazhu Fu, Xi Peng, Tianyi Zhou, and Qinghua Hu. 2019. Reciprocal multi-layer subspace learning for multi-view clustering. In Proceedings of the IEEE/CVF international conference on computer vision. 8172--8180.
[29]
Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, Vol. 2 (2020), 429--450.
[30]
Xuelong Li, Han Zhang, Rong Wang, and Feiping Nie. 2020. Multiview clustering: A scalable and parameter-free bipartite graph fusion method. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 1 (2020), 330--344.
[31]
Naiyao Liang, Zuyuan Yang, Zhenni Li, Weijun Sun, and Shengli Xie. 2020. Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints. Knowledge-Based Systems, Vol. 194 (2020), 105582.
[32]
Jialu Liu, Chi Wang, Jing Gao, and Jiawei Han. 2013. Multi-view clustering via joint nonnegative matrix factorization. In Proceedings of the 2013 SIAM international conference on data mining. SIAM, 252--260.
[33]
Yang Liu, Yan Kang, Chaoping Xing, Tianjian Chen, and Qiang Yang. 2020. A secure federated transfer learning framework. IEEE Intelligent Systems, Vol. 35, 4 (2020), 70--82.
[34]
Yongli Liu, Xiaoqin Zhang, Guiying Tang, and Di Wang. 2019. Multi-view subspace clustering based on tensor schatten-p norm. In 2019 IEEE International Conference on Big Data (Big Data). IEEE, 5048--5055.
[35]
Han Lu, Quanxue Gao, Qianqian Wang, Ming Yang, and Wei Xia. 2023. Centerless multi-view K-means based on the adjacency matrix. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 8949--8956.
[36]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR, 1273--1282.
[37]
Xi Peng, Zhenyu Huang, Jiancheng Lv, Hongyuan Zhu, and Joey Tianyi Zhou. 2019. COMIC: Multi-view clustering without parameter selection. In International conference on machine learning. PMLR, 5092--5101.
[38]
Anguita Davide Ghio Alessandro Oneto Luca Reyes-Ortiz, Jorge and Xavier Parra. 2012. Human Activity Recognition Using Smartphones. UCI Machine Learning Repository.
[39]
Terry Sejnowski and R. Gorman. [n.,d.]. Connectionist Bench (Sonar, Mines vs. Rocks). UCI Machine Learning Repository.
[40]
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. 2012. Indoor segmentation and support inference from rgbd images. In Computer Vision--ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7--13, 2012, Proceedings, Part V 12. Springer, 746--760.
[41]
Jiankai Sun, Xin Yang, Yuanshun Yao, Aonan Zhang, Weihao Gao, Junyuan Xie, and Chong Wang. 2021. Vertical federated learning without revealing intersection membership. arXiv preprint arXiv:2106.05508 (2021).
[42]
Daniel J Trosten, Sigurd Lokse, Robert Jenssen, and Michael Kampffmeyer. 2021. Reconsidering representation alignment for multi-view clustering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1255--1265.
[43]
Huibing Wang, Guangqi Jiang, Jinjia Peng, Ruoxi Deng, and Xianping Fu. 2022. Towards adaptive consensus graph: multi-view clustering via graph collaboration. IEEE Transactions on Multimedia (2022).
[44]
Hao Wang, Yan Yang, and Bing Liu. 2019. GMC: Graph-based multi-view clustering. IEEE Transactions on Knowledge and Data Engineering, Vol. 32, 6 (2019), 1116--1129.
[45]
Jie Wen, Chengliang Liu, Shijie Deng, Yicheng Liu, Lunke Fei, Ke Yan, and Yong Xu. 2023. Deep double incomplete multi-view multi-label learning with incomplete labels and missing views. IEEE transactions on neural networks and learning systems (2023).
[46]
Jie Xu, Yazhou Ren, Guofeng Li, Lili Pan, Ce Zhu, and Zenglin Xu. 2021. Deep embedded multi-view clustering with collaborative training. Information Sciences, Vol. 573 (2021), 279--290.
[47]
Yaming Yang, Ziyu Guan, Jianxin Li, Wei Zhao, Jiangtao Cui, and Quan Wang. 2021. Interpretable and efficient heterogeneous graph convolutional network. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 2 (2021), 1637--1650.
[48]
Yaming Yang, Ziyu Guan, Wei Zhao, Weigang Lu, and Bo Zong. 2022. Graph substructure assembling network with soft sequence and context attention. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 5 (2022), 4894--4907.
[49]
Changqing Zhang, Huazhu Fu, Qinghua Hu, Xiaochun Cao, Yuan Xie, Dacheng Tao, and Dong Xu. 2018. Generalized latent multi-view subspace clustering. IEEE transactions on pattern analysis and machine intelligence, Vol. 42, 1 (2018), 86--99.
[50]
Chenglong Zhang, Xinjie Zhu, Zidong Wang, Yan Zhong, Weiguo Sheng, Weiping Ding, and Bingbing Jiang. 2024. Discriminative Multi-View Fusion via Adaptive Regression. IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. early access (2024), 1--13. https://doi.org/10.1109/TETCI.2024.3375342
[51]
Xiaoqian Zhang, Huaijiang Sun, Zhigui Liu, Zhenwen Ren, Qiongjie Cui, and Yanmeng Li. 2019. Robust low-rank kernel multi-view subspace clustering based on the schatten p-norm and correntropy. Information Sciences, Vol. 477 (2019), 430--447.
[52]
Jie Zhao, Xinghua Zhu, Jianzong Wang, and Jing Xiao. 2021. Efficient client contribution evaluation for horizontal federated learning. In ICASSP 2021--2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3060--3064.
[53]
Yujiao Zhao, Yu Yun, Xiangdong Zhang, Qin Li, and Quanxue Gao. 2022. Multi-view spectral clustering with adaptive graph learning and tensor schatten p-norm. Neurocomputing, Vol. 468 (2022), 257--264.
Index Terms
- Federated Fuzzy C-means with Schatten-p Norm Minimization
Recommendations
Image compressive sensing via Truncated Schatten-p Norm regularization
Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Self-weighted graph learning for multi-view clustering
AbstractThe graph-based multi-view clustering has received extensive attention in recent years due to its competitiveness in characterizing the relationship between data and its well defined mathematic. However, the existing graph-based ...
Bi-Nuclear Tensor Schatten-
p Norm Minimization for Multi-View Subspace ClusteringMulti-view subspace clustering aims to integrate the complementary information contained in different views to facilitate data representation. Currently, low-rank representation (LRR) serves as a benchmark method. However, we observe that these LRR-based ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 160Total Downloads
- Downloads (Last 12 months)160
- Downloads (Last 6 weeks)87
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in