Scalable Multi-view Unsupervised Feature Selection with Structure Learning and Fusion
Authors: 
Chenglong Zhang, Xinyan Liang, Peng Zhou, Zhaolong Ling, Yingwei Zhang, Xingyu Wu, Weiguo Sheng, Bingbing JiangAuthors Info & Claims
Chenglong Zhang, Xinyan Liang, Peng Zhou, Zhaolong Ling, Yingwei Zhang, Xingyu Wu, Weiguo Sheng, Bingbing JiangAuthors Info & ClaimsPages 5479 - 5488
Abstract
To tackle the high-dimensional data with multiple representations, multi-view unsupervised feature selection has emerged as a significant learning paradigm. However, previous methods suffer from the following dilemmas: (i) They focus on selecting the features that preserve the similarity structure of data, whereas neglecting the discriminative information in the cluster structure; (ii) The orthogonal constraint is often imposed on the pseudo cluster labels, breaking the locality in the cluster label space; (iii) Learning the similarity or cluster structure from all samples is time-consuming. To this end, a Scalable Multi-view Unsupervised Feature Selection with structure learning and fusion (SMUFS) is proposed to jointly exploit the cluster structure and the similarity relations of data. Specifically, SMUFS introduces the sample-view weights to adaptively fuse the membership matrices that indicate cluster structures and serve as the pseudo cluster labels, such that a unified membership matrix across views can be effectively obtained to guide feature selection. Meanwhile, SMUFS performs graph learning from the membership matrix, preserving the locality of cluster labels and improving their discriminative capability. Further, an acceleration strategy has been developed to make SMUFS scalable for large-scale data. An iterative optimization is designed to solve the formulated objective function, and extensive experiments demonstrate the superiority of SMUFS.
References
[1]
Xiangpin Bai, Lei Zhu, Cheng Liang, Jingjing Li, Xiushan Nie, and Xiaojun Chang. 2020. Multi-view feature selection via nonnegative structured graph learning. Neurocomputing, Vol. 387 (2020), 110--122.
[2]
Jiangxia Cao, Xixun Lin, Shu Guo, Luchen Liu, Tingwen Liu, and Bin Wang. 2021. Bipartite graph embedding via mutual information maximization. In Proceedings of the ACM International Conference on Web Search and Data Mining. 635--643.
[3]
Jinrong Cui, Yuting Li, Yulu Fu, and Jie Wen. 2023. Multi-view Self-Expressive Subspace Clustering Network. In Proceedings of the ACM International Conference on Multimedia. 417--425.
[4]
Xiao Dong, Lei Zhu, Xuemeng Song, Jingjing Li, and Zhiyong Cheng. 2018. Adaptive collaborative similarity learning for unsupervised multi-view feature selection. In Proceedings of the International Joint Conference on Artificial Intelligence. 2064--2070.
[5]
Siguo Fang, Dong Huang, Changdong Wang, and Yong Tang. 2024. Joint multi-view unsupervised feature selection and graph learning. IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. 8, 1 (2024), 16--31.
[6]
Yang Fang, Bei Luo, Ting Zhao, Dong He, Bingbing Jiang, and Qilie Liu. 2022. ST-SIGMA: Spatio-temporal semantics and interaction graph aggregation for multi-agent perception and trajectory forecasting. CAAI Transactions on Intelligence Technology, Vol. 7, 4 (2022), 744--757.
[7]
Yang Fang, Bailian Xie, Uswah Khairuddin, Zijian Min, Bingbing Jiang, and Weisheng Li. 2024. DPT-tracker: Dual pooling transformer for efficient visual tracking. CAAI Transactions on Intelligence Technology (2024).
[8]
Gene H Golub and Charles F Van Loan. 2013. Matrix computations. JHU press.
[9]
Zhibin Gu, Songhe Feng, Ruiting Hu, and Gengyu Lyu. 2023. ONION: Joint unsupervised feature selection and robust subspace extraction for graph-based multi-view clustering. ACM Transactions on Knowledge Discovery from Data, Vol. 17, 5 (2023), 1--23.
[10]
Chenping Hou, Feiping Nie, Hong Tao, and Dongyun Yi. 2017. Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 9 (2017), 1998--2011.
[11]
Jin Huang, Feiping Nie, and Heng Huang. 2015. A new simplex sparse learning model to measure data similarity for clustering. In Proceedings of the International Joint Conference on Artificial Intelligence. 3569--3575.
[12]
Shudong Huang, Yixi Liu, Hecheng Cai, Yuze Tan, Chenwei Tang, and Jiancheng Lv. 2023. Smooth representation learning from multi-view data. Information Fusion, Vol. 100 (2023), 101916.
[13]
Shudong Huang, Yixi Liu, Yazhou Ren, Ivor W Tsang, Zenglin Xu, and Jiancheng Lv. 2022. Learning smooth representation for multi-view subspace clustering. In Proceedings of the ACM International Conference on Multimedia. 3421--3429.
[14]
Shudong Huang, Hongjie Wu, Yazhou Ren, Ivor Tsang, Zenglin Xu, Wentao Feng, and Jiancheng Lv. 2022. Multi-view subspace clustering on topological manifold. Advances in Neural Information Processing Systems, Vol. 35 (2022), 25883--25894.
[15]
Bingbing Jiang, Xingyu Wu, Kui Yu, and Huanhuan Chen. 2019. Joint semi-supervised feature selection and classification through Bayesian approach. In Proceedings of the AAAI Conference on Artificial Intelligence. 3983--3990.
[16]
Bingbing Jiang, Xingyu Wu, Xiren Zhou, Yi Liu, Anthony G Cohn, Weiguo Sheng, and Huanhuan Chen. 2024. Semi-supervised multiview feature selection with adaptive graph learning. IEEE Transactions on Neural Networks and Learning Systems, Vol. 35, 3 (2024), 3651--3629.
[17]
Bingbing Jiang, Junhao Xiang, Xingyu Wu, Yadi Wang, Huanhuan Chen, Weiwei Cao, and Weiguo Sheng. 2022. Robust multi-view learning via adaptive regression. Information Sciences, Vol. 610 (2022), 916--937.
[18]
Bingbing Jiang, Chenglong Zhang, Yan Zhong, Yi Liu, Yingwei Zhang, Xingyu Wu, and Weiguo Sheng. 2023. Adaptive collaborative fusion for multi-view semi-supervised classification. Information Fusion, Vol. 96 (2023), 37--50.
[19]
Zechao Li and Jinhui Tang. 2015. Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Transactions on Image Processing, Vol. 24, 12 (2015), 5343--5355.
[20]
Xinyan Liang, Pinhan Fu, Qian Guo, Keyin Zheng, and Yuhua Qian. 2024. DC-NAS: Divide-and-Conquer Neural Architecture Search for Multi-Modal Classification. In Proceedings of the AAAI Conference on Artificial Intelligence. 13754--13762.
[21]
Xinyan Liang, Yuhua Qian, Qian Guo, Honghong Cheng, and Jiye Liang. 2022. AF: An association-based fusion method for multi-modal classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 44, 12 (2022), 9236--9254.
[22]
Chengliang Liu, Zhihao Wu, Jie Wen, Yong Xu, and Chao Huang. 2022. Localized sparse incomplete multi-view clustering. IEEE Transactions on Multimedia, Vol. 25 (2022), 5539--5551.
[23]
Hongfu Liu, Haiyi Mao, and Yun Fu. 2016. Robust multi-view feature selection. In Proceedings of the IEEE International Conference on Data Mining. IEEE, 281--290.
[24]
Feiping Nie, Xia Dong, Lai Tian, Rong Wang, and Xuelong Li. 2020. Unsupervised feature selection with constrained l_2,0-Norm and optimized graph. IEEE Transactions on Neural Networks and Learning Systems, Vol. 33, 4 (2020), 1702--1713.
[25]
Feiping Nie, Wei Zhu, and Xuelong Li. 2016. Unsupervised feature selection with structured graph optimization. In Proceedings of the AAAI Conference on Artificial Intelligence. 1302--1308.
[26]
Dan Shi, Lei Zhu, Jingjing Li, Zheng Zhang, and Xiaojun Chang. 2023. Unsupervised adaptive feature selection with binary hashing. IEEE Transactions on Image Processing, Vol. 32 (2023), 838--853.
[27]
Zhaoyin Shi, Long Chen, Weiping Ding, Chuanbin Zhang, and Yingxu Wang. 2023. Parameter-free robust ensemble framework of fuzzy clustering. IEEE Transactions on Fuzzy Systems, Vol. 31, 12 (2023), 4205--4219.
[28]
Saúl Solorio-Fernández, J Ariel Carrasco-Ochoa, and José Fco Martínez-Trinidad. 2020. A review of unsupervised feature selection methods. Artificial Intelligence Review, Vol. 53, 2 (2020), 907--948.
[29]
Chuan Tang, Kun Sun, Chang Tang, Xiao Zheng, Xinwang Liu, Jun-Jie Huang, and Wei Zhang. 2023. Multi-view subspace clustering via adaptive graph learning and late fusion alignment. Neural Networks, Vol. 165 (2023), 333--343.
[30]
Chang Tang, Xiao Zheng, Wei Zhang, Xinwang Liu, Xinzhong Zhu, and En Zhu. 2023. Unsupervised feature selection via multiple graph fusion and feature weight learning. Science China Information Sciences, Vol. 66, 5 (2023), 152101.
[31]
Chang Tang, Xinzhong Zhu, Xinwang Liu, and Lizhe Wang. 2019. Cross-view local structure preserved diversity and consensus learning for multi-view unsupervised feature selection. In Proceedings of the AAAI Conference on Artificial Intelligence. 5101--5108.
[32]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, Vol. 9, 11 (2008), 2579--2605.
[33]
Xinhang Wan, Jiyuan Liu, Xinbiao Gan, Xinwang Liu, Siwei Wang, Yi Wen, Tianjiao Wan, and En Zhu. 2024. One-step multi-view clustering with diverse representation. IEEE Transactions on Neural Networks and Learning Systems (2024).
[34]
Qianqian Wang, Zhengming Ding, Zhiqiang Tao, Quanxue Gao, and Yun Fu. 2021. Generative partial multi-view clustering with adaptive fusion and cycle consistency. IEEE Transactions on Image Processing, Vol. 30 (2021), 1771--1783.
[35]
Qianqian Wang, Zhiqiang Tao, Quanxue Gao, and Licheng Jiao. 2022. Multi-view subspace clustering via structured multi-pathway network. IEEE Transactions on Neural Networks and Learning Systems, Vol. 35, 5 (2022), 7244--7250.
[36]
Qianqian Wang, Zhiqiang Tao, Wei Xia, Quanxue Gao, Xiaochun Cao, and Licheng Jiao. 2023. Adversarial multiview clustering networks with adaptive fusion. IEEE Transactions on Neural Networks and Learning Systems, Vol. 34, 10 (2023), 7635--7647.
[37]
Siwei Wang, Xinwang Liu, En Zhu, Chang Tang, Jiyuan Liu, Jingtao Hu, Jingyuan Xia, and Jianping Yin. 2019. Multi-view Clustering via Late Fusion Alignment Maximization. In Proceedings of the International Joint Conference on Artificial Intelligence. 3778--3784.
[38]
Yiming Wang, Dongxia Chang, Zhiqiang Fu, Jie Wen, and Yao Zhao. 2024. Partially View-aligned Representation Learning via Cross-view Graph Contrastive Network. IEEE Transactions on Circuits and Systems for Video Technology (2024).
[39]
Yi Wen, Siwei Wang, Ke Liang, Weixuan Liang, Xinhang Wan, Xinwang Liu, Suyuan Liu, Jiyuan Liu, and En Zhu. 2023. Scalable Incomplete Multi-View Clustering with Structure Alignment. In Proceedings of the ACM International Conference on Multimedia. 3031--3040.
[40]
Xingyu Wu, Bingbing Jiang, Xiangyu Wang, Taiyu Ban, and Huanhuan Chen. 2023. Feature selection in the data stream based on incremental markov boundary learning. IEEE Transactions on Neural Networks and Learning Systems, Vol. 34, 10 (2023), 6740--6754.
[41]
Xingyu Wu, Bingbing Jiang, Kui Yu, Huanhuan Chen, and Chunyan Miao. 2020. Multi-label causal feature selection. In Proceedings of the AAAI Conference on Artificial Intelligence. 6430--6437.
[42]
Cai Xu, Jiajun Si, Ziyu Guan, Wei Zhao, Yue Wu, and Xiyue Gao. 2024. Reliable conflictive multi-view learning. In Proceedings of the AAAI Conference on Artificial Intelligence. 16129--16137.
[43]
Yu-Meng Xu, Chang-Dong Wang, and Jian-Huang Lai. 2016. Weighted multi-view clustering with feature selection. Pattern Recognition, Vol. 53 (2016), 25--35.
[44]
Weiqing Yan, Jindong Xu, Jinglei Liu, Guanghui Yue, and Chang Tang. 2022. Bipartite graph-based discriminative feature learning for multi-view clustering. In Proceedings of the ACM International Conference on Multimedia. 3403--3411.
[45]
Mengbo You, Aihong Yuan, Min Zou, Dongjian He, and Xuelong Li. 2023. Robust unsupervised feature selection via multi-group adaptive graph representation. IEEE Transactions on Knowledge and Data Engineering, Vol. 35, 3 (2023), 3030--3044.
[46]
Chao Zhang, Wentao Fan, Bo Wang, Chunlin Chen, and Huaxiong Li. 2024. Self-paced semi-supervised feature selection with application to multi-modal Alzheimer's disease classification. Information Fusion, Vol. 107 (2024), 102345.
[47]
Chenglong Zhang, Yang Fang, Xinyan Liang, Han Zhang, Peng Zhou, Jie Wu, Xingyu andYang, Bingbing Jiang, and Weiguo Sheng. 2024. Efficient Multi-view Unsupervised Feature Selection with Adaptive Structure Learning and Inference. In Proceedings of the International Joint Conference on Artificial Intelligence. 5443--5452.
[48]
Chenglong Zhang, Bingbing Jiang, Zidong Wang, Jie Yang, Yangfeng Lu, Xingyu Wu, and Weiguo Sheng. 2023. Efficient multi-view semi-supervised feature selection. Information Sciences, Vol. 649 (2023), 119675.
[49]
Chen Zhang, Siwei Wang, Jiyuan Liu, Sihang Zhou, Pei Zhang, Xinwang Liu, En Zhu, and Changwang Zhang. 2021. Multi-view clustering via deep matrix factorization and partition alignment. In Proceedings of the ACM international Conference on Multimedia. 4156--4164.
[50]
Chenglong Zhang, Xinjie Zhu, Zidong Wang, Yan Zhong, Weiguo Sheng, Weiping Ding, and Bingbing Jiang. 2024 e. Discriminative Multi-View Fusion via Adaptive Regression. IEEE Transactions on Emerging Topics in Computational Intelligence (2024).
[51]
Han Zhang, Feiping Nie, and Xuelong Li. 2023. Large-scale clustering with structured optimal bipartite graph. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 8 (2023), 9950--9963.
[52]
Han Zhang, Danyang Wu, Feiping Nie, Rong Wang, and Xuelong Li. 2021. Multilevel projections with adaptive neighbor graph for unsupervised multi-view feature selection. Information Fusion, Vol. 70 (2021), 129--140.
[53]
Jia Zhang, Yidong Lin, Min Jiang, Shaozi Li, Yong Tang, Jinyi Long, Jian Weng, and Kay Chen Tan. 2024. Fast multilabel feature selection via global relevance and redundancy optimization. IEEE Transactions on Neural Networks and Learning Systems, Vol. 35, 4 (2024), 5721--5734.
[54]
Yue Zhang, Sirui Yang, Weitian Huang, Changdong Wang, and Hongmin Cai. 2024 d. Unified Representation Learning for Multi-View Clustering by Between/Within View Deep Majorization. IEEE Transactions on Emerging Topics in Computational Intelligence, Vol. 8 (2024), 615--626.
[55]
Peng Zhou, Jiangyong Chen, Liang Du, and Xuejun Li. 2023. Balanced spectral feature selection. IEEE Transactions on Cybernetics, Vol. 53, 7 (2023), 4232--4244.
[56]
Peng Zhou, Xia Wang, and Liang Du. 2023. Bi-level ensemble method for unsupervised feature selection. Information Fusion, Vol. 100 (2023), 101910.
Index Terms
- Scalable Multi-view Unsupervised Feature Selection with Structure Learning and Fusion
Recommendations
Collaborative and Discriminative Subspace Learning for unsupervised multi-view feature selection
AbstractBy effectively exploiting the consistent information of multi-view data, multi-view feature selection seeks to select crucial features from multiple heterogeneous data views to improve the clustering and classification accuracy of multi-view data ...
Consensus cluster structure guided multi-view unsupervised feature selection
AbstractAs the volume of high-dimensional multi-view data continues to grow, there has been a significant development in multi-view unsupervised feature selection methods, particularly those that perform graph learning and feature selection ...
Multi-level correlation learning for multi-view unsupervised feature selection
AbstractRecently, unsupervised multi-view feature selection has received a lot of interest. However, current methods facilitate feature selection by preserving the local consistency of multi-view data, which is defined based on similarities of data ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- The National Natural Science Foundation of China
- The ?Pioneer? and ?Lead- ing Goose? R&D Program of Zhejiang Province
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 177Total Downloads
- Downloads (Last 12 months)177
- Downloads (Last 6 weeks)87
Reflects downloads up to 03 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in