skip to main content
research-article

Multi-Label Feature Selection Via Adaptive Label Correlation Estimation

Published: 10 August 2023 Publication History

Abstract

In multi-label learning, each instance is associated with multiple labels simultaneously. Multi-label data often have noisy, irrelevant, and redundant features of high dimensionality. Multi-label feature selection has received considerable attention as an effective means for dealing with high-dimensional multi-label data. Many multi-label feature selection methods exploit label correlations to help select features. However, finding label correlations and selecting features in existing multi-label feature selection methods are often two separate processes, the existence of noises and outliers in training data makes the label correlations exploited from label space less reliable. Therefore, the learned label correlations may mislead the feature selection process and result in the selection of less informative features. This article proposes a novel algorithm named ROAD, i.e., multi-label featuRe selectiOn via ADaptive label correlation estimation. ROAD jointly performs adaptive label correlation exploration and feature selection with alternating optimization to obtain reliable estimation of label correlations, which can more effectively reveal the intrinsic manifold structure among labels and lead to the selection of a more proper feature subset. Comprehensive experiments on several frequently used datasets validate the superiority of ROAD against the state-of-the-art multi-label feature selection algorithms.

References

[1]
Richard H. Bartels and George W. Stewart. 1972. Solution of the matrix equation AX + XB = C [F4]. Communications of the ACM 15, 9 (1972), 820–826.
[2]
Hamid Bayati, Mohammad Bagher Dowlatshahi, and Amin Hashemi. 2022. MSSL: A memetic-based sparse subspace learning algorithm for multi-label classification. International Journal of Machine Learning and Cybernetics 13, 11 (2022), 3607–3624.
[3]
Stephen Boyd, Stephen P. Boyd, and Lieven Vandenberghe. 2004. Convex Optimization. Cambridge University Press.
[4]
Zhiling Cai and William Zhu. 2018. Multi-label feature selection via feature manifold learning and sparsity regularization. International Journal of Machine Learning and Cybernetics 9, 8 (2018), 1321–1334.
[5]
Xiaoya Che, Degang Chen, and Jusheng Mi. 2020. A novel approach for learning label correlation with application to feature selection of multi-label data. Information Sciences 512 (2020), 795–812.
[6]
Janez Demšar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, Jan (2006), 1–30.
[7]
Olive Jean Dunn. 1961. Multiple comparisons among means. Journal of the American Statistical Association 56, 293 (1961), 52–64.
[8]
Yuling Fan, Baihua Chen, Weiqin Huang, Jinghua Liu, Wei Weng, and Weiyao Lan. 2022. Multi-label feature selection based on label correlations and feature redundancy. Knowledge-Based Systems 241 (2022), 108256.
[9]
Amin Hashemi, Mohammad Bagher Dowlatshahi, and Hossein Nezamabadi-Pour. 2020. MFS-MCDM: Multi-label feature selection using multi-criteria decision making. Knowledge-Based Systems 206 (2020), 106365.
[10]
Amin Hashemi, Mohammad Bagher Dowlatshahi, and Hossein Nezamabadi-pour. 2021. An efficient Pareto-based feature selection algorithm for multi-label classification. Information Sciences 581 (2021), 428–447.
[11]
Liang Hu, Lingbo Gao, Yonghao Li, Ping Zhang, and Wanfu Gao. 2022. Feature-specific mutual information variation for multi-label feature selection. Information Sciences 593 (2022), 449–471.
[12]
Ying Hu, Yong Zhang, and Dunwei Gong. 2020. Multiobjective particle swarm optimization for feature selection with fuzzy cost. IEEE Transactions on Cybernetics 51, 2 (2020), 874–888.
[13]
Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2015. Learning label specific features for multi-label classification. In Proceedings of the 2015 IEEE International Conference on Data Mining. 181–190.
[14]
Jun Huang, Guorong Li, Qingming Huang, and Xindong Wu. 2016. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering 28, 12 (2016), 3309–3323.
[15]
Rui Huang, Weidong Jiang, and Guangling Sun. 2018. Manifold-based constraint Laplacian score for multi-label feature selection. Pattern Recognition Letters 112 (2018), 346–352.
[16]
Ling Jian, Jundong Li, Kai Shu, and Huan Liu. 2016. Multi-label informed feature selection. In Proceedings of the International Joint Conference on Artificial Intelligence. 1627–1633.
[17]
Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P Trevino, Jiliang Tang, and Huan Liu. 2017. Feature selection: A data perspective. Computing Surveys 50, 6 (2017), 1–45.
[18]
Yonghao Li, Liang Hu, and Wanfu Gao. 2022. Label correlations variation for robust multi-label feature selection. Information Sciences 609 (2022), 1075–1097.
[19]
Yonghao Li, Liang Hu, and Wanfu Gao. 2023. Multi-label feature selection via robust flexible sparse regularization. Pattern Recognition 134 (2023), 109074.
[20]
Yaojin Lin, Qinghua Hu, Jinghua Liu, and Jie Duan. 2015. Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168 (2015), 92–103.
[21]
Yaojin Lin, Qinghua Hu, Jinghua Liu, Jinjin Li, and Xindong Wu. 2017. Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Transactions on Fuzzy Systems 25, 6 (2017), 1491–1507.
[22]
Yaojin Lin, Qinghua Hu, Jinghua Liu, Xingquan Zhu, and Xindong Wu. 2021. MULFE: Multi-label learning via label-specific feature space ensemble. ACM Transactions on Knowledge Discovery from Data 16, 1, Article 5 (2021), 24 pages.
[23]
Jinghua Liu, Yaojin Lin, Weiping Ding, Hongbo Zhang, and Jixiang Du. 2023. Fuzzy mutual information-based multilabel feature selection with label dependency and streaming labels. IEEE Transactions on Fuzzy Systems 31, 1 (2023), 77–91.
[24]
Yang Liu, Kaiwen Wen, Quanxue Gao, Xinbo Gao, and Feiping Nie. 2018. SVM based multi-label learning with missing labels for image annotation. Pattern Recognition 78 (2018), 307–317.
[25]
Mohsen Miri, Mohammad Bagher Dowlatshahi, Amin Hashemi, Marjan Kuchaki Rafsanjani, Brij B. Gupta, and W. Alhalabi. 2022. Ensemble feature selection for multi-label text classification: An intelligent order statistics approach. International Journal of Intelligent Systems 37, 12 (2022), 11319–11341.
[26]
Feiping Nie, Heng Huang, Xiao Cai, and Chris Ding. 2010. Efficient and robust feature selection via joint \(\mathbf {\ell _{2,1}}\)-norms minimization. Advances in Neural Information Processing Systems 23 (2010), 1813–1821.
[27]
Feiping Nie, Xiaoqian Wang, and Heng Huang. 2014. Clustering and projected clustering with adaptive neighbors. In Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 977–986.
[28]
Mohsen Paniri, Mohammad Bagher Dowlatshahi, and Hossein Nezamabadi-Pour. 2020. MLACO: A multi-label feature selection algorithm based on ant colony optimization. Knowledge-Based Systems 192 (2020), 105285.
[29]
Xian-Fang Song, Yong Zhang, Dun-Wei Gong, and Xiao-Zhi Gao. 2021. A fast hybrid feature selection based on correlation-guided clustering and particle swarm optimization for high-dimensional data. IEEE Transactions on Cybernetics 52, 9 (2021), 9573–9586.
[30]
James Joseph Sylvester. 1884. Sur l’équation en matrices px = xq. Comptes Rendus de l’Académie des Sciences 99, 2 (1884), 67–71.
[31]
Lichen Wang, Zhengming Ding, Seungju Han, Jae-Joon Han, Changkyu Choi, and Yun Fu. 2019. Generative correlation discovery network for multi-label learning. In Proceedings of the 2019 IEEE International Conference on Data Mining. 588–597.
[32]
Tong Wei and Yu-Feng Li. 2019. Learning compact model for large-scale multi-label data. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 5385–5392.
[33]
Xi-Zhu Wu and Zhi-Hua Zhou. 2017. A unified view of multi-label performance measures. In Proceedings of the International Conference on Machine Learning. 3780–3788.
[34]
Bing Xue, Mengjie Zhang, Will N. Browne, and Xin Yao. 2015. A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20, 4 (2015), 606–626.
[35]
Dianlong You, Yang Wang, Jiawei Xiao, Yaojin Lin, Maosheng Pan, Zhen Chen, Limin Shen, and Xindong Wu. 2023. Online multi-label streaming feature selection with label correlation. IEEE Transactions on Knowledge and Data Engineering 35, 3 (2023), 2901–2915.
[36]
Kui Yu, Yajing Yang, and Wei Ding. 2022. Causal feature selection with missing data. ACM Transactions on Knowledge Discovery from Data 16, 4, Article 66 (2022), 24 pages.
[37]
Ying Yu, Witold Pedrycz, and Duoqian Miao. 2014. Multi-label classification by exploiting label correlations. Expert Systems with Applications 41, 6 (2014), 2989–3004.
[38]
Jia Zhang, Yidong Lin, Min Jiang, Shaozi Li, Yong Tang, and Kay Chen Tan. 2020. Multi-label feature selection via global relevance and redundancy optimization. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. 2512–2518.
[39]
Jia Zhang, Zhiming Luo, Candong Li, Changen Zhou, and Shaozi Li. 2019. Manifold regularized discriminative feature selection for multi-label learning. Pattern Recognition 95 (2019), 136–150.
[40]
Min-Ling Zhang, Jun-Peng Fang, and Yi-Bo Wang. 2021. BiLabel-specific features for multi-label classification. ACM Transactions on Knowledge Discovery from Data 16, 1, Article 18 (2021), 23 pages.
[41]
Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern Recognition 40, 7 (2007), 2038–2048.
[42]
Min-Ling Zhang and Zhi-Hua Zhou. 2013. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2013), 1819–1837.
[43]
Zan Zhang, Lin Liu, Jiuyong Li, and Xindong Wu. 2023. Integrating global and local feature selection for multi-label learning. ACM Transactions on Knowledge Discovery from Data 17, 1 (2023), 1–37.
[44]
Pengfei Zhu, Qian Xu, Qinghua Hu, Changqing Zhang, and Hong Zhao. 2018. Multi-label feature selection with missing labels. Pattern Recognition 74 (2018), 488–502.
[45]
Yue Zhu, James T. Kwok, and Zhi-Hua Zhou. 2017. Multi-label learning with global and local label correlation. IEEE Transactions on Knowledge and Data Engineering 30, 6 (2017), 1081–1094.

Cited By

View all
  • (2025)A Novel Compound Fault Diagnosis Method for Rotating Machinery Based on Dynamic Adaptive MWPE and Dual-Graph Regularization StrategyIEEE Sensors Journal10.1109/JSEN.2024.352332325:4(6850-6868)Online publication date: 15-Feb-2025
  • (2025)Source Code Error Understanding Using BERT for Multi-Label ClassificationIEEE Access10.1109/ACCESS.2024.352506113(3802-3822)Online publication date: 2025
  • (2025)Local Clustering for Functional DataJournal of Computational and Graphical Statistics10.1080/10618600.2024.2431057(1-16)Online publication date: 10-Feb-2025
  • Show More Cited By

Index Terms

  1. Multi-Label Feature Selection Via Adaptive Label Correlation Estimation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Knowledge Discovery from Data
    ACM Transactions on Knowledge Discovery from Data  Volume 17, Issue 9
    November 2023
    373 pages
    ISSN:1556-4681
    EISSN:1556-472X
    DOI:10.1145/3604532
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 10 August 2023
    Online AM: 10 June 2023
    Accepted: 30 May 2023
    Revised: 05 May 2023
    Received: 31 January 2023
    Published in TKDD Volume 17, Issue 9

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Feature selection
    2. multi-label learning
    3. adaptive label correlation estimation

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Program for Changjiang Scholars and Innovative Research Team in University (PCSIRT) of the Ministry of Education of China
    • Fundamental Research Funds for the Central Universities

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)263
    • Downloads (Last 6 weeks)22
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)A Novel Compound Fault Diagnosis Method for Rotating Machinery Based on Dynamic Adaptive MWPE and Dual-Graph Regularization StrategyIEEE Sensors Journal10.1109/JSEN.2024.352332325:4(6850-6868)Online publication date: 15-Feb-2025
    • (2025)Source Code Error Understanding Using BERT for Multi-Label ClassificationIEEE Access10.1109/ACCESS.2024.352506113(3802-3822)Online publication date: 2025
    • (2025)Local Clustering for Functional DataJournal of Computational and Graphical Statistics10.1080/10618600.2024.2431057(1-16)Online publication date: 10-Feb-2025
    • (2025) Functional Projection K -means Journal of Computational and Graphical Statistics10.1080/10618600.2024.2429706(1-12)Online publication date: 3-Jan-2025
    • (2025)Partial multi-label feature selection with feature noisePattern Recognition10.1016/j.patcog.2024.111310(111310)Online publication date: Jan-2025
    • (2025)Three-way multi-label classification: A review, a framework, and new challengesApplied Soft Computing10.1016/j.asoc.2025.112757171(112757)Online publication date: Mar-2025
    • (2025)Warping resilient robust anomaly detection for multivariate time seriesMachine Language10.1007/s10994-024-06689-7114:2Online publication date: 1-Feb-2025
    • (2024)Privacy Preserving Human Mobility Generation Using Grid-Based Data and Graph AutoencodersISPRS International Journal of Geo-Information10.3390/ijgi1307024513:7(245)Online publication date: 9-Jul-2024
    • (2024)Graph Representation Learning for Street-Level Crime PredictionISPRS International Journal of Geo-Information10.3390/ijgi1307022913:7(229)Online publication date: 1-Jul-2024
    • (2024)Learnable Transform-Assisted Tensor Decomposition for Spatio-Irregular Multidimensional Data RecoveryACM Transactions on Knowledge Discovery from Data10.1145/370123519:1(1-23)Online publication date: 21-Oct-2024
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media