Abstract
Multi-label learning (MLL) can be viewed as an extension of multi-class learning (MCL) that supports nonexclusive labels. Random k-labelset ensemble (RAkEL) is a popular algorithm that transforms MLL into a series of MCL tasks to exploit label correlations. However, its effectiveness is impacted by the randomness of labelset construction. In this paper, we propose an MLL algorithm with nearest k-labelsets ensemble (NkEL) possessing three techniques. First, we select a labelset with a size of k for each label using the nearest-neighbor technique. Thus, NkEL considers high-order label correlations and has strong adaptability. Second, for each MCL problem, we build a neural network to provide numerical rather than categorical predictions. Therefore, the output values represent the confidence levels of different classes. Third, we propose an intra-labelset ensemble strategy for each label. This approach alleviates the limitations imposed by low class separability with the support of the total probability theorem. Experiments are conducted on datasets derived from various domains to compare the proposed method with fourteen popular algorithms. The results obtained in terms of six ranking-based and two classification-based measures demonstrate the feasibility and effectiveness of NkEL. The source code is available at github.com/fansmale/nkel.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
References
Chen S, Wang R, Lu J, Wang X (2022) Stable matching-based two-way selection in multi-label active learning with imbalanced data. Inform Sci 610:281–299. https://doi.org/10.1016/j.ins.2022.07.182
Qian K, Min X-Y, Cheng Y, Min F (2023) Weight matrix sharing for multi-label learning. Pattern Recogn 136:109156. https://doi.org/10.1016/j.patcog.2022.109156
Karagoz GN, Yazici A, Dokeroglu T, Cosar A (2020) Analysis of multiobjective algorithms for the classification of multi-label video datasets. IEEE Access 8:163937–163952. https://doi.org/10.1109/ACCESS.2020.3022317
Li X, Wu H, Li M, Liu H (2022) Multi-label video classification via coupling attentional multiple instance learning with label relation graph. Pattern Recog Lett 156:53–59. https://doi.org/10.1016/j.patrec.2022.01.003
Li F, Zuo Y, Lin H, Wu J (2023) Boostxml: gradient boosting for extreme multilabel text classification with tail labels. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2023.3285294
Bittencourt MM, Silva RM, Almeida TA (2020) Ml-mdltext: an efficient and lightweight multilabel text classifier with incremental learning. Appl Soft Comput 96:106699. https://doi.org/10.1016/j.asoc.2020.106699
Cai S, Li L, Han X, Huang S, Tian Q, Huang Q (2023) Semantic and correlation disentangled graph convolutions for multilabel image recognition. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2023.3333542
Gao B-B, Zhou H-Y (2021) Learning to discover multi-class attentional regions for multi-label image recognition. IEEE Trans Image Process 30:5920–5932. https://doi.org/10.1109/TIP.2021.3088605
Yang Y, Ding M (2019) Decision function with probability feature weighting based on Bayesian network for multi-label classification. Neural Comput Appl 31:4819–4828. https://doi.org/10.1007/s00521-017-3323-y
Majzoubi M, Choromanska A (2020) Ldsm: logarithm-depth streaming multi-label decision trees. In: AISTATS, pp 4247–4257
Valero-Mas JJ, Gallego AJ, Alonso-Jiménez P, Serra X (2023) Multilabel prototype generation for data reduction in k-nearest neighbour classification. Pattern Recogn 130:109190. https://doi.org/10.1016/j.patcog.2022.109190
Zhang M-L, Li Y-K, Liu X-Y, Geng X (2018) Binary relevance for multi-label learning: an overview. Front Comp Sci 12(2):191–202. https://doi.org/10.1007/s11704-017-7031-7
Tsoumakas G, Katakis I, Vlahavas I (2010) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089. https://doi.org/10.1109/TKDE.2010.164
Wang R, Kwong S, Wang X, Jia Y (2021) Active k-labelsets ensemble for multi-label classification. Pattern Recogn 109:107583. https://doi.org/10.1016/j.patcog.2020.107583
Zhang X-Y, Min F, Song G-J, Yu H (2023) LSTC: When label-specific features meet third-order label correlations. Inform Sci 632:617–636. https://doi.org/10.1016/j.ins.2023.03.056
Zhang M-L, Wu L (2014) LIFT: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120. https://doi.org/10.1109/TPAMI.2014.2339815
Fürnkranz J, Hüllermeier E, Loza Mencía E, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153. https://doi.org/10.1007/s10994-008-5064-8
Huang J, Li G, Wang S, Xue Z, Huang Q (2017) Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing 257:164–174. https://doi.org/10.1016/j.neucom.2016.12.073
Read J, Pfahringer B, Holmes G, Frank E (2009) Classifier chains for multi-label classification. In: ECML PKDD, pp 254–269
Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771. https://doi.org/10.1016/j.patcog.2004.03.009
Zheng X, Li P, Chu Z, Hu X (2019) A survey on multi-label data stream classification. IEEE Access 8:1249–1275. https://doi.org/10.1109/ACCESS.2019.2962059
Gnedenko BV (2018) Theory of Probability, 6th edn. Routledge, New York
Moyano JM, Gibaja EL, Cios KJ, Ventura S (2018) Review of ensembles of multi-label classifiers: models, experimental study and prospects. Inform Fusion 44:33–45. https://doi.org/10.1016/j.inffus.2017.12.001
Yang Y, Jiang J (2018) Adaptive bi-weighting toward automatic initialization and model selection for hmm-based hybrid meta-clustering ensembles. IEEE Trans Cybern 49(5):1657–1668. https://doi.org/10.1109/TCYB.2018.2809562
Ganaie MA, Hu M, Malik A, Tanveer M, Suganthan P (2022) Ensemble deep learning: a review. Eng Appl Artif Intell 115:105151. https://doi.org/10.1016/j.engappai.2022.105151
Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85:333–359. https://doi.org/10.1007/s10994-011-5256-5
Wei X, Yu Z, Zhang C, Hu Q (2018) Ensemble of label specific features for multi-label classification. In: ICME, pp 1–6
Wang X-Z, Wang R, Xu C (2018) Discovering the relationship between generalization and uncertainty by incorporating complexity of classification. IEEE Trans Cybern 48(2):703–715. https://doi.org/10.1109/TCYB.2017.2653223
Tsoumakas G, Dimou A, Spyromitros E, Mezaris V, Kompatsiaris I, Vlahavas I Correlation-based pruning of stacked binary relevance models for multi-label learning. In: Proceedings of the 1st international workshop on learning from multi-label data, pp 101–116
Xia Y, Chen K, Yang Y (2021) Multi-label classification with weighted classifier selection and stacked ensemble. Inform Sci 557:421–442. https://doi.org/10.1016/j.ins.2020.06.017
Zhu Y, Kwok JT, Zhou Z-H (2018) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094. https://doi.org/10.1109/TKDE.2017.2785795
Kumar S, Rastogi R (2022) Low rank label subspace transformation for multi-label learning with missing labels. Inform Sci 596:53–72. https://doi.org/10.1016/j.ins.2022.03.015
Deng T, Jia Q, Wang J, Fujita H (2023) Transformed Schatten-1 penalty based full-rank latent label learning for incomplete multi-label classification. Inform Sci 650:119699. https://doi.org/10.1016/j.ins.2023.119699
Ye H-J, Zhan D-C, Li N, Jiang Y (2019) Learning multiple local metrics: global consideration helps. IEEE Trans Pattern Anal Mach Intell 42(7):1698–1712. https://doi.org/10.1109/tpami.2019.2901675
Lu Y, Li W, Li H, Jia X (2023) Predicting label distribution from tie-allowed multi-label ranking. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2023.3300310
Li J, Cheng J-H, Shi J-Y, Huang F (2012) Brief introduction of Back Propagation (BP) neural network algorithm and its improvement. In: Advances in computer science and information engineering, pp 553–558
Tan A, Liang J, Wu W-Z, Zhang J (2022) Semi-supervised partial multi-label classification via consistency learning. Pattern Recogn 131:108839. https://doi.org/10.1016/j.patcog.2022.108839
Qian W, Xu F, Qian J, Shu W, Ding W (2023) Multi-label feature selection based on rough granular-ball and label distribution. Inform Sci 650:119698. https://doi.org/10.1016/j.ins.2023.119698
Wu X-Z, Zhou Z-H (2017) A unified view of multi-label performance measures. In: International conference on machine learning, PMLR, pp 3780–3788
Baker O, Yuan Q (2021) Machine learning: Factorization machines and normalized discounted cumulative gain for tourism recommender system optimisation. In: 2021 IEEE International Conference on Computing (ICOCO), pp 31–36
Ma Z, Chen S (2021) Expand globally, shrink locally: discriminant multi-label learning with missing labels. Pattern Recogn 111:107675. https://doi.org/10.1016/j.patcog.2020.107675
Zhang M-L, Zhou Z-H (2006) Multi-label neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351. https://doi.org/10.1109/TKDE.2006.162
Zhang M-L, Zhou Z-H (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048. https://doi.org/10.1016/j.patcog.2006.12.019
Zhang M-L (2012) An improved multi-label lazy learning approach. J Comput Res Develop 49(11):2271–2282
Huang J, Qin F, Zheng X, Cheng Z, Yuan Z, Zhang W, Huang Q (2019) Improving multi-label classification with missing labels by learning label-specific features. Inform Sci 492:124–146. https://doi.org/10.1016/j.ins.2019.04.021
Xie M-K, Huang S-J (2022) Partial multi-label learning with noisy label identification. IEEE Trans Pattern Anal Mach Intell 44(7):3676–3687. https://doi.org/10.1109/TPAMI.2021.3059290
Min X-Y, Qian K, Zhang B-W, Song G, Min F (2022) Multi-label active learning through serial-parallel neural networks. Knowledge-Based Syst 251:109226. https://doi.org/10.1016/j.knosys.2022.109226
Yu Z-B, Zhang M-L (2021) Multi-label classification with label-specific feature generation: a wrapped approach. IEEE Trans Pattern Anal Mach Intell 44(9):5199–5210. https://doi.org/10.1109/TPAMI.2021.3070215
Mao J-X, Wang W, Zhang M-L (2023) Label specific multi-semantics metric learning for multi-label classification: global consideration helps. In: Proceedings of the thirty-second international joint conference on artificial intelligence, pp 4055–4063
Xiong J, Yu L, Niu X, Leng Y (2023) XRR: extreme multi-label text classification with candidate retrieving and deep ranking. Inform Sci 622:115–132. https://doi.org/10.1016/j.ins.2022.11.158
Acknowledgements
This study is partially supported by the Nanchong Municipal Government-Universities Scientiffc Cooperation Project (Nos. 23XNSYSX0062, SXHZ051, 23XNSYSX0013), and National Supercomputing Center in Chengdu.
Author information
Authors and Affiliations
Contributions
Xi-Yan Zhong: Methodology, Investigation, Software, Writing-original draft. Yu-Li Zhang: Investigation, Writing-review & editing. Dan-Dong Wang: Investigation, Writing-review & editing. Fan Min: Supervision, Methodology, Writing-review & editing.
Corresponding author
Ethics declarations
Competing interests
The authors have no relevant financial or nonfinancial interests to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhong, XY., Zhang, YL., Wang, DD. et al. NkEL: nearest k-labelsets ensemble for multi-label learning. Appl Intell 55, 81 (2025). https://doi.org/10.1007/s10489-024-05968-z
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05968-z