Skip to main content

Advertisement

NkEL: nearest k-labelsets ensemble for multi-label learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Multi-label learning (MLL) can be viewed as an extension of multi-class learning (MCL) that supports nonexclusive labels. Random k-labelset ensemble (RAkEL) is a popular algorithm that transforms MLL into a series of MCL tasks to exploit label correlations. However, its effectiveness is impacted by the randomness of labelset construction. In this paper, we propose an MLL algorithm with nearest k-labelsets ensemble (NkEL) possessing three techniques. First, we select a labelset with a size of k for each label using the nearest-neighbor technique. Thus, NkEL considers high-order label correlations and has strong adaptability. Second, for each MCL problem, we build a neural network to provide numerical rather than categorical predictions. Therefore, the output values represent the confidence levels of different classes. Third, we propose an intra-labelset ensemble strategy for each label. This approach alleviates the limitations imposed by low class separability with the support of the total probability theorem. Experiments are conducted on datasets derived from various domains to compare the proposed method with fourteen popular algorithms. The results obtained in terms of six ranking-based and two classification-based measures demonstrate the feasibility and effectiveness of NkEL. The source code is available at github.com/fansmale/nkel.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. https://www.uco.es/kdis/mllresources/

  2. http://palm.seu.edu.cn/zhangml/files/Image.rar

  3. http://www.lamda.nju.edu.cn/code_BPMLL.ashx

  4. http://www.lamda.nju.edu.cn/files/MLkNN.rar

  5. http://www.lamda.nju.edu.cn/code_GLOCAL.ashx

  6. http://www.escience.cn/people/huangjun/index.html

  7. http://www.xiemk.pro/code/PMLNIcode.zip

  8. https://github.com/sanjayksau/lrmml-2

  9. https://gitee.com/fansmale/masp

  10. https://palm.seu.edu.cn/zhangml/files/WRAP.rar

  11. https://palm.seu.edu.cn/zhangml/files/LIMIC.rar

  12. http://manikvarma.org/downloads/XC/XMLRepository.html

References

  1. Chen S, Wang R, Lu J, Wang X (2022) Stable matching-based two-way selection in multi-label active learning with imbalanced data. Inform Sci 610:281–299. https://doi.org/10.1016/j.ins.2022.07.182

  2. Qian K, Min X-Y, Cheng Y, Min F (2023) Weight matrix sharing for multi-label learning. Pattern Recogn 136:109156. https://doi.org/10.1016/j.patcog.2022.109156

    Article  MATH  Google Scholar 

  3. Karagoz GN, Yazici A, Dokeroglu T, Cosar A (2020) Analysis of multiobjective algorithms for the classification of multi-label video datasets. IEEE Access 8:163937–163952. https://doi.org/10.1109/ACCESS.2020.3022317

    Article  Google Scholar 

  4. Li X, Wu H, Li M, Liu H (2022) Multi-label video classification via coupling attentional multiple instance learning with label relation graph. Pattern Recog Lett 156:53–59. https://doi.org/10.1016/j.patrec.2022.01.003

    Article  MATH  Google Scholar 

  5. Li F, Zuo Y, Lin H, Wu J (2023) Boostxml: gradient boosting for extreme multilabel text classification with tail labels. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2023.3285294

    Article  Google Scholar 

  6. Bittencourt MM, Silva RM, Almeida TA (2020) Ml-mdltext: an efficient and lightweight multilabel text classifier with incremental learning. Appl Soft Comput 96:106699. https://doi.org/10.1016/j.asoc.2020.106699

    Article  Google Scholar 

  7. Cai S, Li L, Han X, Huang S, Tian Q, Huang Q (2023) Semantic and correlation disentangled graph convolutions for multilabel image recognition. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2023.3333542

    Article  MATH  Google Scholar 

  8. Gao B-B, Zhou H-Y (2021) Learning to discover multi-class attentional regions for multi-label image recognition. IEEE Trans Image Process 30:5920–5932. https://doi.org/10.1109/TIP.2021.3088605

    Article  MATH  Google Scholar 

  9. Yang Y, Ding M (2019) Decision function with probability feature weighting based on Bayesian network for multi-label classification. Neural Comput Appl 31:4819–4828. https://doi.org/10.1007/s00521-017-3323-y

    Article  MATH  Google Scholar 

  10. Majzoubi M, Choromanska A (2020) Ldsm: logarithm-depth streaming multi-label decision trees. In: AISTATS, pp 4247–4257

  11. Valero-Mas JJ, Gallego AJ, Alonso-Jiménez P, Serra X (2023) Multilabel prototype generation for data reduction in k-nearest neighbour classification. Pattern Recogn 130:109190. https://doi.org/10.1016/j.patcog.2022.109190

    Article  Google Scholar 

  12. Zhang M-L, Li Y-K, Liu X-Y, Geng X (2018) Binary relevance for multi-label learning: an overview. Front Comp Sci 12(2):191–202. https://doi.org/10.1007/s11704-017-7031-7

    Article  MATH  Google Scholar 

  13. Tsoumakas G, Katakis I, Vlahavas I (2010) Random k-labelsets for multilabel classification. IEEE Trans Knowl Data Eng 23(7):1079–1089. https://doi.org/10.1109/TKDE.2010.164

    Article  MATH  Google Scholar 

  14. Wang R, Kwong S, Wang X, Jia Y (2021) Active k-labelsets ensemble for multi-label classification. Pattern Recogn 109:107583. https://doi.org/10.1016/j.patcog.2020.107583

    Article  MATH  Google Scholar 

  15. Zhang X-Y, Min F, Song G-J, Yu H (2023) LSTC: When label-specific features meet third-order label correlations. Inform Sci 632:617–636. https://doi.org/10.1016/j.ins.2023.03.056

    Article  MATH  Google Scholar 

  16. Zhang M-L, Wu L (2014) LIFT: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120. https://doi.org/10.1109/TPAMI.2014.2339815

    Article  MATH  Google Scholar 

  17. Fürnkranz J, Hüllermeier E, Loza Mencía E, Brinker K (2008) Multilabel classification via calibrated label ranking. Mach Learn 73(2):133–153. https://doi.org/10.1007/s10994-008-5064-8

    Article  MATH  Google Scholar 

  18. Huang J, Li G, Wang S, Xue Z, Huang Q (2017) Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing 257:164–174. https://doi.org/10.1016/j.neucom.2016.12.073

    Article  MATH  Google Scholar 

  19. Read J, Pfahringer B, Holmes G, Frank E (2009) Classifier chains for multi-label classification. In: ECML PKDD, pp 254–269

  20. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771. https://doi.org/10.1016/j.patcog.2004.03.009

    Article  MATH  Google Scholar 

  21. Zheng X, Li P, Chu Z, Hu X (2019) A survey on multi-label data stream classification. IEEE Access 8:1249–1275. https://doi.org/10.1109/ACCESS.2019.2962059

    Article  MATH  Google Scholar 

  22. Gnedenko BV (2018) Theory of Probability, 6th edn. Routledge, New York

    Book  MATH  Google Scholar 

  23. Moyano JM, Gibaja EL, Cios KJ, Ventura S (2018) Review of ensembles of multi-label classifiers: models, experimental study and prospects. Inform Fusion 44:33–45. https://doi.org/10.1016/j.inffus.2017.12.001

    Article  Google Scholar 

  24. Yang Y, Jiang J (2018) Adaptive bi-weighting toward automatic initialization and model selection for hmm-based hybrid meta-clustering ensembles. IEEE Trans Cybern 49(5):1657–1668. https://doi.org/10.1109/TCYB.2018.2809562

    Article  MathSciNet  MATH  Google Scholar 

  25. Ganaie MA, Hu M, Malik A, Tanveer M, Suganthan P (2022) Ensemble deep learning: a review. Eng Appl Artif Intell 115:105151. https://doi.org/10.1016/j.engappai.2022.105151

    Article  Google Scholar 

  26. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85:333–359. https://doi.org/10.1007/s10994-011-5256-5

    Article  MathSciNet  MATH  Google Scholar 

  27. Wei X, Yu Z, Zhang C, Hu Q (2018) Ensemble of label specific features for multi-label classification. In: ICME, pp 1–6

  28. Wang X-Z, Wang R, Xu C (2018) Discovering the relationship between generalization and uncertainty by incorporating complexity of classification. IEEE Trans Cybern 48(2):703–715. https://doi.org/10.1109/TCYB.2017.2653223

    Article  MATH  Google Scholar 

  29. Tsoumakas G, Dimou A, Spyromitros E, Mezaris V, Kompatsiaris I, Vlahavas I Correlation-based pruning of stacked binary relevance models for multi-label learning. In: Proceedings of the 1st international workshop on learning from multi-label data, pp 101–116

  30. Xia Y, Chen K, Yang Y (2021) Multi-label classification with weighted classifier selection and stacked ensemble. Inform Sci 557:421–442. https://doi.org/10.1016/j.ins.2020.06.017

    Article  MathSciNet  MATH  Google Scholar 

  31. Zhu Y, Kwok JT, Zhou Z-H (2018) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094. https://doi.org/10.1109/TKDE.2017.2785795

    Article  MATH  Google Scholar 

  32. Kumar S, Rastogi R (2022) Low rank label subspace transformation for multi-label learning with missing labels. Inform Sci 596:53–72. https://doi.org/10.1016/j.ins.2022.03.015

    Article  MATH  Google Scholar 

  33. Deng T, Jia Q, Wang J, Fujita H (2023) Transformed Schatten-1 penalty based full-rank latent label learning for incomplete multi-label classification. Inform Sci 650:119699. https://doi.org/10.1016/j.ins.2023.119699

    Article  MATH  Google Scholar 

  34. Ye H-J, Zhan D-C, Li N, Jiang Y (2019) Learning multiple local metrics: global consideration helps. IEEE Trans Pattern Anal Mach Intell 42(7):1698–1712. https://doi.org/10.1109/tpami.2019.2901675

    Article  MATH  Google Scholar 

  35. Lu Y, Li W, Li H, Jia X (2023) Predicting label distribution from tie-allowed multi-label ranking. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2023.3300310

  36. Li J, Cheng J-H, Shi J-Y, Huang F (2012) Brief introduction of Back Propagation (BP) neural network algorithm and its improvement. In: Advances in computer science and information engineering, pp 553–558

  37. Tan A, Liang J, Wu W-Z, Zhang J (2022) Semi-supervised partial multi-label classification via consistency learning. Pattern Recogn 131:108839. https://doi.org/10.1016/j.patcog.2022.108839

    Article  MATH  Google Scholar 

  38. Qian W, Xu F, Qian J, Shu W, Ding W (2023) Multi-label feature selection based on rough granular-ball and label distribution. Inform Sci 650:119698. https://doi.org/10.1016/j.ins.2023.119698

    Article  MATH  Google Scholar 

  39. Wu X-Z, Zhou Z-H (2017) A unified view of multi-label performance measures. In: International conference on machine learning, PMLR, pp 3780–3788

  40. Baker O, Yuan Q (2021) Machine learning: Factorization machines and normalized discounted cumulative gain for tourism recommender system optimisation. In: 2021 IEEE International Conference on Computing (ICOCO), pp 31–36

  41. Ma Z, Chen S (2021) Expand globally, shrink locally: discriminant multi-label learning with missing labels. Pattern Recogn 111:107675. https://doi.org/10.1016/j.patcog.2020.107675

    Article  Google Scholar 

  42. Zhang M-L, Zhou Z-H (2006) Multi-label neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351. https://doi.org/10.1109/TKDE.2006.162

    Article  MATH  Google Scholar 

  43. Zhang M-L, Zhou Z-H (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048. https://doi.org/10.1016/j.patcog.2006.12.019

    Article  MATH  Google Scholar 

  44. Zhang M-L (2012) An improved multi-label lazy learning approach. J Comput Res Develop 49(11):2271–2282

  45. Huang J, Qin F, Zheng X, Cheng Z, Yuan Z, Zhang W, Huang Q (2019) Improving multi-label classification with missing labels by learning label-specific features. Inform Sci 492:124–146. https://doi.org/10.1016/j.ins.2019.04.021

    Article  MathSciNet  MATH  Google Scholar 

  46. Xie M-K, Huang S-J (2022) Partial multi-label learning with noisy label identification. IEEE Trans Pattern Anal Mach Intell 44(7):3676–3687. https://doi.org/10.1109/TPAMI.2021.3059290

    Article  MATH  Google Scholar 

  47. Min X-Y, Qian K, Zhang B-W, Song G, Min F (2022) Multi-label active learning through serial-parallel neural networks. Knowledge-Based Syst 251:109226. https://doi.org/10.1016/j.knosys.2022.109226

    Article  MATH  Google Scholar 

  48. Yu Z-B, Zhang M-L (2021) Multi-label classification with label-specific feature generation: a wrapped approach. IEEE Trans Pattern Anal Mach Intell 44(9):5199–5210. https://doi.org/10.1109/TPAMI.2021.3070215

    Article  MATH  Google Scholar 

  49. Mao J-X, Wang W, Zhang M-L (2023) Label specific multi-semantics metric learning for multi-label classification: global consideration helps. In: Proceedings of the thirty-second international joint conference on artificial intelligence, pp 4055–4063

  50. Xiong J, Yu L, Niu X, Leng Y (2023) XRR: extreme multi-label text classification with candidate retrieving and deep ranking. Inform Sci 622:115–132. https://doi.org/10.1016/j.ins.2022.11.158

    Article  Google Scholar 

Download references

Acknowledgements

This study is partially supported by the Nanchong Municipal Government-Universities Scientiffc Cooperation Project (Nos. 23XNSYSX0062, SXHZ051, 23XNSYSX0013), and National Supercomputing Center in Chengdu.

Author information

Authors and Affiliations

Authors

Contributions

Xi-Yan Zhong: Methodology, Investigation, Software, Writing-original draft. Yu-Li Zhang: Investigation, Writing-review & editing. Dan-Dong Wang: Investigation, Writing-review & editing. Fan Min: Supervision, Methodology, Writing-review & editing.

Corresponding author

Correspondence to Fan Min.

Ethics declarations

Competing interests

The authors have no relevant financial or nonfinancial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, XY., Zhang, YL., Wang, DD. et al. NkEL: nearest k-labelsets ensemble for multi-label learning. Appl Intell 55, 81 (2025). https://doi.org/10.1007/s10489-024-05968-z

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05968-z

Keywords