Abstract
Multi-label classification is a challenging issue because it simultaneously embraces the characteristics of the imbalanced class distribution for each label and the uncertain label correlation among the whole label space. The decision-theoretic rough set can describe the roughness of concepts in the sense of minimizing decision risk but fails to consider the case where concepts are compatible. We argue that it is feasible to analyze the uncertainty of coarse-grained logical labels with limited label correlation assumptions and reduce the classification error for those uncertain instances by learning fine-grained numerical labels. Consequently, we develop a multi-granular label information system by introducing a multi-granular threshold with a three-way-based label enhancement (MGT-LEML) model. With the second-order label correlation assumption, we deduce the pseudo-positive and pseudo-negative classes for each label. The decision-theoretic rough set evaluates the possibility of misclassification independently, and a novel uncertain measure called instance uncertainty degree determines whether it is necessary to conduct label enhancement afterward. In this way, instances with the most uncertain classifications across label space compute fine-granule numerical labels by label enhancement, whereas remaining unchanged otherwise. We analyze the comparison results among nine algorithms on eight benchmarks with six metrics to demonstrate the superiority of the proposed MGT-LEML algorithm over state-of-the-art multi-label classification algorithms. Compared with the HNOML algorithm, our algorithm achieves significant improvement. Concretely, the performance is reduced by 2.9% in Hamming Loss, 12.4% in Ranking Loss, 14.3% in One Error, 465.5% in Coverage, and is increased by 14.2% in Average Precision.

Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
Notes
code available at http://www.lamda.nju.edu.cn/code_MLkNN.ashx
code available at http://cse.seu.edu.cn/PersonalPage/zhangml/index.htm
code available at https://jiunhwang.github.io/
code available at http://www.optimal-group.org/Resource/MLTSVM.html
code available at http://www.lamda.nju.edu.cn/code_Glocal.ashx
code available at http://github.com/KKimura360/fast_RAkEL_matlab
code available at https://www.csie.ntu.edu.tw/~cjlin/liblinear/
code available at https://github.com/JianghongMA/MC-GM
References
Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837
Gibaja E, Ventura S (2015) A tutorial on multilabel learning. ACM Comput Surv 47(3):1–38
Liu WW, Shen XB, Wang HB, Tsang IW (2022) The emerging trends of multi-label learning. IEEE Trans Pattern Anal Mach Intell 44(3):7955–7974
Van Landeghem J, Blaschko M, Anckaert B, Moens MF (2022) Benchmarking scalable predictive uncertainty in text classification. IEEE Access 10:43703–43737
Luo LY, Yu LQ, Chen H, Liu QD, Wang X, Xu JQ, Heng PA (2020) Deep mining external imperfect data for chest x-ray disease screening. IEEE Trans Med Imaging 39(11):3583–3594
Zhang H, Jiang LX, Xu WQ (2021) Multiple noisy label distribution propagation for crowdsourcing. Paper presented at the proceedings of 28th international joint conference on artificial intelligence, Macao, Peoples R China, 10–16 August 2019
Luo JQ, He B, Ou Y (2021) Topic-based label distribution learning to exploit label ambiguity for scene classification. Neural Comput Appl 33(23):16181–16196
Li PP, Hu YB, Wu X, He R, Sun ZN (2020) Deep label refinement for age estimation. Pattern Recognit. 100:107178
He ZZ, Li X, Zhang ZF, Wu F, Geng X, Zhang YQ, Yang MH, Zhuang YT (2017) Data-dependent label distribution learning for age estimation. IEEE Trans Image Process 26(8):3846–3858
Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit. 37(9):1757–1771
Tsoumakas G, Vlahavas I (2007) Random k-labelsets: an ensemble method for multilabel classification. Paper presented at the 18th European conference on machine learning (ECML 2007)/11th European conference on principles and practice of knowledge discovery in databases (PKDD 2007), Warsaw University, Poland, 17–21 September 2007
Huang J, Li GR, Huang QM, Wu XD (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323
Zhang YJ, Miao DQ, Zhang ZF, Xu JF, Luo S (2018) A three-way selective ensemble model for multi-label classification. Int J Approx Reason 103:394–413
Zhang ML, Zhou ZH (2007) Ml-knn: A lazy learning approach to multi-label learning. Pattern Recognit. 40(7):2038–2048
Wu QY, Tan MK, Song HJ, Chen J, Ng MK (2016) Ml-forest: a multi-label tree ensemble method for multi-label classification. IEEE Trans Knowl Data Eng 28(10):2665–2680
Chen YH, Shao C, Li N, Deng NY (2016) Mltsvm: a novel twin support vector machine to multi-label learning. Pattern Recognit. 52:61–74
Xu SP, Yang XB, Yu HL, Yu DJ, Yang JY, Tsang ECC (2016) Multi-label learning with label-specific feature reduction. Knowl-Based Syst 104:52–61
Geng X (2016) Label distribution learning. IEEE Trans Knowl Data Eng 28(7):1734–1748
Xu SP, Ju HR, Shang L, Pedrycz W, Yang XB, Li C (2020) Label distribution learning: a local collaborative mechanism. Int J Approx Reason 121:59–84
Xu SP, Shang L, Shen FR (2017) Latent semantics encoding for label distribution learning. Paper presented at the 28th international joint conference on artificial intelligence (IJCAI’19), China, Macao, Aug 2019. p 3982–3988
Tao A, Xu N, Geng X (2018) Labeling information enhancement for multi-label learning with low-rank subspace. Paper presented at the 15th Pacific Rim international conference on artificial intelligence (PRICAI), Nanjing, Peoples R China, 28–31 Aug 2018
Li YK, Zhang ML, Geng X (2015) Leveraging implicit relative labeling importance information for effective multi-label learning. Paper presented at the IEEE international conference on data mining (ICDM), Atlantic City, NJ, 14–17 Nov 2015
Xu N, Tao A, Geng X (2018) Label enhancement for label distribution. Paper presented at the 27th international joint conference on artificial intelligence (IJCAI), Stockholm, Sweden, 13–19 Jul 2018
Shao RF, Xu N, Geng X (2018) Multi-label learning with label enhancement. Paper presented at the 18th IEEE international conference on data mining workshops (ICDMW), Singapore, Singapore, 17–20 Nov 2018
Xu WH, Guo DD, Qian YH, Ding WP (2022) Two-way concept-cognitive learning method: A fuzzy-based progressive learning. IEEE Trans. Fuzzy Syst. 1–15. https://doi.org/10.1109/TFUZZ.2022.3216110
Xu WH, Yuan KH, Ding WP (2023) An emerging fuzzy feature selection method using composite entropy-based uncertainty measure and data distribution. IEEE Trans. Emerg. Top Comput. Intell. 7(1):76–88
Xu WH, Pan YZ, Chen XW, Ding WP, Qian YH (2022) A novel dynamic fusion approach using information entropy for interval-valued ordered datasets. IEEE Trans. Big Data. https://doi.org/10.1109/TBDATA.2022.3215494
Yao YY (2009) Three-way decision: an interpretation of rules in rough set theory. Paper presented at the 4th international conference on rough sets and knowledge technology (RSKT), Gold Coast, Australia, 14–16 Jul 2009
Yao YY (2018) Three-way decision and granular computing. Int J Approx Reason 103:107–123
Yao YY (2020) Tri-level thinking: models of three-way decision. Int J Mach Learn Cybern 11:947–959
Yao YY (2021) The geometry of three-way decision. Appl Intell 51(9):6298–6325
Zhang XY, Gou HY, Lv ZY, Miao DQ (2021) Double-quantitative distance measurement and classification learning based on the tri-level granular structure of neighborhood system. Knowl-Based Syst 217:106799
Zhang K, Dai JH, Zhan JM (2021) A new classification and ranking decision method based on three-way decision theory and topsis models. Inf Sci 568:54–85
Liu JB, Li HX, Huang B, Liu Y, Liu D (2021) Convex combination-based consensus analysis for intuitionistic fuzzy three-way group decision. Inf Sci 574:542–566
Liang DC, Fu YY, Xu ZS (2022) Three-way group consensus decision based on hierarchical social network consisting of decision makers and participants. Inf Sci 585:289–312
Xu WH, Guo DD, Mi JS, Qian YH, Zheng KY, Ding WP (2023) Two-way concept-cognitive learning via concept movement viewpoint. IEEE Trans. Neural Netw. Learn. Syst. 1–15. https://doi.org/10.1109/TNNLS.2023.3235800
Yuan KH, Xu WH, Li WT, Ding WP (2022) An incremental learning mechanism for object classification based on progressive fuzzy three-way concept. Inf Sci 584:127–147
Ren FJ, Wang L (2017) Sentiment analysis of text based on three-way decisions. J. Intell. Fuzzy. Syst. 33(1):245–254
Zhang YJ, Zhao TN, Miao DQ, Pedrycz W (2022) Granular multilabel batch active learning with pairwise label correlation. IEEE. Trans. Syst. Man. Cybern. -Syst. 52(5):3079–3091
Qian WB, Huang JT, Wang YL, Xie YH (2021) Label distribution feature selection for multi-label classification with rough set. Int J Approx Reason 128:32–55
Zhang ML, Wu L (2015) Lift: Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120
Zhu Y, Kwok JT, Zhou ZH (2018) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094
Zhang J, Luo ZM, Li CD, Zhou CG, Li SZ (2019) Manifold regularized discriminative feature selection for multi-label learning. Pattern Recognit. 95:136–150
Min XY, Qian K, Zhang BW, Song GJ, Min F (2022) Multi-label active learning through serial-parallel neural networks. Knowl-Based Syst 251:109226
Kimura K, Kudo M, Sun L, Koujaku S (2017) Fast random k-labelsets for large-scale multi-label classification. Paper presented at the 23rd international conference on pattern recognition (ICPR), Mexican Assoc Comp Vis Robot & Neural Comp, Mexico, 04–08 Dec 2016
Nazmi S, Yan XY, Homaifar A, Docuettee E (2020) Evolving multi-label classification rules by exploiting high-order label correlations. Neurocomput. 417:176–186
Wang, R., Kwong, S., Wang, X., Jia, Y.: Active k-labelsets ensemble for multi-label classification 109, 107583 (2021)
Huang J, Qin F, Zheng X, Cheng ZK, Yuan ZX, Zhang WG, Huang QM (2019) Improving multi-label classification with missing labels by learning label-specific features. Inf Sci 492:124–146
Lin YJ, Hu QH, Liu JH, Zhu XQ, Wu XD (2021) Mulfe: multi-label learning via label-specific feature space ensemble. ACM Trans Knowl Discov Data 16(1):5
Cheng YS, Zhang C, Pang SF (2022) Multi-label space reshape for semantic-rich label-specific features learning. Int J Mach Learn Cybern 13:1005–1019
Wang YB, Hang JY, Zhang ML (2022) Stable label-specific features generation for multi-label learning via mixture-based clustering ensemble. IEEE-CAA J. Automatica Sin. 9(7):1248–1261
Long XD, Qian WB, Wang YL, Shu WH (2021) Cost-sensitive feature selection on multi-label data via neighborhood granularity and label enhancement. Appl Intell 51(4):2210–2232
Liu XY, Zhu JH, Zheng QH, Li ZY, Liu RX, Wang J (2021) Bidirectional loss function for label enhancement and distribution learning. Knowl-Based Syst 213:106690
Liu XY, Zhu JH, Li ZY, Tian ZQ, Jia XY, Chen L (2021) Unified framework for learning with label distribution. Inf. Fusion. 75:116–130
Li WW, Chen J, Gao PX, Huang ZQ (2022) Label enhancement with label-specific feature learning. Int J Mach Learn Cybern 13(10):2857–2867
Qian WB, Xiong CZ, Qian YH, Wang YL (2022) Label enhancement-based feature selection via fuzzy neighborhood discrimination index. Knowl-Based Syst 250:109119
Tan C, Chen S, Ji GL, Geng X (2022) Multilabel distribution learning based on multioutput regression and manifold learning. IEEE Trans. Cybern. 52(6):5064–5078
Tsoumakas G, Spyromitros-Xiousfis E, Vilcke I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12(7):2411–2414
Zhang CQ, Yu ZW, Fu HZ, Zhu PF, Chen L, Hu QH (2020) Hybrid noise-oriented multilabel learning. IEEE Trans. Cybern. 50(6):2837–2850
Ma JH, Chiu BCY, Chow TWS (2022) Multilabel classification with group-based mapping: a framework with local feature selection and local label correlation. IEEE Trans. Cybern. 52(6):4596–4610
Schapire R, Singer Y (2000) A boosting-based system for text categorization. Mach Learn 39(2/3):135–168
Demsar J (2006) Statistical comparisons of classifier over multiple data sets. J Mach Learn Res 7:1–30
Acknowledgements
We gratitude the following fund programs: China Postdoctoral Science Foundation (Grant No. 2022M713491), National Natural Science Foundation of China (Grant No. 61976158, 61976160, 62076182, 62163016, 62006172, 61906137) and Jiangxi “Double Thousand Plan” (Grant No. 20212ACB202001). These funding projects helped in the completion of the article. The authors have no relevant financial or non-financial interests to disclose.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhao, T., Zhang, Y., Miao, D. et al. Multi-granular labels with three-way decisions for multi-label classification. Int. J. Mach. Learn. & Cyber. 14, 3737–3752 (2023). https://doi.org/10.1007/s13042-023-01861-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-023-01861-2