Abstract
In the age of big data, the number of class labels is increasing rapidly and there exists a hierarchical structure between different class labels. In the present paper, we revisit the existing granular computing approach to hierarchical classification. By revealing some limitations of approximation capacity, we develop a novel model for hierarchical classification. Then, we present a formal approach to feature selection for hierarchical decision tables by using fuzzy rough set theory. Correspondingly, an algorithm using relative discernibility relation is designed to select relevant feature subsets. Considering the fact that real data may vary dynamically with time, we also propose an incremental approach to hierarchical feature selection by using fuzzy rough set technique. An incremental algorithm for hierarchical feature selection is provided based on the sibling strategy. The experimental results demonstrate that the proposed approach is feasible and valid.
Similar content being viewed by others
References
Aho AV, Hopcroft JE, Ullman JD (1976) On finding lowest common ancestors in trees. Assoc Comput Mach 5(1):115–132
Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Neural Information Processing Systems
Bai S, Lin Y, Lv Y, Chen J, Wang C (2021) Kernelized fuzzy rough sets based online streaming feature selection for large-scale hierarchical classification. Appl Intell 51(3):1602–1615
Chris HQ, Dubchak I (2001) Multi-class protein fold recognition using support vector machines and neural networks. Bioinformatics 17(4):349–358
Chen D, Zhang L, Zhao S, Hu Q, Zhu P (2012) A novel algorithm for finding reducts with fuzzy rough sets. IEEE Trans Fuzzy Syst 20(2):385–389
Chen D, Zhao S, Zhang L, Zhang X (2012) Sample pair selection for attribute reduction with rough set. IEEE Trans Knowl Data Eng 24(11):2080–2093
Chen H, Li T, Luo C, Horng S, Wang G (2014) A rough set-based method for updating decision rules on attribute values’ coarsening and refining. IEEE Trans knowl Data Engin 26(12):2886–2899
Chen D, Yang Y, Dong Z (2016) An incremental algorithm for attribute reduction with variable precision rough sets. Appl Soft Comput 45:129–149
Cheng Y, Zhang Q, Wang G, Hu B (2020) Optimal scale selection and attribute reduction in multi-scale decision tables based on three-way decision. Inf Sci 541:36–59
Deng J, Dong W, Socher W, Li LJ, Li K, Li FF (2009) Imagenet: A large-scale hierarchical image database. IEEE Conf Comput Vision Pattern Recog 2009:248–255
Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17:191–209
Deng J, Zhan J, Ding W, Liu P, Pedrycz W (2022) A novel prospect-theory-based three-way decision methodology in multi-scale information systems. Artif Intell Review. https://doi.org/10.1007/s10462-022-10339-6
Escalante HJ (2010) The segmented and annotated IAPR TC-12 benchmark. Comput Vis Image Und 114(4):419–428
Fan W, He C, Zeng A, Lin K (2022) An Incremental Approach Based on Hierarchical Classification in Multikernel Fuzzy Rough Sets Under the Variation of Object Set. Int Conf on Intell Comput. https://doi.org/10.1007/978-3-031-13832-4_1
Gaussier E, Paliouras G, Androutsopoulos I (2015) Evaluation measures for hierarchical classification: a unified view and novel approaches. Data Mining Knowl Discov 29:820–865
Hong TP, Liou YL, Wang SL (2009) Fuzzy rough sets with hierarchical quantitative attributes. Expert Syst Appl 36(3):6790–6799
Hu Q, Wang Y, Zhou Y, Qian Y, Liang J (2018) Review on hierarchical learning methods for large-scale classification task. Sci Sin Inf 48(5):487–500
Huang B, Wu WZ, Yan J, Li H, Zhou X (2020) Inclusion measure-based multi-granulation decision-theoretic rough sets in multi-scale intuitionistic fuzzy information tables. Inf Sci 507:421–448
Liu ZT (1999) An incremental arithmetic for the smallest reduction of attributes. Acta Electro Sin 27(11):96–98
Lampert CH, Nickisch H, Harmeling S (2009) Learning to detect unseen object classes by between-class attribute transfer. IEEE Conf on Comput Vision and Pattern Recog 2009:951–958
Li N, Xie JY (2011) A feature subset selection algorithm based on neighborhood rough set for incremental updating datasets. Comput Technol Dev 21(11):149–152
Liang J, Wang F, Dang C, Qian Y (2012) A group incremental approach to feature selection applying rough set technique. IEEE Trans on Knowl and Data Eng 26(2):294–308
Li DP, Ju Y, Zou Q (2016) Protein folds prediction with hierarchical structured svm. Curr Proteomics 13(2):79–85
Li F, Hu BQ (2017) A new approach of optimal scale selection to multi-scale decision tables. Inf Sci 381:193–208
Luo C, Li T, Chen H, Fujita H, Zhang Y (2018) Incremental rough set approach for hierarchical multicriteria classification. Inf Sci 429:72–87
Li W, Li J, Huang J, Dai W, Zhang X (2021) A new rough set model based on multi-scale covering. Int J Mach Learn Cyb 12(1):243–256
Liu X, Zhou Y, Zhao H (2021) Robust hierarchical feature selection driven by data and knowledge. Inf Sci 551:341–357
Li S, Yang J, Wang G, Zhang Q, Hu J (2022) Granularity Selection for Hierarchical Classification Based on Uncertainty Measure. IEEE Trans on Fuzzy Syst 30(11):4841–4855
Lin Y, Liu H, Zhao H, Hu Q, Zhu X, Wu X (2022) Hierarchical Feature Selection Based on Label Distribution Learning. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2022.3177246
Morsi NN, Yakout M (1998) Axiomatics for fuzzy rough sets. Fuzzy Sets Syst 100(1):327–342
Qian W, Shu W, Zhang C (2016) Feature selection from the perspective of knowledge granulation in dynamic set-valued information system. J Inf Sci Eng 32(3):783–798
Qiu Z, Zhao H (2022) A fuzzy rough set approach to hierarchical feature selection based on Hausdorff distance. Appl Intell 52:11089–11102
Struyf J, Dzeroski S, Blockeel H, Clare A (2005) Hierarchical multi classification with predictive clustering trees in functional genomics. Springer, Cham
Shu W, Shen H (2014) Updating attribute reduction in incomplete decision systems with the variation of attribute set. Int J Approx Reason 55(3):867–884
Shu W, Shen H (2014) Incremental feature selection based on rough set in dynamic incomplete data. Pattern Recognit 47(12):3890–3906
She Y, Li J, Yang H (2015) A local approach to rule induction in multi-scale decision tables. Knowl Based Syst 89:398–410
Song J, Zhang P, Qin S, Gong J (2015) A method of the feature selection in hierarchical text classification based on the category discrimination and position information. In Proc Int Conf Ind Informat Comput Technol Intell Technol Ind Inf Integration 2015:132–135
Sohn K (2016) Improved deep metric learning with multi-class n-pair loss objective. Adv Neural Inf Process Syst 29:1857–1865
She Y, Qian Z, He X, Wang J, Qian T, Zheng W (2021) On generalization reducts in multi-scale decision tables. Inf Sci 555:104–124
She Y, Zhao Z, Hu M, Zheng W, He X (2021) On selection of optimal cuts in complete multi-scale decision tables. Artif Intell Review 54(8):6125–6148
She Y, Wu J, He X (2022) Research on feature selection algorithm of hierarchical classification based on sample pair selection. J Kunming Univ Sci Technol 47(5):92–102 ((in Chinese))
Tuo Q, Zhao H, Hu Q (2019) Hierarchical feature selection with subtree based graph regularization. Knowl-Based Syst 163:996–1008
Wu WZ, Leung Y (2011) Theory and applications of granular labelled partitions in multi-scale decision tables. Inf Sci 181(18):3878–3897
Wang F, Liang J, Dang C (2013) Attribute reduction for dynamic data sets. Appl Soft Comput 13(1):676–689
Wu WZ, Leung Y (2013) Optimal scale selection for multi-scale decision tables. Int J Approx Reason 54(8):1107–1129
Wu XD, Zhu XQ, Wu GQ, Ding W (2014) Data mining with big data. IEEE Trans on Knowl and Data Eng 26(1):9–107
Wu WZ, Qian YH, Li TJ, Gu SM (2017) On rule acquisition in incomplete multi-scale decision tables. Inf Sci 378:282–302
Wang Y, Wang Z, Hu Q, Zhou Y, Su H (2021) Hierarchical semantic risk minimization for large-scale classification. IEEE Trans Cyber 53(9):9546–9558
Wang Y, Hu Q, Chen H, Qian Y (2022) Uncertainty instructed multi-granularity decision for large-scale hierarchical classification. Inf Sci 586:644–661
Yang Y, Chen D, Wang H (2016) Active sample selection based incremental algorithm for attribute reduction with rough sets. IEEE Trans Fuzzy Syst 25(4):825–838
Yang Y, Chen D, Wang H, Wang X (2018) Incremental perspective for feature selection based on fuzzy rough sets. IEEE Trans on Fuzzy Syst 26(3):1257–1273
Yang X, Li Y, Liu D (2021) Hierarchical fuzzy rough approximations with three-way multi-granularity learning. IEEE Trans on Fuzzy Syst 30(9):3486–3500
Zhao H, Hu Q, Zhu P, Wang Y, Wang P (2009) A recursive regularization based feature selection framework for hierarchical classification. IEEE Trans Knowl Data Eng 33(7):2833–2846
Zeng A, Li T, Liu D, Zhang J (2015) Chen H (2015) A fuzzy rough set approach for incremental feature selection on hybrid information systems. Fuzzy Sets Syst 258:39–60
Zhao S, Han Y, Zou Q, Hu Q (2016) Hierarchical support vector machine based structural classification with fused hierarchies. Neurocomputing 214:86–92
Zhao H, Wang P, Hu Q, Zhu P (2019) Fuzzy rough set based feature selection for large-scale hierarchical classification. IEEE Trans on Fuzzy Syst 27(10):1891–1903
Zhan J, Xu W (2020) Two types of coverings based multigranulation rough fuzzy sets and applications to decision making. Artif Intell Review 53(1):167–198
Zhang Q, Cheng Y, Zhao F, Wang G, Xia S (2021) Optimal scale combination selection integrating three-way decision with hasse diagram. IEEE Trans Neural Netw Learn Syst 33(8):3675–3689
Acknowledgements
The authors are very thankful to editors and three referees for their suggestive reports and valuable comments which are conducive to enhancing the presentation of the paper. This work was in part by the National Nature Science Foundation of China under Grants (Nos. 61976244 and 12001422), the Nature Science Foundation of Shaanxi Province under Grants (Nos. 2021JQ-580 and 2023-JC-YB-597)
Author information
Authors and Affiliations
Contributions
YS: Conceptualization, Methodology, Investigation, Writing-original draft. JW: Methodology, Investigation, Writing—original draft. XH: Writing—Reviewing and Editing.
Corresponding author
Ethics declarations
Competing Interest
The authors declared that they have no conflicts of interest to this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
She, Y., Wu, J. & He, X. An incremental approach to hierarchical feature selection by applying fuzzy rough set technique. Artif Intell Rev 56 (Suppl 2), 2571–2598 (2023). https://doi.org/10.1007/s10462-023-10584-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-023-10584-3