Skip to main content
SpringerLink
Log in

Multi-label feature selection using geometric series of relevance matrix

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Multi-label learning deals with data in which an instance may belong to multiple class labels simultaneously. Due to the increasing applications of multi-label data, especially in text and image classification problems, multi-label feature selection has attracted much attention in recent years. The main problem of the existing multi-label feature selection algorithms is that they are not able to consider all possible subsets of feature space in evaluating a candidate feature, because of computational concerns. Therefore they ignore much of the useful higher-order information, hidden in larger subsets. This paper proposes a new approach to address the higher-order relevance and redundancy analysis. The proposed method generates an adjacency matrix using pairwise features relevance and redundancy then uses the geometric series of the matrix as a part of the evaluation function. In this way it is able to consider all possible subsets of feature space in evaluating a single candidate feature. Experimental results on five benchmarks from different domains clearly demonstrate the superiority of the proposed method against seven state-of-the-art multi-label feature selection algorithms. The codes are available at https://github.com/Sadegh28/Inf-MLFS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bidgoli AA, Ebrahimpour-Komleh H, Rahnamayan S (2021) Reference-point-based multi-objective optimization algorithm with opposition-based voting scheme for multi-label feature selection. Inf Sci 547:1–17

    Article  MathSciNet  Google Scholar 

  2. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recogn 37(9):1757–1771

    Article  Google Scholar 

  3. Briggs F, Huang Y, Raich R, Eftaxias K, Lei Z, Cukierski W, Hadley SF, Hadley A, Betts M, Fern XZ, et al (2013) The 9th annual mlsp competition: New methods for acoustic classification of multiple simultaneous bird species in a noisy environment. In 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–8. IEEE

  4. Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Electr Eng 40(1):16–28

    Article  Google Scholar 

  5. Cover TM, Thomas JA (2006) Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, USA

    Google Scholar 

  6. Dionisio A, Menezes R, Mendes DA (2004) Mutual information: a measure of dependency for nonlinear time series. Phys A 344(1–2):326–329

    Article  MathSciNet  Google Scholar 

  7. Diplaris S, Tsoumakas G, Mitkas PA, Vlahavas I (2005) Protein classification with multiple algorithms. In: Panhellenic Conference on Informatics, pages 448–456. Springer

  8. Doquire G, Verleysen M (2011) Feature selection for multi-label classification problems. In: International Work-Conference on Artificial Neural Networks, pages 9–16. Springer

  9. Dumais ST (2004) Latent semantic analysis. Ann Rev Inf Sci Technol 38(1):188–230

    Article  Google Scholar 

  10. Eskandari S (2022) Pyit-mlfs: a python-based information theoretical multi-label feature selection library. International Journal of Research in Industrial Engineering

  11. Gharroudi O, Elghazel H, Aussem A (2014) A comparison of multi-label feature selection methods using the random forest paradigm. In: Canadian Conference on Artificial Intelligence, pages 95–106. Springer

  12. Gretton A, Bousquet O, Smola A, Schölkopf B (2005) Measuring statistical dependence with hilbert-schmidt norms. In: International Conference on Algorithmic Learning Theory, pages 63–77. Springer

  13. Gu Q, Li Z, Han J (2011) Correlated multi-label feature selection. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pages 1087–1096

  14. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  15. Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2020) Mfs-mcdm: Multi-label feature selection using multi-criteria decision making. Knowl-Based Syst 206:106365

    Article  Google Scholar 

  16. He Z, Yu W (2010) Stable feature selection for biomarker discovery. Comput Biol Chem 34(4):215–225

    Article  Google Scholar 

  17. Hong R, Wang M, Gao Y, Tao D, Li X, Wu X (2013) Image annotation by multiple-instance learning with discriminative feature mapping and selection. IEEE Trans Cybern 44(5):669–680

    Article  Google Scholar 

  18. Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Pattern Recogn 104:107344

    Article  Google Scholar 

  19. Huang J, Li G, Huang Q, Wu X (2017) Joint feature selection and classification for multilabel learning. IEEE Trans Cybern 48(3):876–889

    Article  Google Scholar 

  20. Hubbard JH, Hubbard BB (2015) Vector calculus, linear algebra, and differential forms: a unified approach. Matrix Editions

  21. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. In IJCAI 16:1627–33

  22. Kashef S, Nezamabadi-pour H, Nikpour B (2018) Multilabel feature selection: a comprehensive review and guiding experiments. Wiley Interdiscip Rev Data Min Knowl Dis 8(2):e1240

    Google Scholar 

  23. Khinchin AY (2013) Mathematical foundations of information theory. Courier Corporation

  24. Lee J, Kim D-W (2013) Feature selection for multi-label classification using multivariate mutual information. Pattern Recogn Lett 34(3):349–357

    Article  Google Scholar 

  25. Lee J, Kim D-W (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42(4):2013–2025

    Article  Google Scholar 

  26. Lee J, Kim D-W (2017) Scls: multi-label feature selection based on scalable criterion for large label set. Pattern Recogn 66:342–352

    Article  MathSciNet  Google Scholar 

  27. Li L, Liu H, Ma Z, Mo Y, Duan Z, Zhou J, Zhao J (2014) Multi-label feature selection via information gain. In International Conference on Advanced Data Mining and Applications, pages 345–355. Springer

  28. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103

    Article  Google Scholar 

  29. Liu SM, Chen J-H (2015) A multi-label classification based approach for sentiment classification. Expert Syst Appl 42(3):1083–1093

    Article  Google Scholar 

  30. Lv S, Shi S, Wang H, Li F (2021) Semi-supervised multi-label feature selection with adaptive structure learning and manifold learning. Knowl-Based Syst 214:106757

    Article  Google Scholar 

  31. Nguyen BH, Xue B, Zhang M (2020) A survey on swarm intelligence approaches to feature selection in data mining. Swarm Evol Comput 54:100663

    Article  Google Scholar 

  32. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  33. Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238

    Article  Google Scholar 

  34. Pereira RB, Plastino A, Zadrozny B, Merschmann LH (2018) Categorizing feature selection methods for multi-label classification. Artif Intell Rev 49(1):57–78

    Article  Google Scholar 

  35. Pestian J, Brew C, Matykiewicz P, Hovermale DJ, Johnson N, Cohen KB, Duch W (2007) A shared task involving multi-label classification of clinical free text. In: Biological, translational, and clinical language processing, pages 97–104

  36. Qian W, Huang J, Wang Y, Xie Y (2021) Label distribution feature selection for multi-label classification with rough set. Int J Approx Reason 128:32–55

    Article  MathSciNet  Google Scholar 

  37. Read J (2008) A pruned problem transformation method for multi-label classification. In: Proc. 2008 New Zealand Computer Science Research Student Conference (NZCSRS 2008), volume 143150, page 41

  38. Roffo G, Melzi S, Cristani M (2015) Infinite feature selection. In: Proceedings of the IEEE International Conference on Computer Vision, pages 4202–4210

  39. Shao H, Li G, Liu G, Wang Y (2013) Symptom selection for multi-label data of inquiry diagnosis in traditional chinese medicine. Sci China Inf Sci 56(5):1–13

    Article  MathSciNet  Google Scholar 

  40. Spolaôr N, Cherman EA, Monard MC, Lee HD (2013) A comparison of multi-label feature selection methods using the problem transformation approach. Electron Notes Theor Comput Sci 292:135–151

    Article  Google Scholar 

  41. Spolaôr N, Monard MC, Tsoumakas G, Lee HD (2016) A systematic review of multi-label feature selection and a new method based on label construction. Neurocomputing 180:3–15

    Article  Google Scholar 

  42. Szymański P, Kajdanowicz T (2017) A scikit-based python environment for performing multi-label classification. arXiv preprint arXiv:1702.01460

  43. Trohidis K, Tsoumakas G, Kalliris G, Vlahavas IP et al (2008) Multi-label classification of music into emotions. In ISMIR 8:325–330

    Google Scholar 

  44. Tsoumakas G, Spyromitros-Xioufis E, Vilcek J, Vlahavas I (2011) Mulan: a java library for multi-label learning. J Mach Learn Res 12:2411–2414

    MathSciNet  MATH  Google Scholar 

  45. Tsoumakas G, Vlahavas I (2007) Random k-labelsets: An ensemble method for multilabel classification. In: European Conference on Machine Learning, pages 406–417. Springer

  46. Vergara JR, Estévez PA (2014) A review of feature selection methods based on mutual information. Neural Comput Appl 24(1):175–186

    Article  Google Scholar 

  47. Yang Y-H, Chen HH (2012) Machine recognition of music emotion: a review. ACM Trans Intel Syst Technol (TIST) 3(3):1–30

    Article  Google Scholar 

  48. Yu K, Yu S, Tresp V (2005) Multi-label informed latent semantic indexing. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 258–265

  49. Zhang M-L, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Inf Sci 179(19):3218–3229

    Article  Google Scholar 

  50. Zhang P, Gao W, Hu J, Li Y (2021) Multi-label feature selection based on the division of label topics. Inf Sci 553:129–153

    Article  MathSciNet  Google Scholar 

  51. Zhang P, Liu G, Gao W (2019) Distinguishing two types of labels for multi-label feature selection. Pattern Recogn 95:72–82

    Article  Google Scholar 

  52. Zhang P, Liu G, Gao W, Song J (2021) Multi-label feature selection considering label supplementation. Pattern Recogn 120:108137

    Article  Google Scholar 

  53. Zhang Y, Zhou Z-H (2010) Multilabel dimensionality reduction via dependence maximization. ACM Trans Knowl Discov Data (TKDD) 4(3):1–21

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Guilan, Rasht, Iran

    Sadegh Eskandari

Authors
  1. Sadegh Eskandari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sadegh Eskandari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eskandari, S. Multi-label feature selection using geometric series of relevance matrix. J Supercomput 78, 14402–14418 (2022). https://doi.org/10.1007/s11227-022-04451-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04451-1

Keywords

Search

Navigation