Skip to main content
SpringerLink
Log in

Feature distribution-based label correlation in multi-label classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In multi-label classification, multiple label variables in output space are equally important and can be predicted according to a common set of input variables. To improve the accuracy and efficiency of multi-label learner, measuring and utilizing label correlation is the core breakthrough. Extensive research on label correlation focuses on the co-occurrence or mutual exclusion frequency of label values in output space. In this paper, to handle the multi-label learning tasks, a novel method, named FL-MLC, is proposed by considering the influence of feature-label dependencies on inter-label correlations. In order to describe the intrinsic relationship between feature variable and label variable, the discriminant weight of any feature to label is first defined. Therefore, the concept of feature distribution for inputs on label is proposed to reflect the discriminant weights of features to the label. The corresponding calculation process is also designed based on multiple kernel learning and kernel alignment. Furthermore, the feature distributions on different labels are integrated into the feature distribution-based label correlation by using two different aggregation strategies. Obviously, arbitrary label variables with highly similar feature distributions have strong relevance. Thus, the feature distribution-based label correlation is applied to adjust the distance between the parameters for different labels in the predictive learner of FL-MLC method. Finally, the experimental results on twelve real-world datasets demonstrate that our methods achieves good effectiveness and versatility for multi-label classification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. http://mulan.sourceforge.net/datasets.html.

References

  1. Beavis B, Dobbs IM (1990) Static optimization, optimization and stability theory for economic analysis. Cambridge University Press, Cambridge, pp 40–46

    MATH  Google Scholar 

  2. Cortes C, Mohri M, Rostamizadeh A (2012) Algorithms for learning kernels based on centered alignment. J Mach Learn Res 13(1):795–828

    MathSciNet  MATH  Google Scholar 

  3. Ralescu D, Adams G (1980) The fuzzy integral. J Math Anal Appl 75(2):562–570

    Article  MathSciNet  Google Scholar 

  4. Bach F.R, Lanckriet G.R.G, Jordan M.I (2004) Multiple kernel learning, conic duality, and the SMO algorithm, In International Conference on Machine Learning,

  5. Sun FM, Tang JH, Li HJ et al (2014) Multilabel image categorization with sparse factor representation. IEEE Trans Image Process 23(3):1028–1037

    Article  MathSciNet  Google Scholar 

  6. Huang J, Li GR, Huang QM et al (2016) Learning label-specific features and class-dependent labels for multi-label classification. IEEE Trans Knowl Data Eng 28(12):3309–3323

    Article  Google Scholar 

  7. Huang J, Li GR, Huang QM et al (2015) Learning label specific features for multi-label classification. IEEE International Conference on Data Mining 181–190

  8. Huang J, Li GR, Wang SH et al (2017) Multi-label classification by exploiting local positive and negative pairwise label correlation. Neurocomputing 257:164–174

    Article  Google Scholar 

  9. Lee JS, Kim DW (2017) SCLS: Multi-label feature selection based on scalable criterion for large label set. Pattern Recognit 66:342–352

    Article  MathSciNet  Google Scholar 

  10. Lanckriet GRG, Cristianini N, Bartlett P et al (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5(1):27–72

    MathSciNet  MATH  Google Scholar 

  11. Kira K, Rendell LA (1992) The feature selection problem: traditional methods and a new algorithm. In: Proceedings of Ninth National Conference on Artificial Intelligence 129–134

  12. Schietgat L, Vens C, Struyf J et al (2010) Predicting gene function using hierarchical multi-label decision tree ensembles. BMC Bioinform 11(1):2

    Article  Google Scholar 

  13. Chen LL, Chen DG, Wang H (2019) Alignment based kernel selection for multi-label learning. Neural Process Lett 49(3):1157–1177

    Article  MathSciNet  Google Scholar 

  14. Chen LL, Chen DG (2019) Alignment based feature selection for multi-label learning. Neural Process Lett 50:2323–2344

    Article  Google Scholar 

  15. Chen LL, Chen DG, Wang H (2019) Fuzzy kernel alignment with application to attribute reduction of heterogeneous data. IEEE Trans Fuzzy Syst 27(7):1469–1478

    Article  Google Scholar 

  16. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  Google Scholar 

  17. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl ata Eng 26(8):1819–1837

    Article  Google Scholar 

  18. Zhang ML, Zhou ZH (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  19. Zhang ML, Wu L (2015) LIFT: Multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  Google Scholar 

  20. Friedman M (1940) A comparison of alternative tests of significance for the problem of m ranking. Ann Math Stat 11:86–92

    Article  MathSciNet  Google Scholar 

  21. Boutell MR, Luo J, Shen X et al (2004) Learning multi-label scene classification. Pattern Recognit 37(9):1757–1771

    Article  Google Scholar 

  22. Sugeno M (1974) Theory of fuzzy integrals and its applications, PH.D. dissertation, Tokyo Institute of Technology,

  23. Gönen M, Alpaydın E (2012) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268

    MathSciNet  MATH  Google Scholar 

  24. Cristianini N, Kandola J, Elisseeff A et al (2006) On kernel target alignment, innovations in machine. Learning 194:205–256

    Google Scholar 

  25. Dunn OJ (1961) Multiple comparisons among means. J Am Stat Assoc 56:52–64

    Article  MathSciNet  Google Scholar 

  26. Schapire RE, Singer Y (2000) BoosTexter: a boosting-based system for text categorization. Mach Learn 39(2–3):135–168

    Article  Google Scholar 

  27. Huang S.J (2014) Research on label relationship exploitation in multi-label learning, PH.D. dissertation, Nanjing University,

  28. Wang T, Zhao D, Tian S (2015) An overview of kernel alignment and its applications. Artificial Intell Rev 43:179–192

    Article  Google Scholar 

  29. Coleman TF, Li YY (1992) A reflective newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J Opt 6(4):1–37

    MathSciNet  Google Scholar 

  30. Cheng W, Hüllermeier E (2009) Combining instance-based learning and logistic regression for multilabel classification. Mach Learn 76(2/3):211–225

    Article  Google Scholar 

  31. Wang XZ, Wang R, Feng HM et al (2014) A new approach to classifier fusion based on upper integral. IEEE Trans Cybernet 44(5):620–635

    Article  Google Scholar 

  32. Wang XZ, Zhai JH, Lu SX (2008) Induction of multiple fuzzy decision trees based on rough set technique. Inform Sci 178(16):3188–3202

    Article  MathSciNet  Google Scholar 

  33. Zhen XT, Yu MY, He XF et al (2018) Multi-target regression via robust low-rank learning. IEEE Trans Pattern Anal Mach Intell 40(2):497–504

    Article  Google Scholar 

  34. Li YW, Lin YJ, Liu JH et al (2018) Feature selection for multi-label learning based on kernelized fuzzy rough sets. Neurocomputing 318:271–286

    Article  Google Scholar 

  35. Lin YJ, Hu QH, Liu JH et al (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103

    Article  Google Scholar 

  36. Lin YJ, Hu QH, Liu JH et al (2017) Streaming feature selection for multilabel learning based on fuzzy mutual information. IEEE Trans Fuzzy Syst 25(6):1491–1507

    Article  Google Scholar 

  37. Barutcuoglu Z, Schapire RE, Troyanskaya OG (2006) Hierarchical multi-label prediction of gene function. Bioinformatics 22(7):830–836

    Article  Google Scholar 

Download references

Acknowledgements

This paper is supported by grants of National Natural Science Foundation of China (12071131, 62076 088), the fund of North China Electric Power University and Fundamental Research Funds for the Central Universities (JB2019125).

Author information

Authors and Affiliations

  1. School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

    Xiaoya Che

  2. School of Mathematics and Physics, North China Electric Power University, Beijing, 102206, China

    Degang Chen

  3. College of Mathematics and Information Science, Hebei Normal University, Shijiazhuang, 050016, China

    Jusheng Mi

Authors
  1. Xiaoya Che
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Degang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jusheng Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Degang Chen.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Che, X., Chen, D. & Mi, J. Feature distribution-based label correlation in multi-label classification. Int. J. Mach. Learn. & Cyber. 12, 1705–1719 (2021). https://doi.org/10.1007/s13042-020-01268-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01268-3

Keywords

Search

Navigation