Skip to main content
SpringerLink
Log in

Hierarchical multilabel classification by exploiting label correlations

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Hierarchical multilabel classification (HMC) aims to classify the complex data such as text with multiple topics and image with multiple semantics, in which the multiple labels are organized in hierarchical structures such as trees and direct acyclic graphs (DAG). To reduce the computational complexity, HMC methods generally assume that the class labels of different branches in a hierarchical structure are conditional independent. However, these class-independent HMC methods neglect the correlation between labels and thereby the precision of classification is affected. To tackle the problem, in this paper we propose a hierarchical multilabel classification method with class label correlation (HMC-CLC) which exploits the label correlations of different branches to benefit the discrimination of HMC. Specifically, in the training stage, for each label in the hierarchy, we use feature incremental learning to encode the labels of different branches into the input space. Based on this, the label correlations of different branches are reflected by the weights of classification model in corresponding dimensions. Then in the test stage, considering that the different samples have different label distributions, we propose a greedy label selection method to dynamically decide the correlated labels of different branches for each label. Therefore, for the same label in the hierarchy, the correlated labels could be different in different samples. Experimental results on a number of real-world data sets show that the proposed method outperforms the state-of-the-art HMC methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bi W, Kwok JT (2015) Bayes-optimal hierarchical multilabel classification. IEEE Trans Knowl Data Eng 27(11):2907–2918

    Article  Google Scholar 

  2. Bi W, Kwok JT (2012) Hierarchical multilabel classification with minimum Bayes risk. In: 2012 IEEE 12th International Conference on Data Mining, pp 101–110. IEEE

  3. Bielza C, Li G, Larranaga P (2011) Multi-dimensional classification with Bayesian networks. Int J Approx Reason 52(6):705–727

    Article  MathSciNet  Google Scholar 

  4. Cerri R, Barros CR, de Carvalho ACPLF, Jin Y (2016) Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC Bioinform 17(1):373

    Article  Google Scholar 

  5. Cesa-Bianchi N, Gentile C, Zaniboni L (2006) Incremental algorithms for hierarchical classification. J Mach Learn Res 7(Jan):31–54

    MathSciNet  MATH  Google Scholar 

  6. Cesa-Bianchi N, Gentile C, Zaniboni L (2006) Hierarchical classification: combining Bayes with svm. In: Proceedings of the 23rd international conference on Machine learning, pp 177–184. ACM

  7. Cesa-Bianchi N, Valentini G (2009) Hierarchical cost-sensitive algorithms for genome-wide gene function prediction. In: Machine Learning in Systems Biology, pp 14–29

  8. Chen B, Huang X, Xiao L, Cai Z, Jing L (2020) Hyperbolic interaction model for hierarchical multi-label classification. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7–12, 2020, pp 7496–7503. AAAI Press

  9. Chen T, Wang Z, Li G, Lin L (2018) Recurrent attentional reinforcement learning for multi-label image recognition. In: Thirty-Second AAAI Conference on Artificial Intelligence, pp 6730–6737

  10. Dembczynski K, Cheng W, Hüllermeier E (2010) Bayes optimal multilabel classification via probabilistic classifier chains. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 279–286

  11. Feng S, Fu P, Zheng W (2018) A hierarchical multi-label classification method based on neural networks for gene function prediction. Biotechnol Biotechnol Equip 32(6):1613–1621

    Article  Google Scholar 

  12. Huang S, Zhou Z (2012) Multi-label learning by exploiting label correlations locally. In: Twenty-Sixth AAAI Conference on Artificial Intelligence

  13. Huang W, Chen E, Liu Q, Chen Y, Huang Z, Liu Y, Zhao Z, Zhang D, Wang S (2019) Hierarchical multi-label text classification: An attention-based recurrent network approach. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1051–1060

  14. Jia X, Zhu S, Li W (2020) Joint label-specific features and correlation information for multi-label learning. J Comput Sci Technol 35:247–258

    Article  Google Scholar 

  15. Kongsorot Y, Horata P, Musikawan P, Sunat K (2019) Kernel extreme learning machine based on fuzzy set theory for multi-label classification. Int J Mach Learn Cybern 10(5):979–989

    Article  Google Scholar 

  16. Lin J, Su Q, Yang P, Ma S, Sun X (2018) Semantic-unit-based dilated convolution for multi-label text classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 4554–4564

  17. Liu SM, Chen JH (2015) A multi-label classification based approach for sentiment classification. Expert Syst Appl 42(3):1083–1093

    Article  Google Scholar 

  18. Liu H, Ma Z, Han J, Chen Z, Zheng Z (2018) Regularized partial least squares for multi-label learning. Int J Mach Learn Cybern 9(2):335–346

    Article  Google Scholar 

  19. Platt JC (1999) Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 10(3):61–74

    Google Scholar 

  20. Punera K, Rajan S, Ghosh J (2005) Automatically learning document taxonomies for hierarchical classification. In: Special interest tracks and posters of the 14th international conference on World Wide Web, pp 1010–1011. ACM

  21. Ramírez-Corona M, Sucar L, Morales EF (2016) Hierarchical multilabel classification based on path evaluation. Int J Approx Reason 68:179–193

    Article  Google Scholar 

  22. Read J, Pfahringer B, Holmes G, Frank E (2011) Classifier chains for multi-label classification. Mach Learn 85(3):333

    Article  MathSciNet  Google Scholar 

  23. Rios A, Kavuluru R (2018) Few-shot and zero-shot multi-label learning for structured label spaces. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3132–3142

  24. Silla CN Jr, Freitas AA (2011) A survey of hierarchical classification across different application domains. Data Min Knowl Dis 22(1–2):31–72

    Article  MathSciNet  Google Scholar 

  25. Sucar LE, Bielza C, Morales EF, Hernandez-Leal P, Zaragoza JH, Larrañaga P (2014) Multi-label classification with Bayesian network-based chain classifiers. Pattern Recogn Lett 41:14–22

    Article  Google Scholar 

  26. Tsoumakas G, Katakis I (2007) Multi-label classification: an overview. Int J Data Warehous Min (IJDWM) 3(3):1–13

    Article  Google Scholar 

  27. Vens C, Struyf J, Schietgat L, Džeroski S, Blockeel H (2008) Decision trees for hierarchical multi-label classification. Mach Learn 73(2):185

    Article  Google Scholar 

  28. Wen L, Luo C, Wu W, Min F (2020) Multi-label symbolic value partitioning through random walks. Neurocomputing 387:195–209

    Article  Google Scholar 

  29. Yeung D, Wang X, Chen D (2008) Preface: recent advances in granular computing. Inform Sci Inform Comput Sci Intell Syst Appl 178(16):3161–3162

    Google Scholar 

  30. Yi W, Lu M, Liu Z (2011) Multi-valued attribute and multi-labeled data decision tree algorithm. Int J Mach Learn Cybern 2(2):67–74

    Article  Google Scholar 

  31. Yu D, Fu B, Xu G, Qin A (2019) Constrained nonnegative matrix factorization-based semi-supervised multilabel learning. Int J Mach Learn Cybern 10(5):1093–1100

    Article  Google Scholar 

  32. Zaragoza JH, Sucar L, Morales EF, Bielza C, Larrañaga P (2011) Bayesian chain classifiers for multidimensional classification. In: International Joint Conference on Artificial Intelligence, pp. 2192–2197

  33. Zhang C, Li D, Liang J (2018) Hesitant fuzzy linguistic rough set over two universes model and its applications. Int J Mach Learn Cybern 9(4):577–588

    Article  Google Scholar 

  34. Zhang C, Li D, Liang J (2020) Multi-granularity three-way decisions with adjustable hesitant fuzzy linguistic multigranulation decision-theoretic rough sets over two universes. Inform Sci 507:665–683

    Article  MathSciNet  Google Scholar 

  35. Zhang M, Li Y, Liu X, Geng X (2018) Binary relevance for multi-label learning: an overview. Front Comput Sci 12(2):191–202

    Article  Google Scholar 

  36. Zhang M, Wu L (2014) Lift: multi-label learning with label-specific features. IEEE Trans Pattern Anal Mach Intell 37(1):107–120

    Article  Google Scholar 

  37. Zhang J, Wu Q, Shen C, Zhang J, Lu J (2018) Multilabel image classification with regional latent semantic dependencies. IEEE Trans Multimed 20(10):2801–2813

    Article  Google Scholar 

  38. Zhang M, Zhou Z (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  39. Zhang M, Zhou Z (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  40. Zhang M, Zhang K (2010) Multi-label learning by exploiting label dependency. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 999–1008. ACM

  41. Zhu Y, Kwok JT, Zhou Z (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094

    Article  Google Scholar 

  42. Zhu Y, Ting K, Zhou Z (2018) Multi-label learning with emerging new labels. IEEE Trans Knowl Data Eng 30(10):1901–1914

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Nos. 61976134, 61573235, 61672331), Open Project Foundation of Intelligent Information Processing Key Laboratory of Shanxi Province (No. CICIP2018001) and the Key R&D program of Shanxi Province (International Cooperation 201903D421041).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Zhikang Xu, Bofeng Zhang & Xiaodong Yue

  2. School of Computer and Information Technology, Shanxi University, Taiyuan, Shanxi, 030006, China

    Deyu Li

Authors
  1. Zhikang Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bofeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Deyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaodong Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bofeng Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Z., Zhang, B., Li, D. et al. Hierarchical multilabel classification by exploiting label correlations. Int. J. Mach. Learn. & Cyber. 13, 115–131 (2022). https://doi.org/10.1007/s13042-021-01371-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01371-z

Keywords

Search

Navigation