Skip to main content
SpringerLink
Log in

Decision function with probability feature weighting based on Bayesian network for multi-label classification

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The multi-label classification problem involves finding a multi-valued decision function that predicts an instance to a vector of binary classes. Two methods are widely used to build multi-label classifiers: the binary relevance method and the chain classifier. Both can induce a polynomial multi-valued decision function by using Bayesian network-augmented naive Bayes classifiers as base models. In this paper, we propose a feature weighting approach to improve the classification accuracy of the decision function. This method, called probability feature weighting, estimates the conditional probability of the positive class through deep computation of the frequency ratio of features from the training data. Moreover, we identify irrelevant variables in terms of probability to simplify the decision function. Experiments showed that the decision function with a probability feature weighting rarely degrades the quality of the model and drastically improves it in many cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.imdb.com/interfaces/plain.

  2. http://mulan.sourceforge.net.

References

  1. Bielza C, Li G, Larranga P (2011) Multi-dimensional classification with Bayesian network. Int J Approx Reason 52:705–727

    Article  MathSciNet  Google Scholar 

  2. Zhang ML, Zhou ZH (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  3. Agrawal S, Agrawal J, Kaur S, Sharma S (2016) A comparative study of fuzzy PSO and fuzzy SVD-based RBF neural network for multi-label classification. Neural Comput Appl. https://doi.org/10.1007/s00521-016-2446-x

    Article  Google Scholar 

  4. Vens C, Struyf J, Schietgat L (2008) Decision trees for hierarchical multi-label classification. Mach Lean 73:185–214. https://doi.org/10.1007/s10994-008-5077-3

    Article  Google Scholar 

  5. Blockeel H, Schietgat L, Struyf J, Dzeroki S et al (2006) Decision tree for hierarchical multilabel classification: a case study in functional genomics. Springer, Berlin, pp 18–29

    Google Scholar 

  6. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Pattern Recognit 37:1757–1771

    Article  Google Scholar 

  7. Godbole S, Sarawagi S (2004) Discriminative methods for multi-labeled classification. Springer, Berlin, pp 22–30

    Google Scholar 

  8. Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172:1897–1916

    Article  MathSciNet  Google Scholar 

  9. Tsoumakas G, Vlahavas I (2007) Random \(k\)-labelsets: an ensemble method for multilabel classification. Machine learning ECML 2007. Lecture notes in computer science, vol 4701. Springer, Berlin, Heidelberg

  10. Schapire RE, Singer Y (2000) Boos Texter: a boosting-based system for text categorization. Mach Learn 39:135–168. https://doi.org/10.1023/A:1007649029923

    Article  MATH  Google Scholar 

  11. Zhang M-L, Zhou Z-H (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recognit 40(7):2038–2048

    Article  Google Scholar 

  12. Sucar LE, Bielza C, Morales EF et al (2014) Multi-label classification with Bayesian network-based chain classifiers. Pattern Recognit Lett 41:14–22

    Article  Google Scholar 

  13. Read J, Pfahringer B, Holmes G et al (2011) Classifier chains for multi-label classification. Mach Learn 85:333–359. https://doi.org/10.1007/s10994-011-5256-5

    Article  MathSciNet  Google Scholar 

  14. Varando G, Bielza C, Larrañaga P (2014) Expressive power of binary relevance and chain classifiers based on Bayesian networks for multi-label classification. Springer, Berlin, pp 519–534

    MATH  Google Scholar 

  15. Varando G, Bielza C, Larraãnaga P (2016) Decision function for chain classifiers based on Bayesian network for multi-label classification. Int J Approx Reason 68:164–178

    Article  MathSciNet  Google Scholar 

  16. Jiang L, Li C, Wang S et al (2016) Deep feature weighting for naive Bayes and its application to text classification. Eng Appl Artif Intell 52:26–39

    Article  Google Scholar 

  17. Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29(2–3):131–163

    Article  Google Scholar 

  18. Ouali A, Cherif AR, Krebs M-O (2006) Data mining based Bayesian networks for best classification. Comput Stat Data Anal 51(2):1278–1292

    Article  MathSciNet  Google Scholar 

  19. Varando G, Bielza C, Larrañaga P (2015) Decision boundary for disctete Bayesian network classifiers. J Mach Learn Res 16:2725–2749

    MathSciNet  MATH  Google Scholar 

  20. O’Donnell R, Servedio RA (2010) New degree bounds for polynomial threshold functions. Combinatorica 30(3):327–358. https://doi.org/10.1007/s00493-010-2173-3

    Article  MathSciNet  MATH  Google Scholar 

  21. Tan J, Zhang Z, Zhen L et al (2012) Adaptive feature selection via a new version of support vector machine. Neural Comput Appl. https://doi.org/10.1007/s00521-012-1018-y

    Article  Google Scholar 

  22. Hall MA (2000) Correlation-based feature selection for discrete and numeric class. Machine learning. In: Proceedings of the seventeenth international conference on machine learning, vol 1. Morgan Kaufmann Publishers Inc, pp 359–366

  23. Wang S, Jiang L, Li C (2014) A CFS-based feature weighting approach to native Bayes text classifiers. Springer, Berlin, pp 555–562

    Google Scholar 

  24. Li Z, Lu W, Sun Z, Xing W (2016) A parallel feature selection method study for text classification. Neural Comput Appl. https://doi.org/10.1007/s00521-016-2351-3

    Article  Google Scholar 

  25. Hall M (2007) A decision tree-based attribute weighting filter for native Bayes. Knowl Based Syst 20:120–126

    Article  Google Scholar 

  26. Jiang L, Cai Z, Wang D et al (2012) Improving Tree augmented Naive Bayes for class probability estimation. Knowl Based Syst 26:239–245

    Article  Google Scholar 

  27. Tsai C-J, Lee C-L, Yang W-P (2008) A discretization algorithm based on class-attribute contingency coefficient. Inf Sci 178:714–731

    Article  Google Scholar 

  28. Tsoumakas G, Katakis L (2007) Multi-label classification: an overview. Int J Data Wareh Min 3(3):1–13

    Article  Google Scholar 

  29. Read J, Bielza C, Larrañaga P (2014) Multi-dimensional classification with super-classes. IEEE Trans Knowl Data Eng 26(7):1720–1733

    Article  Google Scholar 

  30. de Waal PR, van der Gaag LC (2007) Inference and Learning in multi-dimensional Bayesian network classifiers. Springer, Berlin, pp 501–511

    MATH  Google Scholar 

Download references

Acknowledgements

The authors thank the editor and the anonymous reviewers for helpful comments and suggestions. This work was supported by the National Natural Science Foundation of China (Grant No. 61573266).

Author information

Authors and Affiliations

  1. School of Mathematics and Statistics, Xidian University, Xi’an, 710126, Shaanxi, China

    Youlong Yang & Mengxiao Ding

Authors
  1. Youlong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mengxiao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mengxiao Ding.

Ethics declarations

Conflict of interest

We wish to confirm that there are no known conflicts of interest associated with this publication. We also confirm that the manuscript has been read and approved by all named authors and that there are no other persons who satisfied the criteria for authorship but are not listed.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Ding, M. Decision function with probability feature weighting based on Bayesian network for multi-label classification. Neural Comput & Applic 31, 4819–4828 (2019). https://doi.org/10.1007/s00521-017-3323-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3323-y

Keywords

Search

Navigation