Skip to main content
Log in

Semi-supervised learning for k-dependence Bayesian classifiers

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Bayesian network classifiers (BNCs) are powerful tools for graphically encoding the dependency relationships among variables in a directed acyclic graph and reasoning under conditions of uncertainty. Ever increasing data quantity makes ever more urgent the need of BNCs that are highly scalable and can perform significantly better in terms of classification. Numerous approaches have been proposed to mine conditional dependencies among attributes implicated in labeled training data under the framework of supervised learning, whereas the specific characteristics of unlabeled testing instances receive less attention. That may lead to overfitting and degradation in classification performance. In this paper, we argue that the knowledge learned from labeled training dataset and that from unlabeled testing instance are complementary in nature. The testing instance is pre-assigned with any possible label to make it complete, then log-likelihood function is introduced and redefined to measure the extents to which the learned BNC fits training or testing data. Heuristic search strategy is applied to learn two kinds of arbitrary k-dependence BNCs (general BNC for modeling training dataset and local BNC for modeling testing instance), which will work as an ensemble to make the final prediction under the framework of semi-supervised learning. The experimental evaluation on 40 publicly available datasets from the UCI machine learning repository reveals that the proposed algorithm achieves competitive classification performance compared with state-of-the-art BNCs and their variants, such as CFWNB, WATAN, FKDB, SKDB and IWAODE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Saadatfar H, Khosravi S, Joloudari J, Mosavi A, Shamshirband S (2020) A new K-Nearest neighbors classifier for big data based on efficient data pruning. Mathematics 8(2):286–302

    Article  Google Scholar 

  2. Shao Y, Deng N, Yang Z, Chen W, Wang Z (2012) Probabilistic outputs for twin support vector machines. Knowl Based Syst 33:145–151

    Article  Google Scholar 

  3. Tanju O, Kalaylioglu Z (2018) A cluster tree based model selection approach for logistic regression classifier. J Stat Comput Simul 88:1394–1414

    Article  MathSciNet  Google Scholar 

  4. Zhang Y, Lu S, Zhou X, Yang M, Wu L, Liu B, Phillips P, Wang S (2016) Comparison of machine learning methods for stationary wavelet entropy-based multiple sclerosis detection: decision tree, k-nearest neighbors, and support vector machine. Simulation 92:861–871

    Article  Google Scholar 

  5. Liu Y, Wang L, Mammadov M (2020) Learning semi-lazy Bayesian network classifier under the c.i.i.d assumption. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2020.106422

  6. Liu L, Peng T (2014) Clustering-based method for positive and unlabeled text categorization enhanced by improved tfidf. J Inf Sci Eng 30(5):1463–1481

    Google Scholar 

  7. Han J, Zuo W, Liu L, Xu Y, Peng T (2016) Building text classifiers using positive, unlabeled and outdated examples. Concurr Comput Practice Exp 28(13):3691–3706

    Article  Google Scholar 

  8. Zheng F, Webb G, Suraweera P, Zhu L (2013) Subsumption resolution: an efficient and effective technique for semi-naive Bayesian learning. Machine learning 87:93–125

    Article  MathSciNet  Google Scholar 

  9. Duan Z, Wang L, Chen S, Sun M (2020) Instance-based weighting filter for superparent one-dependence estimators. Knowl Based Syst 151:106085

    Article  Google Scholar 

  10. Wang L, Wang G, Duan Z, Lou H, Sun M (2019) Optimizing the topology of Bayesian network classifiers by applying conditional entropy to mine causal relationships between attributes. IEEE Access 7(2):134271–134279

    Article  Google Scholar 

  11. Maillo J, Garcia S, Luengo J, Herrera F, Triguero I (2020) Fast and scalable approaches to accelerate the fuzzy k-Nearest neighbors classifier for big data. IEEE Trans Fuzzy Syst 28:874–886

    Article  Google Scholar 

  12. Zhang Y, Wang L, Duan Z, et al. (2019) Structure learning of Bayesian network based on adaptive thresholding. Entropy 21(7):665–691

    Article  Google Scholar 

  13. Jiang L, Zhang L, Yu L, et al. (2019) Class-specific attribute weighted naive Bayes. Pattern Recognit 88:321–330

    Article  Google Scholar 

  14. Jiang L, Zhang H, Cai Z (2008) A novel Bayes model: Hidden naive Bayes. IEEE Trans Knowl Data Eng 21(6):1361–1371

    Google Scholar 

  15. Jiang L, Zhang L, Li C, Wu J (2018) A correlation-based feature weighting filter for naive Bayes. IEEE Trans Knowl Data Eng 31:201–213

    Article  Google Scholar 

  16. Alhussan A, El Hindi K (2016) Selectively fine-tuning Bayesian network learning algorithm. Int J Pattern Recognit Artif Intell 30:165–182

    Article  MathSciNet  Google Scholar 

  17. Long Y, Wang L, Sun M (2019) Structure extension of tree-augmented Naive Bayes. Entropy 21(8):721–746

    Article  MathSciNet  Google Scholar 

  18. Jiang L, Cai Z, Wang D, Zhang H (2012) Improving tree augmented naive bayes for class probability estimation. Knowl Based Syst 26:239–245

    Article  Google Scholar 

  19. Sahami M (1996) Learning limited dependence Bayesian classifiers. In: Proceedings of the second international conference on knowledge discovery and data mining, vol 96, pp 335–338

  20. Martinez A, Webb G, Chen S, Zaidi N (2016) Scalable learning of Bayesian network classifiers. J Mach Learn Res 17(2):1515–1549

    MathSciNet  MATH  Google Scholar 

  21. Chen S, Martinez A, Webb G, Wang L (2017) Selective AnDE for large data learning: a low-bias memory constrained approach. Knowl Inf Syst 50(2):475–503

    Article  Google Scholar 

  22. Jiang L, Zhang H, Cai Z, Wang D (2012) Weighted average of one-dependence estimators. J Exp Theor Artif Intell 24(7):219–230

    Article  Google Scholar 

  23. Chen S, Martinez A, Webb G, Wang L (2016) Sample-based attribute selective AnDE for large data. IEEE Trans Knowl Data Eng 29(1):172–185

    Article  Google Scholar 

  24. He H, Han D, Dezert J (2020) Disagreement based semi-supervised learning approaches with belief functions. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2019.105426

  25. Blum A, Chawla S (2018) Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the 18th international conference on machine learning, pp 2143–2161

  26. Zhou D, Hofmann T, Scholkopf B (2004) Semi-supervised learning on directed graphs. In: Proceedings of the Advances in neural information processing systems, pp 1633-1640

  27. Jiang J (2015) Stochastic block model and exploratory analysis in signed networks. Phys Rev E 91:628–645

    Article  Google Scholar 

  28. Chen Y, Wang X, Yuan B, Tang B (2014) Overlapping community detection in networks with positive and negative links. J Stat Mech Theory Exp 3:1272–1284

    Google Scholar 

  29. Yang B, Liu X, Li Y, Zhao X (2017) Stochastic blockmodeling and variational Bayes learning for signed network analysis. IEEE Trans Knowl Data Eng 29:2026–2039

    Article  Google Scholar 

  30. Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the international conference on machine learning, pp 29:2026–2039

  31. Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Proceedings of the advances in neural information processing systems, pp 368–374

  32. Zheng F, Webb G, Suraweera P, Zhu L (2012) Subsumption resolution: an efficient and effective technique for semi-naive Bayesian learning. Mach Learn 87(2):93–125

    Article  MathSciNet  Google Scholar 

  33. Zaidi N, Webb G (2013) Fast and effective single pass Bayesian learning. In: Proceedings of the pacific-asia conference on knowledge discovery and data mining, pp 149–160

  34. Yu L, Jiang L, Wang D, Zhang L (2017) Attribute value weighted average of one-dependence estimators. Entropy 19(3): 501

    Article  Google Scholar 

  35. Barutcuoglu Z, Alpaydin E (2003) A comparison of model aggregation methods for regression. In: Proceedings of the artificial neural networks and neural information processing, pp 76–83

  36. Liu Y, Wang L, Mammadov M (2021) Hierarchical Independence Thresholding for learning Bayesian network classifiers. Knowledge-Based Systems 2021(212): 106627

  37. Bache K, Lichman M UCI Machine Learning Repository, Available online: https://archive.ics.uci.edu/ml/datasets.html

  38. Fayyad U, Irani K (1993) Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the Thirteenth international joint conference on artificial intelligence, pp 155–169

  39. Duda R, Hart P, Stork D (2012) Pattern classification,2nd Edition

  40. Kohavi R, Wolpert DH (1996) Bias plus variance decomposition for zero-one loss functions

  41. Brain D, Webb G (1999) On the effect of dataset size on bias and variance in classification learning

Download references

Acknowledgements

The authors would like to thank the editor and the anonymous reviewers for their insightful comments and suggestions. And this work was supported by the National Key Research and Development Program of China (No. 2019YFC1804804) and the Scientific and Technological Developing Scheme of Jilin Province (No. 20200201281JC).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to XinHao Zhang.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Table 6 Experimental results of zero-one loss
Table 7 Experimental results of bias
Table 8 Experimental results of variance

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Zhang, X., Li, K. et al. Semi-supervised learning for k-dependence Bayesian classifiers. Appl Intell 52, 3604–3622 (2022). https://doi.org/10.1007/s10489-021-02531-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02531-y

Keywords

Navigation