Abstract
Bayesian network classifiers (BNCs) are powerful tools for graphically encoding the dependency relationships among variables in a directed acyclic graph and reasoning under conditions of uncertainty. Ever increasing data quantity makes ever more urgent the need of BNCs that are highly scalable and can perform significantly better in terms of classification. Numerous approaches have been proposed to mine conditional dependencies among attributes implicated in labeled training data under the framework of supervised learning, whereas the specific characteristics of unlabeled testing instances receive less attention. That may lead to overfitting and degradation in classification performance. In this paper, we argue that the knowledge learned from labeled training dataset and that from unlabeled testing instance are complementary in nature. The testing instance is pre-assigned with any possible label to make it complete, then log-likelihood function is introduced and redefined to measure the extents to which the learned BNC fits training or testing data. Heuristic search strategy is applied to learn two kinds of arbitrary k-dependence BNCs (general BNC for modeling training dataset and local BNC for modeling testing instance), which will work as an ensemble to make the final prediction under the framework of semi-supervised learning. The experimental evaluation on 40 publicly available datasets from the UCI machine learning repository reveals that the proposed algorithm achieves competitive classification performance compared with state-of-the-art BNCs and their variants, such as CFWNB, WATAN, FKDB, SKDB and IWAODE.
Similar content being viewed by others
References
Saadatfar H, Khosravi S, Joloudari J, Mosavi A, Shamshirband S (2020) A new K-Nearest neighbors classifier for big data based on efficient data pruning. Mathematics 8(2):286–302
Shao Y, Deng N, Yang Z, Chen W, Wang Z (2012) Probabilistic outputs for twin support vector machines. Knowl Based Syst 33:145–151
Tanju O, Kalaylioglu Z (2018) A cluster tree based model selection approach for logistic regression classifier. J Stat Comput Simul 88:1394–1414
Zhang Y, Lu S, Zhou X, Yang M, Wu L, Liu B, Phillips P, Wang S (2016) Comparison of machine learning methods for stationary wavelet entropy-based multiple sclerosis detection: decision tree, k-nearest neighbors, and support vector machine. Simulation 92:861–871
Liu Y, Wang L, Mammadov M (2020) Learning semi-lazy Bayesian network classifier under the c.i.i.d assumption. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2020.106422
Liu L, Peng T (2014) Clustering-based method for positive and unlabeled text categorization enhanced by improved tfidf. J Inf Sci Eng 30(5):1463–1481
Han J, Zuo W, Liu L, Xu Y, Peng T (2016) Building text classifiers using positive, unlabeled and outdated examples. Concurr Comput Practice Exp 28(13):3691–3706
Zheng F, Webb G, Suraweera P, Zhu L (2013) Subsumption resolution: an efficient and effective technique for semi-naive Bayesian learning. Machine learning 87:93–125
Duan Z, Wang L, Chen S, Sun M (2020) Instance-based weighting filter for superparent one-dependence estimators. Knowl Based Syst 151:106085
Wang L, Wang G, Duan Z, Lou H, Sun M (2019) Optimizing the topology of Bayesian network classifiers by applying conditional entropy to mine causal relationships between attributes. IEEE Access 7(2):134271–134279
Maillo J, Garcia S, Luengo J, Herrera F, Triguero I (2020) Fast and scalable approaches to accelerate the fuzzy k-Nearest neighbors classifier for big data. IEEE Trans Fuzzy Syst 28:874–886
Zhang Y, Wang L, Duan Z, et al. (2019) Structure learning of Bayesian network based on adaptive thresholding. Entropy 21(7):665–691
Jiang L, Zhang L, Yu L, et al. (2019) Class-specific attribute weighted naive Bayes. Pattern Recognit 88:321–330
Jiang L, Zhang H, Cai Z (2008) A novel Bayes model: Hidden naive Bayes. IEEE Trans Knowl Data Eng 21(6):1361–1371
Jiang L, Zhang L, Li C, Wu J (2018) A correlation-based feature weighting filter for naive Bayes. IEEE Trans Knowl Data Eng 31:201–213
Alhussan A, El Hindi K (2016) Selectively fine-tuning Bayesian network learning algorithm. Int J Pattern Recognit Artif Intell 30:165–182
Long Y, Wang L, Sun M (2019) Structure extension of tree-augmented Naive Bayes. Entropy 21(8):721–746
Jiang L, Cai Z, Wang D, Zhang H (2012) Improving tree augmented naive bayes for class probability estimation. Knowl Based Syst 26:239–245
Sahami M (1996) Learning limited dependence Bayesian classifiers. In: Proceedings of the second international conference on knowledge discovery and data mining, vol 96, pp 335–338
Martinez A, Webb G, Chen S, Zaidi N (2016) Scalable learning of Bayesian network classifiers. J Mach Learn Res 17(2):1515–1549
Chen S, Martinez A, Webb G, Wang L (2017) Selective AnDE for large data learning: a low-bias memory constrained approach. Knowl Inf Syst 50(2):475–503
Jiang L, Zhang H, Cai Z, Wang D (2012) Weighted average of one-dependence estimators. J Exp Theor Artif Intell 24(7):219–230
Chen S, Martinez A, Webb G, Wang L (2016) Sample-based attribute selective AnDE for large data. IEEE Trans Knowl Data Eng 29(1):172–185
He H, Han D, Dezert J (2020) Disagreement based semi-supervised learning approaches with belief functions. Knowledge-Based Systems. https://doi.org/10.1016/j.knosys.2019.105426
Blum A, Chawla S (2018) Learning from labeled and unlabeled data using graph mincuts. In: Proceedings of the 18th international conference on machine learning, pp 2143–2161
Zhou D, Hofmann T, Scholkopf B (2004) Semi-supervised learning on directed graphs. In: Proceedings of the Advances in neural information processing systems, pp 1633-1640
Jiang J (2015) Stochastic block model and exploratory analysis in signed networks. Phys Rev E 91:628–645
Chen Y, Wang X, Yuan B, Tang B (2014) Overlapping community detection in networks with positive and negative links. J Stat Mech Theory Exp 3:1272–1284
Yang B, Liu X, Li Y, Zhao X (2017) Stochastic blockmodeling and variational Bayes learning for signed network analysis. IEEE Trans Knowl Data Eng 29:2026–2039
Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the international conference on machine learning, pp 29:2026–2039
Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Proceedings of the advances in neural information processing systems, pp 368–374
Zheng F, Webb G, Suraweera P, Zhu L (2012) Subsumption resolution: an efficient and effective technique for semi-naive Bayesian learning. Mach Learn 87(2):93–125
Zaidi N, Webb G (2013) Fast and effective single pass Bayesian learning. In: Proceedings of the pacific-asia conference on knowledge discovery and data mining, pp 149–160
Yu L, Jiang L, Wang D, Zhang L (2017) Attribute value weighted average of one-dependence estimators. Entropy 19(3): 501
Barutcuoglu Z, Alpaydin E (2003) A comparison of model aggregation methods for regression. In: Proceedings of the artificial neural networks and neural information processing, pp 76–83
Liu Y, Wang L, Mammadov M (2021) Hierarchical Independence Thresholding for learning Bayesian network classifiers. Knowledge-Based Systems 2021(212): 106627
Bache K, Lichman M UCI Machine Learning Repository, Available online: https://archive.ics.uci.edu/ml/datasets.html
Fayyad U, Irani K (1993) Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the Thirteenth international joint conference on artificial intelligence, pp 155–169
Duda R, Hart P, Stork D (2012) Pattern classification,2nd Edition
Kohavi R, Wolpert DH (1996) Bias plus variance decomposition for zero-one loss functions
Brain D, Webb G (1999) On the effect of dataset size on bias and variance in classification learning
Acknowledgements
The authors would like to thank the editor and the anonymous reviewers for their insightful comments and suggestions. And this work was supported by the National Key Research and Development Program of China (No. 2019YFC1804804) and the Scientific and Technological Developing Scheme of Jilin Province (No. 20200201281JC).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
Rights and permissions
About this article
Cite this article
Wang, L., Zhang, X., Li, K. et al. Semi-supervised learning for k-dependence Bayesian classifiers. Appl Intell 52, 3604–3622 (2022). https://doi.org/10.1007/s10489-021-02531-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-02531-y