Improving performance of naive bayes classifier by including hidden variables

Abstract

A basic task in many applications is classification of observed instances into a predetermined number of categories or classes. Popular classifiers for probabilistic classification are Bayesian network classifiers and particularly the restricted form known as the Naive Bayes classifier. Naive Bayes performs well in many domains but suffers the limitation that its classification performance cannot improve significantly with an increasing sample size. The expressive power of Naive Bayes is inadequate to capture higher order relationships in the data. This paper presents a method for improving predictive performance of the Naive Bayes classifier by augmenting its structure with additional variables learned from the training data. The resulting classifier retains the advantages of simplicity and efficiency and achieves better predictive performance than Naive Bayes. The approach proposed here can be extended to more general Bayesian network classifiers.