Abstract
Naive Bayesian classifiers which make independence assumptions perform remarkably well on some data sets but poorly on others. We explore ways to improve the Bayesian classifier by searching for dependencies among attributes. We propose and evaluate two algorithms for detecting dependencies among attributes and show that the backward sequential elimination and joining algorithm provides the most improvement over the naive Bayesian classifier. The domains on which the most improvement occurs are those domains on which the naive Bayesian classifier is significantly less accurate than a decision tree learner. This suggests that the attributes used in some common databases are not independent conditioned on the class and that the violations of the independence assumption that affect the accuracy of the classifier can be detected from training data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Almuallim, H., and Dietterich, T. G. (1991). Learning with many irrelevant features. In Ninth National Conference on Artificial Intelligence, 547–552. MIT Press.
Caruana, R., and Freitag, D. (1994). Greedy attribute selection. In Cohen, W., and Hirsh, H., eds., Machine Learning: Proceedings of the Eleventh International Conference. Morgan Kaufmann
Cooper, G., and Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9, 309–347.
Danyluk, A., and Provost, F. (1993). Small disjuncts in action: Learning to diagnose errors in the telephone network local loop. Machine Learning Conference, pp 81–88.
Duda, R., and Hart, P. (1973). Pattern classification and scene analysis. New York: John Wiley & Sons.
John, G. Kohavi, R., and Pfleger, K. (1994). Irrelevant Features and the subset selection problem Proceedings of the Eleventh International Conference on Machine Learning. New Brunswick, NJ.
Kittler, J. (1986). Feature selection and extraction. In Young & Fu, (eds.), Handbook of pattern recognition and image processing. New York: Academic Press.
Kononenko, I. (1990). Comparison of inductive and naive Bayesian learning approaches to automatic knowledge acquisition. In B. Wielinga (Eds..), Current trends in knowledge acquisition. Amsterdam: IOS Press.
Kononenko, I. (1991). Semi-naive Bayesian classifier. Proceedings of the Sixth European Working Session on Learning. (pp. 206–219). Porto, Portugal: Pittman.
Langley, P. (1993). Induction of recursive Bayesian classifiers. Proceedings of the 1993 European Conference on Machine Learning. (pp. 153–164). Vienna: Springer-Verlag.
Langley, P., and Sage, S. (1994). Induction of selective Bayesian classifiers. Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence. Seattle, WA
Moore, A. W., and Lee, M. S. (1994). Efficient algorithms for minimizing cross validation error. In Cohen, W. W., and Hirsh, H., eds., Machine Learning: Proceedings of the Eleventh International Conference. Morgan Kaufmann.
Murphy, P. M., and Aha, D. W. (1995). UCI Repository of machine learning databases. Irvine: University of California, Department of Information & Computer Science. Machine-readable data repository ftp://ics.uci.edu:/pub/machine-learning-databases.
Pazzani, M., Merz, C., Murphy, P., Ali, K., Hume, T., and Brunk, C. (1994). Reducing Misclassification Costs. Proceedings of the Eleventh International Conference on Machine Learning. New Brunswick, NJ.
Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo, CA: Morgan Kaufmann.
Quinlan, J.R. (1986). Induction of decision trees. Machine Learning, 1, 81–106.
Rachlin, Kasif, Salzberg, and Aha, (1994). Towards a better understanding of memory-based reasoning systems. Proceedings of the Eleventh International Conference on Machine Learning. New Brunswick, NJ.
Ragavan, H., and Rendell, L. (1993). Lookahead feature construction for learning hard concepts. Machine Learning: Proceedings of the Tenth International Conference. Morgan Kaufmann
Schlimmer, J. (1987). Incremental adjustment of representations for learning. Machine Learning: Proceedings of the Fourth International Workshop. Morgan Kaufmann
Schaffer, C. (1994). A conservation law of generalization performance. Proceedings of the Eleventh International Conference on Machine Learning. New Brunswick, NJ.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1996 Springer-Verlag New York, Inc.
About this chapter
Cite this chapter
Pazzani, M.J. (1996). Searching for Dependencies in Bayesian Classifiers. In: Fisher, D., Lenz, HJ. (eds) Learning from Data. Lecture Notes in Statistics, vol 112. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-2404-4_23
Download citation
DOI: https://doi.org/10.1007/978-1-4612-2404-4_23
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-94736-5
Online ISBN: 978-1-4612-2404-4
eBook Packages: Springer Book Archive