Abstract
Averaged One-Dependence Estimators [1], simply AODE, is a recently proposed algorithm which weakens the attribute independence assumption of naïve Bayes by averaging all the probability estimates of a collection of one-dependence estimators and demonstrates significantly high classification accuracy. In this paper, we study the selective AODE problem and proposed a Cross-Entropy based method to search the optimal subset over the whole one-dependence estimators. We experimentally test our algorithm in term of classification accuracy, using the 36 UCI data sets recommended by Weka, and compare it to C4.5[5], naïve Bayes, CL-TAN[6], HNB[7], AODE and LAODE[3]. The experiment results show that our method significantly outperforms all the other algorithms used to compare, and remarkably reduces the number of one-dependence estimators used compared to AODE.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Webb, G.I., Boughton, J., Wang, Z.: Not so naïve bayes: Aggregating one-dependence estimators. Machine Learning 58, 5–24 (2005)
Yang, Y., Webb, G.I., et al.: To Select or To Weight: A Comparative Study of Model Selection and Model Weighting for SPODE Ensembles. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 170–181. Springer, Heidelberg (2006)
Zhang, F., Webb, G.I.: Efficient lazy elimination for averaged one-dependence estimators. In: Proceedings of 23rd International conference on Machine Learning (ICML) (2006)
Cerquides, J., Mantaras, R.L.D.: Robust Bayesian linear classifier ensembles. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 70–81. Springer, Heidelberg (2005)
Quinlan, J.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo (1993)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29, 131–163 (1997)
Zhang, H., Jiang, L.X., Su, J.: Hidden Naive Bayes. In: Proceeding of 20th National conference on Artificial Intelligence (AAAI), pp. 919–924 (2005)
Jiang, L.X., Zhang, H.: Weighted Averaged One-Dependence Estimators. In: Yang, Q., Webb, G. (eds.) PRICAI 2006. LNCS (LNAI), vol. 4099, pp. 970–974. Springer, Heidelberg (2006)
Chickering, D.M.: Learning Bayesian networks is NP-Complete. In: Fisher, D., Lenz, H. (eds.) Learning from Data: Artificial Intelligence and Statistics, pp. 121–130 (1996)
The Cross-Entropy Method, http://iew3.technion.ac.il/CE/about.php
Rubinstein, R.Y, Kroese, D.P (eds.): The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer, New York (2004)
De Boer, P-T., Kroese, D.P, Mannor, S., Rubinstein, R.Y.A: Tutorial on the Cross-Entropy Method. Annals of Operations Research. 134, 19–67 (2005)
Blake, C., Merz, C.J.: UCI repository of machine learning databases. In: Department of ICS, University of California, Irvine, http://www.ics.uci.edu/~mlearn/MLRepository.html.
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Technology with Java Implementation. Morgan Kaufmann, San Francisco (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Q., Zhao, Bh. (2007). Discriminatively Learning Selective Averaged One-Dependence Estimators Based on Cross-Entropy Method. In: Wang, Y., Cheung, Ym., Liu, H. (eds) Computational Intelligence and Security. CIS 2006. Lecture Notes in Computer Science(), vol 4456. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74377-4_95
Download citation
DOI: https://doi.org/10.1007/978-3-540-74377-4_95
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74376-7
Online ISBN: 978-3-540-74377-4
eBook Packages: Computer ScienceComputer Science (R0)