Abstract
The performance of the Naïve Bayes classifier can be improved by appropriate preprocessing procedures. This paper presents a comparative study of three preprocessing procedures, namely Principle Component Analysis (PCA), Independent Component Analysis (ICA) and class-conditional ICA, for Naïve Bayes classifier. It is found that all the three procedures keep improving the performance of the Naïve Bayes classifier with the increase of the number of attributes. Although class-conditional ICA has been found to be superior to PCA and ICA in most cases, it may not be suitable for the case where the sample size for each class is not large enough.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Langley, P., Iba, W., Thompson, K.: An Analysis of Bayesian Classifiers. In: Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 223–228. AAAI Press, San Jose (1992)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian Network Classifiers. Machine Learning 29, 131–163 (1997)
Sandberg, R., Winberg, G., Bränden, C., Kaske, A., Ernberg, I., Cöster, J.: Capturing Whole-Genome Characteristics in Short Sequences Using a Naïve Bayesian Classifier. Genome Research 11, 1404–1409 (2001)
Rennie, J.D.M., Shih, L., Teevan, J., Karger, D.R.: Tackling the Poor Assumptions of Naïve Bayes Text Classifiers. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the Twentieth International Conference on Machine Learning, pp. 616–623. AAAI Press, Menlo Park (2003)
Li, Y., Anderson-Sprecher, R.: Facies Identification from Well Logs: A Comparison of Discriminant Analysis and Naïve Bayes Classifier. Journal of Petroleum Science and Engineering 53, 149–157 (2006)
Cheng, J., Greiner, R.: Comparing Bayesian Network Classifiers. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI’99), pp. 101–107. Morgan Kaufmann Publishers, Washington (1999)
Gupta, G.K.: Principal Component Analysis and Bayesian Classifier Based on Character Recognition. In: Gary, E., Zhai, Y. (eds.) Bayesian Inference and Maximum Entropy Methods in Science and Engineering. AIP Conference Proceedings, vol. 707, pp. 465–479. Springer, Berlin (2004)
Prasad, M.N., Sowmya, A., Koch, I.: Feature Subset Selection using ICA for Classifying Emphysema in HRCT Images. In: Kittler, J., Petrou, M., Nixon, M.S. (eds.) Proceedings of the 17th International Conference on Pattern Recognition, pp. 515–518. IEEE-CS Press, Washington (2004)
Bressan, M., Vitria, J.: Improving Naïve Bayes Using Class-conditional ICA. In: Garijo, F.J., Riquelme, J.-C., Toro, M. (eds.) IBERAMIA 2002. LNCS (LNAI), vol. 2527, pp. 1–10. Springer, Heidelberg (2002)
Vitria, J., Bressan, M., Radeva, P.: Bayesian Classification of Cork Stoppers Using Class-conditional Independent Component Analysis. IEEE Transactions on Systems, Man and Cybernetics C 37, 32–38 (2007)
Haykin, S.: Neural Netwoks: A Comprehensive Foundation. Prentice Hall, Englewood Cliffs (1999)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis. John Wiley and Sons, New York (2001)
Hyvärinen, A., Oja, E.: Independent Component Analysis: Algorithms and Applications. Neural Networks 13, 411–430 (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fan, L., Poh, K.L. (2007). A Comparative Study of PCA, ICA and Class-Conditional ICA for Naïve Bayes Classifier. In: Sandoval, F., Prieto, A., Cabestany, J., Graña, M. (eds) Computational and Ambient Intelligence. IWANN 2007. Lecture Notes in Computer Science, vol 4507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73007-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-73007-1_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73006-4
Online ISBN: 978-3-540-73007-1
eBook Packages: Computer ScienceComputer Science (R0)