Skip to main content
SpringerLink
Log in

Landscapes of Naïve Bayes classifiers

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The performance of the Naïve Bayes classifier (NB) is of interest to many researchers. The desire to improve upon the apparent good performance of NB while maintaining its efficiency and simplicity is demonstrated by the variety of adaptations to NB in the literature. This study takes a look at 37 such adaptations. The idea is to give a qualitative overview of the adaptations rather than a quantitative analysis of their performance. Landscapes are produced using Sammon mapping, Principal Component Analysis (PCA) and Self-Organising feature Maps (SOM). Based on these, the methods are split into five main groups—tree structures, feature selection, space transformation, Bayesian networks and joint features. The landscapes can also be used for placing any new variant of NB to obtain its nearest neighbours as an aid for comparison studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. Developed at The Laboratory of Computer and Information Science (CIS), Department of Computer Science and Engineering at the Helsinki University of Technology. http://www.cis.hut.fi/projects/somtoolbox/

References

  1. Aha D (1997) Lazy learning special issue editorial. Artif Intell Rev 11:7–10

    Article  Google Scholar 

  2. Bressan M, Vitria J (2002) Improving Naive Bayes classification using class-conditional ICA. Lect Notes Artif Intell 2527:1–10

    Google Scholar 

  3. Cooper H, Hedges LV (eds) (1994) The handbook of research synthesis. Russell Sage Foundation, New York

  4. Denton A, Perizo W (2004) A kernel-based semi-Naïve Bayesian classifier using P-Trees. In: Proceedings of the SIAM International Conference on Data Mining

  5. Diao L, Hu K, Lu Y, Shi C (2002) A method to boost Naïve Bayesian classifiers. In: Lecture Notes in Computer Science, Proceedings of the 6th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining, pp 115–122

  6. Domingos P, Pazzani M (1997) On the optimality of the simple Bayesian classifier under zero-one loss. Mach Learn 29:103–130

    Article  MATH  Google Scholar 

  7. Duda RO, Hart PE, Stork DG (2001) Pattern classification and scene analysis, 2nd edn. Wiley-Interscience, New York

  8. Freidman N, Geiger D, Goldschmidt M (1997) Bayesian network classifiers. Mach Learn 29(2):131–163

    Article  Google Scholar 

  9. Freund Y (1995) Boosting a weak learning algorithm by majority. Inf Comput 121(2):256–285

    Article  MATH  MathSciNet  Google Scholar 

  10. Freund Y, Schapire RE (1997) A decision theoretic genralization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MATH  MathSciNet  Google Scholar 

  11. Gama J (2003) Iterative Bayes. Theor Comput Sci 292:417–430

    Article  MATH  MathSciNet  Google Scholar 

  12. Huang H, Hsu C (2002) Bayesian classification for data from the same unknown class. IEEE Trans Syst Man Cybern B Cybern 32:137–145

    Article  Google Scholar 

  13. Hunter JE, Schmidt FL (1990) Methods of meta-analysis. Sage publications, Newbury Park

    Google Scholar 

  14. Jain AK, Topchy A, Law MHC, Buhmann JM (2004) Landscape of clustering algorithms. In: Proceedings of the international conference on pattern recognition, ICPR, pp 260–263

  15. James N, Liu K, Bavy N, Li L, Dillion T (2001) An improved Naïve Bayesian classifier technique coupled with a novel input solution method. IEEE Trans Syst Man Cybern C Appl Rev 31(2):249–256

    Article  Google Scholar 

  16. Keogh E, Pazzani M (1999) Learning augmented Bayesian classifiers. A comparison of distribution-based and classification-based approaches. In: Proceedings of the international workshop on artificial intelligence and statistics, pp 225–230

  17. Kleiner A, Sharp B (2000) A new algorithm for learning Bayesian classifiers. In: Proceedings of the 3rd IASTED international conference on artificial intelligence and soft computing, pp 191–197

  18. Kohavi R (1996) Scaling up the accuracy of Naive-Bayes classifiers: a decision-tree hybrid. In: Proceedings of the 2nd international conference on knowledge discovery and data mining, pp 202–207

  19. Kohonen T (1989) Self-organization and associative memory. Springer, Berlin

    Google Scholar 

  20. Kononenko I (1991) Semi-Naïve Bayesian classifier. In: Proceedings of the 6th European Working Session on Learning, pp 206–219

  21. Langley P, Sage S (1994) Induction of selective Bayesian classifiers. In: Proceedings of the 10th conference on UAI, pp 399–406

  22. Lewis D (1998) Naïve Bayes (at forty): the independence assumption in information retrival. In: Proceedings of the 10th European conference on machine learning, pp 4–15

  23. Lipsey M, Wilson DB (2001) Practical meta-analysis (applied social research methods). Sage Publications, London

    Google Scholar 

  24. Ma S, Shi H (2004) Tree augmented Naïve Bayes ensembles. In: Proceedings of the 3rd international conference on machine learning and cybernetics, pp 1497–1502

  25. Manly B (1986) Multivariate statistical methods. a primer. Chapman and Hall, London

    Google Scholar 

  26. Meretakis D, Wüthrich B (1999) Extending Naïve Bayes classifiers using long itemsets. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, pp 165–174

  27. Nürnberger A, Borgelt C, Klose A (1999) Improving Naïve Bayes classifiers using neuro-fuzzy learning. In: Proceedings of the 6th international conference on neural image processing, pp 154–159

  28. Pazzani MJ (1996) Searching for dependencies in Bayesian classifiers. In: Proceedings of information, statistics and induction in science

  29. Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Fransisco

    Google Scholar 

  30. Phillips PJ, Newton EM (2002) Meta-analysis of face recognition algorithms. In: Proceedings of the 5th IEEE international conference on automatics face and gesture recognition, FGR’02, pp 235–241

  31. Ratanamahatana C, Gunopulos D (2002) Scaling up the Naïve Bayesian classifier: using decision trees for feature selection. In: Proceedings of the Workshop on Data Cleaning and Pre-processing, ICDM’02

  32. Ridgeway G, Madigan D, Richardson T, O’Kane J (1998) Interpretable boosted Naïve Bayes classification. In: Proceedings of the 4th international conference on knowledge discovery and data mining, pp 101–104

  33. Robles V, Larranaga P, Menasalvas E, Perez MS, Herves V (2003) Improvement of Naïve Bayes collaborative filtering using interval estimation. In: Proceedings of the IEEE WIC international conference on web intelligence, pp 168–174

  34. Rosell B, Hellerstein L (2004) Naïve Bayes with higher order attributes. In: Lecture Notes in Computer Science, Proceedings of the 17th Conference of the Canadian Society for Computational Studies of Intelligence, pp 105–119

  35. Sahami M (1996) Learning limited dependence Bayesian classifiers. In: Proceedings of the 2nd international conference on knowledge discovery and data mining, pp 334–338

  36. Sammon JW (1969) A non-linear mapping for data structure analysis. IEEE Trans Comput 18:401–405

    Article  Google Scholar 

  37. Schapire RE (1990) The strength of weak learnability. Mach Learn 5(2):197–227

    Google Scholar 

  38. Schapire RE (1999) A brief introduction to boosting. In: Proceedings of the 16th international joint conference on artificial intelligence, pp 1401–1406

  39. Scheiner SM, Gurevitch J (eds) (1993) Design and analysis of ecological experiments. Chapman and Hall, London

  40. Schiffman S, Reynolds ML, Young FW (1981) Introduction to multidimensional scaling. Academic, London

    MATH  Google Scholar 

  41. Sohn SY (1999) Meta-analysis of classification algorithms for pattern recognition. IEEE Trans Pattern Anal Mach Intell 21(11):1137–1144

    Article  Google Scholar 

  42. Störr HP (2002) A compact fuzzy extension of the Naïve Bayesian classification algorithm. In: Proceedings of InTech VJFuzzy 2002, pp 172–177

  43. Ting K, Zheng Z (1999) Improving the performance of boosting for Naïve Bayesian classification. In: Proceedings of the 3rd Pacific-Asia conference on knowledge discovery and data mining, pp 296–305

  44. Tsymbal A, Cunningham P, Pechenizkiy M, Puuronen S (2003) Search strategies for ensemble feature selection in medical diagnostics. Technical report, Trinity College Dublin, Ireland

  45. Tsymbal A, Puuronen S (2002) Ensemble feature selection with the simple Bayesian classification in medical diagnostics. In: Proceedings of the 15th IEEE symposium on computer based medical systems, pp 225–230

  46. Vilalta R, Rish I (2003) A decomposition of classes via clustering to explain and improve Naïve Bayes. In: Proceedings of the 14th European conference on machine learning

  47. Wang L, Yuan S, Li H (2004) Boosting Naïve Bayes by active learning. In: Proceedings of the 3rd international conference on machine learning and cybernetics, pp 1383–1386

  48. Wang Z, Webb GI (2002) Comparison of lazy Bayesian rule and tree-augmented Bayesian learning. In: Proceedings of IEEE international conference on data mining, pp 490–497

  49. Webb G, Boughton J, Wang Z (2005) Not so Naïve Bayes: aggregating one dependence estimators. Mach Learn 58(1):5–24

    Article  MATH  Google Scholar 

  50. Webb G, Pazzani MJ (1998) Adjusted probability Naïve Bayesian induction. In: Proceedings of the 11th Australian conference on artificial intelligence, pp 285–295

  51. Wolf FM (1986) Meta-analysis: quantitative methods for research synthesis. Sage University Paper no. 59. Series on Quantitative Applications in the Social Sciences. Sage publications, London

  52. Zhang H, Su J (2004) Conditional independence trees. Lect Notes Comput Sci 3201:513–524

    Article  Google Scholar 

  53. Zheng Z, Webb GI (2000) Lazy learning of Bayesian rules. Mach Learn 41(1):53–84

    Article  Google Scholar 

  54. Zhipeng X, Hsu W, Liu Z, Lee M (2002) SNNB: a selective neighbourhood based Naïve Bayes for lazy learning. In: Proceedings of advances in knowledge discovery and data mining. PAKDD, pp 104–114

  55. Zheng Z (1998) Naïve Bayesian classifier committees. In: Proceedings of the European conference on machine learning, pp 196–207

Download references

Author information

Authors and Affiliations

  1. School of Informatics, University of Wales Bangor, Bangor, UK

    Zoë Hoare

Authors
  1. Zoë Hoare
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zoë Hoare.

Data matrix

Data matrix

Table 4 Data matrix encoding each of the 38 methods using the 19 selected characteristics
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hoare, Z. Landscapes of Naïve Bayes classifiers. Pattern Anal Applic 11, 59–72 (2008). https://doi.org/10.1007/s10044-007-0079-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-007-0079-5

Keywords

Search

Navigation