A new classifying rule using a fuzzy coverage region classifier is introduced in this paper. The rule enables us to formally alter conditional probability distributions to improve the zero-one loss (misclassification rate) of the naive Bayes classifier. Altering the probability distribution is a justifiable variation for defining a fuzzy set from the probability distribution. By using this approach, the range for altering the probability distribution is identified, for example: the value of a distribution function is allowed to replace its value to the power of 1/p, where p is approximately 1 to infinity. Optimizing the parameters of p in each feature and each class to minimize the zero-one loss improves the performance of the fuzzy coverage region classifier (or that of the naive Bayes classifier). Also, it is suggested that the performance of the non-fuzzy coverage region classifier is hardly influenced by the bias of training data, if the training data only covers the range of the class object.