The contributions of this paper are two-fold: firstly, it employs a multi-objective evolutionary hierarchical algorithm to obtain a non-dominated fuzzy rule classifier set with interpretability and diversity preservation. Secondly, a reduce-error based ensemble pruning method is utilized to decrease the size and enhance the accuracy of the combined fuzzy rule classifiers. In this algorithm, each chromosome represents a fuzzy rule classifier and compose of three different types of genes: control, parameter and rule genes. In each evolution iteration, each pair of classifiers in non-dominated solution set with the same multi-objective qualities are examined in terms of Q statistic diversity values. Then, similar classifiers are removed to preserve the diversity of the fuzzy system. Finally, experimental results on the ten UCI benchmark datasets indicate that our approach can maintain a good trade-off among accuracy, interpretability and diversity of fuzzy classifiers.