Abstract
Multiple classifier systems (MCSs), or simply classifier ensembles, which combine the outputs of a set of base classifiers, have been recently emerged as a method to develop a more accurate classification system. There are two fundamental issues relating to constructing an ensemble of classifiers. The first one is how to construct a set of the base classifiers in such a way that their ensemble can be a successful one; and the second is how to combine a set of base classifiers. This paper deals with the first important issue of ensemble creation. In the paper, a new method for combining classifiers is proposed. The main idea is heuristic retraining of classifiers. Specifically, in the new method named Combinational Classifiers using Heuristic Retraining (CCHR) which proposes a new way for generating diversity in ensemble pool, a classifier is first run, then, focusing on the drawbacks of this base classifier, other classifiers are retrained heuristically. Experimental results show that the MCSs using the proposed method as the constructor of ensemble components outperform those using those using another method as the constructor of ensemble components in terms of testing accuracy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Daryabari, M., Minaei-Bidgoli, B., Parvin, H.: Localizing Program Logical Errors Using Extraction of Knowledge from Invariants. In: Pardalos, P.M., Rebennack, S. (eds.) SEA 2011. LNCS, vol. 6630, pp. 124–135. Springer, Heidelberg (2011)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. John Wiley & Sons, NY (2001)
Fouladgar, M.H., Minaei-Bidgoli, B., Parvin, H.: On Possibility of Conditional Invariant Detection 6881(2), 214–224 (2011)
Fu, Q., Hu, S.X., Zhao, S.Y.: A PSO-based approach for neural network ensemble. Journal of Zhejiang University (Engineering Science) 38(12), 1596–1600 (2004) (in Chinese)
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Transaction on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Haykin, S.: Neural Networks, a comprehensive foundation. Prentice Hall International (1999)
Krogh, A., Vedelsdy, J.: Neural Network Ensembles Cross Validation, and Active Learning. Advances in Neural Information Processing Systems 7, 231–238 (1995)
Kuncheva, L.I.: Combining Pattern Classifiers, Methods and Algorithms. Wiley, New York (2005)
Lam, L.: Classifier Combinations: Implementations and Theoretical Issues. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 77–86. Springer, Heidelberg (2000)
Lazarevic, A., Obradovic, Z.: Effective pruning of neural network classifier ensembles. In: Proc. International Joint Conference on Neural Networks, vol. 2, pp. 796–801 (2001)
Liu, Y., Yao, X.: Evolutionary ensembles with negative correlation learning. IEEE Trans. Evolutionary Computation 4(4), 380–387 (2000)
Melville, P., Mooney, R.: Constructing Diverse Classifier Ensembles Using Artificial Training Examples. In: Proc. of the IJCAI 2003, pp. 505–510 (2003)
Minaei-Bidgoli, B., Parvin, H., Alinejad-Rokny, H., Alizadeh, H., Punch, W.F.: Effects of resampling method and adaptation on clustering ensemble efficacy (2011) Online
Navone, H.D., Verdes, P.F., Granitto, P.M., Ceccatto, H.A.: Selecting Diverse Members of Neural Network Ensembles. In: Proc. 16th Brazilian Symposium on Neural Networks, pp. 255–260 (2000)
Opitz, D., Shavlik, J.: Actively searching for an effective neural network ensemble. Connection Science 8(3-4), 337–353 (1996)
Parvin, H., Minaei-Bidgoli, B.: Linkage Learning Based on Local Optima. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part I. LNCS, vol. 6922, pp. 163–172. Springer, Heidelberg (2011)
Parvin, H., Helmi, H., Minaei-Bidgoli, B., Alinejad-Rokny, H., Shirgahi, H.: Linkage Learning Based on Differences in Local Optimums of Building Blocks with One Optima. International Journal of the Physical Sciences 6(14), 3419–3425 (2011)
Parvin, H., Minaei-Bidgoli, B., Alizadeh, H.: A New Clustering Algorithm with the Convergence Proof. In: König, A., Dengel, A., Hinkelmann, K., Kise, K., Howlett, R.J., Jain, L.C. (eds.) KES 2011, Part I. LNCS, vol. 6881, pp. 21–31. Springer, Heidelberg (2011)
Parvin, H., Minaei, B., Alizadeh, H., Beigi, A.: A Novel Classifier Ensemble Method Based on Class Weightening in Huge Dataset. In: Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H. (eds.) ISNN 2011, Part II. LNCS, vol. 6676, pp. 144–150. Springer, Heidelberg (2011)
Parvin, H., Minaei-Bidgoli, B., Alizadeh, H.: Detection of Cancer Patients Using an Innovative Method for Learning at Imbalanced Datasets. In: Yao, J., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS, vol. 6954, pp. 376–381. Springer, Heidelberg (2011)
Parvin, H., Minaei-Bidgoli, B., Ghaffarian, H.: An Innovative Feature Selection Using Fuzzy Entropy. In: Liu, D. (ed.) ISNN 2011, Part III. LNCS, vol. 6677, pp. 576–585. Springer, Heidelberg (2011)
Parvin, H., Minaei, B., Parvin, S.: A Metric to Evaluate a Cluster by Eliminating Effect of Complement Cluster. In: Bach, J., Edelkamp, S. (eds.) KI 2011. LNCS, vol. 7006, pp. 246–254. Springer, Heidelberg (2011)
Parvin, H., Minaei-Bidgoli, B., Ghatei, S., Alinejad-Rokny, H.: An Innovative Combination of Particle Swarm Optimization, Learning Automaton and Great Deluge Algorithms for Dynamic Environments. International Journal of the Physical Sciences 6(22), 5121–5127 (2011)
Parvin, H., Minaei, B., Karshenas, H., Beigi, A.: A New N-gram Feature Extraction-Selection Method for Malicious Code. In: Dobnikar, A., Lotrič, U., Šter, B. (eds.) ICANNGA 2011, Part II. LNCS, vol. 6594, pp. 98–107. Springer, Heidelberg (2011)
Qiang, F., Shang-xu, H., Sheng-ying, Z.: Clustering-based selective neural network ensemble. Journal of Zhejiang University Science 6A(5), 387–392 (2005)
Qodmanan, H.R., Nasiri, M., Minaei-Bidgoli, B.: Multi objective association rule mining with genetic algorithm without specifying minimum support and minimum confidence. Expert Systems with Applications 38(1), 288–298 (2011)
Roli, F., Kittler, J. (eds.): MCS 2002. LNCS, vol. 2364. Springer, Heidelberg (2002)
Rosen, B.E.: Ensemble learning using decorrelated neural network. Connection Science 8(3-4), 373–384 (1996)
Schapire, R.E.: The strength of weak learn ability. Machine Learning 5(2), 1971–227 (1990)
Zhou, Z.H., Wu, J.X., Jiang, Y., Chen, S.F.: Genetic algorithm based selective neural network ensemble. In: Proc. 17th International Joint Conference on Artificial Intelligence, vol. 2, pp. 797–802 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Parvin, H., Alizadeh, H., Parvin, S., Maleki, B. (2012). A Heuristic Diversity Production Approach. In: Murgante, B., et al. Computational Science and Its Applications – ICCSA 2012. ICCSA 2012. Lecture Notes in Computer Science, vol 7335. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31137-6_49
Download citation
DOI: https://doi.org/10.1007/978-3-642-31137-6_49
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31136-9
Online ISBN: 978-3-642-31137-6
eBook Packages: Computer ScienceComputer Science (R0)